Overview of PrivateS3 (AWS)
PrivateS3 FAQ (AWS)
What is Aviatrix PrivateS3?
Aviatrix PrivateS3 is a feature that allows you to leverage AWS Direct Connect to transfer objects and files between on-prem and S3, while giving you control of the S3 buckets so that you can whitelist them.
How does PrivateS3 work?
PrivateS3 works as follows.
-
The on-prem customer resolves all S3 bucket names under management to the private IP address of the Aviatrix gateway created and managed in the AWS internal NLB.
-
Every 30 minutes the Controller scans S3 buckets in the selected region and accounts.
-
The Controller sends email notifications to the admin for newly discovered S3 buckets. All S3 buckets are denied access by default.
-
The admin logs into the Controller to approve or deny access to the discovered S3 buckets.
-
When an Aviatrix PrivateS3 gateway receives the packets, it uses its FQDN feature to filter out any bucket names that are not on the allowed list, thus preventing data leakage.
What is the security exposure when uploading files to AWS S3 over Direct Connect?
If you leverage the high-speed AWS Direct Connect to transfer files and objects to/from S3, the current solution is to use public VIF where AWS advertises the entire S3 public address ranges to on-prem. This implies that all on-prem users can upload to any S3 bucket, including to their personal S3 buckets on their own personal accounts, leading to confidential data leakage. The current solution is described as below.
In the diagram above, there is no VPC involved when using public VIF. Data is directly transferred to and from S3 riding on the Direct Connect link.
In another scenario where an instance in a VPC is trying to access S3 buckets, you can specify an S3 private Endpoint. The advantage is such that packets do not get routed over the Internet and instead packets are routed to S3 via the AWS network. However, the endpoint is still represented by the public CIDR blocks representing AWS S3 in the region as shown below. In other words, someone with valid credentials to access the S3 Endpoint can transfer objects to their own S3 buckets.
| An Endpoint policy controls who can use the Endpoint service, but it does not control which destination S3 bucket is granted to the request. |
The same data leakage issue occurs if you upload files to S3 over public Internet.
The benefits of PrivateS3?
The key benefits are:
-
Transferring objects/data between on-prem and S3 by leveraging Direct Connect without using public VIF.
-
The ability to control which S3 buckets can be accessed.
-
The ability to deploy multiple Aviatrix Gateways to load balance the data traffic.
Can PrivateS3 work for traffic initiated from a VPC?
PrivateS3 is optimized for managing S3 access from on-prem. For traffic initiated from a VPC, use the Aviatrix FQDN feature for not only S3 access control but also all Internet-bound egress control.
Is there an additional AWS data charge by going through the Aviatrix Gateway?
No, there is no data charge by AWS for using PrivateS3. Normally AWS charges data transfer for data traffic leaving a VPC, however in this case, data transfer is through an AWS VPC endpoint to S3 which is free of charge.
Can PrivateS3 be deployed in a TGW environment?
Yes. You can deploy PrivateS3 in a Spoke VPC in the TGW environment, as shown in the diagram below.
Can the Direct Connect termination VPC be in a different region of managed S3 buckets?
Yes. For example, the Direct Connect private VIF terminates in a VPC in us-west-2 and your S3 buckets are in us-east-1. You should launch the PrivateS3 gateway in a VPC in us-east-1 and make sure there is private connectivity to this VPC from on-prem.
Can a PrivateS3 gateway be in a different region of managed S3 buckets?
Yes. However, in such a case, you will not be able to leverage the S3 Gateway Endpoint service to route packets to S3 within AWS network. PrivateS3 will forward traffic to public Internet to reach S3 in a different region.
Scale out PrivateS3 Solution
You can launch multiple PrivateS3 gateways in a multi-AZ fashion in a VPC. Aviatrix Controller automatically creates and manages the AWS internal NLB to load balance the S3 access requests.
PrivateS3 Testing
There is a simple method to simulate DNS resolution to the PrivateS3 internal NLB.
Launch a Linux instance or host, in sudo mode, and edit file /etc/hosts. Add S3 bucket FQDN names to this file, as shown in the example below, where 172.32.1.212 is the PrivateS3 NLB IP address. This IP address can be found here.
You can then run an AWS CLI command, such as "aws s3 ls" to see the list of S3 buckets on the Access Account in the region where a PrivateS3 gateway is launched.
Below is another example of uploading a file to S3 using AWS CLI.
ubuntu@ip-172-32-1-144:~$ aws s3 cp init-cfg.txt.3 s3://sxw-new-bucket-2 upload: ./init-cfg.txt.3 to s3://sxw-new-bucket-2/init-cfg.txt.3
To test on a Windows machine, you modify the file at c:WindowsSystem32Driversetchosts. An example instruction is shown here.
Troubleshoot PrivateS3
PrivateS3 combines the FQDN feature and the Stateful Firewall feature.
-
Go to Security > Egress Control > Egress FQDN Filter. A tag should have been automatically created. Click Edit to see if the desired S3 bucket name is configured.
-
Go to Gateway, select one PrivateS3 gateway, and click Edit.
-
Scroll down to Destination NAT to make sure the DNAT rule is configured.
Does AWS S3 list command work?
Yes. AWS S3 CLI "list" command requires s3.region.amazonaws.com in the bucket rule where the region is represented. This is automatically populated by the Controller.