Getting Started with SaaS Shield CMK for Amazon S3

This guide will show you how to test for compatibility between your application and the SaaS Shield for Amazon S3 product, and how to get the product installed in your AWS environment for testing.

Checking Client Compatibility

We spent a lot of time ensuring that the S3 proxy's interface was compatible with the Amazon S3 interface. But this is software, and you know what that is like. Your first step in assessing whether SaaS Shield for Amazon S3 will work with your application is to make sure that our interface is compatible with your code.
We have set up a global instance of the product to allow you to do some quick tests for compatibility. We are running SaaS Shield CMK for Amazon S3 in our AWS infrastructure, using a set of users and corresponding buckets that we assign to people to do testing. Just contact us to request access, and we will send you the credentials (access key and secret key) and bucket name you can use to test your S3 interface code.
The URL of the global instance is https://saas-shield-for-s3-demo.ironcorelabs.com, and the buckets are all in the AWS region us-east-1. Inside any of the buckets, if an object name starts with 1, it will be mapped to TENANT1, and if the name starts with 2, it will be mapped to TENANT2. Otherwise, it will not be encrypted.
If you are using one of the AWS SDKs to interact with S3, you will need to update the way you initialize the S3 client in your app. You need to configure the non-AWS endpoint, and you must enable path-style access, if you aren't already using that mode. Also note that the S3 proxy endpoint is configured to require SSL, so make sure you prefix the endpoint URL with https:// or set sslEnabled.
For example, if you are using the AWS SDK for Java, you can do
AmazonS3 s3 = AmazonS3ClientBuilder
    .standard()
    .withEndpointConfiguration(new EndpointConfiguration("https://saas-shield-for-s3-demo.ironcorelabs.com", "us-east-1"))
    .enablePathStyleAccess()
    .build();
Similarly, in Javascript, you can do
var s3 = new AWS.S3(
    { endpoint: 'saas-shield-for-s3-demo.ironcorelabs.com',
      region: 'us-east-1',
      sslEnabled: true,
      s3ForcePathStyle: true
    });
Alternatively, you can use a configuration object to set the S3 client parameters:
var config = {                                
    'endpoint' : 'saas-shield-for-s3-demo.ironcorelabs.com',
    'region' : 'us-east-1',
    'sslEnabled': true,
    's3ForcePathStyle': true
};
AWS.config.update(config);
s3 = new AWS.S3();
You will need to configure your S3 client with this endpoint and region, enable SSL and path-style access, and specify the provided credentials (access key ID and secret access key). You are then ready to run through any of the operations you do against S3 using the provided bucket.
WARNING: we clean all data out of each of these S3 buckets nightly. They are only intended for compatibility testing. Do not place any data in there that you don't want discarded!

Initial Installation

Once you have determined that your S3 client software is compatible with the S3 proxy, you are ready to install SaaS Shield for Amazon S3 into your AWS infrastructure and set up a trial system to do some deeper testing. Just initiate the signup process here, and we will email you a link that will point to an account creation page in the Configuration Broker, where you can enter the information to create an administrator account and the record for your vendor.
Once you have entered that data and have an account, create a new Tenant Security Proxy (TSP) configuration. After you have created the TSP configuration, click the button to download a pair of CloudFormation template files. These templates are used to install and configure SaaS Shield CMK for Amazon S3 in AWS. The first file, the parameter template, contains a set of configuration values that will be stored in AWS using the Parameter Store and the Secure Secret Manager. The second file, the deployment template, installs the S3 Proxy and TSP containers in AWS, sets up networking, load balancer, and other infrastructure, creates a test user, and creates a test S3 bucket that the test user can access. Save these two files before you continue.

DNS Setup

In order to use the S3 Proxy, you need to be able to connect to it using your S3 client. This is much easier to configure if you have a DNS name for the endpoint. In addition, to communicate over an SSL connection, the endpoint needs a certificate for that DNS name. AWS will create a a DNS Zone using Route 53 that is associated with your chosen domain, and it will generate a certificate, but you will need to add records to the DNS provider for the domain you are using. Make sure you have access to the DNS provider or can get assistance from someone that does have access.
  1. Delegate the zone, allowing your root DNS provider to direct requests for the subdomain to AWS.
    1. In the AWS console, looking at the new DNS zone, there should be an NS record. Copy the hostnames from that NS record.
    2. Go to your root DNS provider and add a corresponding NS record for each of these hostnames there.
  2. Record the DNS Zone ID and FQDN (Fully Qualified Domain Name) for later. You will need them when you load the deployment template.

Installing the Product

Make sure you have the two template files that you downloaded from the Configuration Broker, along with the DNS Zone ID and FQDN. To install and configure the product in AWS, follow these steps:
  1. Log into the AWS console.
  2. Select the AWS region where you want to install the product.
  3. Go to CloudFormation in the AWS Console.
  4. Decide what you want to name the CloudFormation stack that will run the product - let's call that StackName.
  5. Create the parameters stack.
    1. Select Create stack.
    2. Select Template is ready, Upload a template file, Choose file. Select the parameters template you downloaded from the Config Broker.
    3. Select Next.
    4. Name the stack. We strongly suggest naming it ${StackName}-params to aid in matching it to the deployment template.
    5. Review the parameters. We recommend leaving them all unchanged.
    6. Select Next, then Next.
    7. Select Create stack.
    8. Wait until the log shows that the stack creation has completed. This should not take long.
  6. Create the deployment stack.
    1. Follow the same steps you used to create the parameters stack, but load the deployment template. You will need to provide a new name for this stack - use the StackName you chose earlier.
    2. CloudFormation will prompt you to enter values for some parameters. The first one to enter is the ParameterStackName - you must enter the name you used when you created the parameters stack.
    3. The S3DNSZoneID parameter value is a drop-down - you should select the ID of the zone you created when you were setting up DNS.
    4. The S3DNSName is the FQDN of the zone you created.
    5. On the last step, acknowledge that the template creates IAM resources (it is creating a user access the test S3 bucket).
As CloudFormation creates the resources in the stack, you can select the Events tab and refresh periodically to watch its progress as it creates resources. The deployment will get to a point where it is blocked trying to create the S3Certificate. This should happen after it has completed creation of the S3DNSBaseEntry and S3DNSWildcardEntry. In order to complete creation of the certificate, AWS is verifying that you control the domain for which it is generating the certificate. This can happen automatically, but you need to add the records to your DNS provider for the check to succeed. Follow these steps:
  1. Select the Resources tab and find the entry with Logical ID S3DNSZone. The Physical ID is a link that will take you to the appropriate DNS Zone entry in Route 56. Navigate there; you should see a list of DNS records. Find the record with type NS - the Value for the record contains four different host names (one for each of the top level domains .com, .org, .net, and .co.uk).
  2. Go to your root DNS provider and add a corresponding NS record for the domain name you are creating, with one of these four hostnames.
After you have added the NS records, CloudFormation will be able to complete validation of the certificate, and it will proceed with the deployment. When the deployment completes successfully, CloudFormation has done the following:
  1. Create a stack that manages several secrets (using the Secure Secret Manager). These entries are all named with a /saas_shield_s3/${ParameterStackName}/ prefix.
  2. Create a second stack that manages a number of different resources necessary to run the product, including the following:
    1. An instance of the S3 Proxy running in Elastic Container Service (ECS)
    2. An instance of the Tenant Security Proxy (TSP) running in ECS
    3. A test bucket named ironcore-test-${DomainName}, where DomainName is the FQDN you provided, with the periods replaced by underscores
    4. A test user named ${StackName}-S3User- followed by a random suffix. This user has permission to access the test bucket, and the credentials for this user are the ones that you will need to send requests to the S3 proxy
    5. A DNS Zone for your selected domain name, and a certificate for that domain and all subdomains of it
    6. Additional network services, load balancers, and supporting infrastructure to run the product
    7. A CloudWatch log group named ${Stackname}
    8. An additional secret, also prefixed by /saas-shield-s3/${ParameterStackName}/, for the proxy/s3-credentials
After the stack has been created, you need to retrieve the values of the credentials for the test user that was created, so you can supply them to your S3 clients. To do this, go back to the Resources tab of the new deployment stack, find the entry with Logical ID S3CredsSecret and click on the link under Physical ID. This will take you to the entry in the Secrets Manager. Find the Secret value section and click Retrieve secret value. This will display the access key ID and the secret key that you need to provide to your S3 client.

Creating Tenants

The S3 proxy is now ready to start processing requests. However, it is using a default tenant mapping file that does not contain any mappings, so it will not actually encrypt any objects uploaded to S3. You are now ready to create a tenant. Return to the Configuration Broker, or log in again, select Tenants, and add a new tenant. Assign a name and a tenant ID, and provide an email address for the tenant administrator. When you enter the data and click Invite, the Configuration Broker will send an email to that address that contains a sign-up link. Clicking on the link will take the user to a page on the Configuration Broker where they can enter the information for the tenant. Once this is done, the admin can add a new KMS Configuration. If you are setting up the initial test system, we suggest that you create a key in your AWS KMS - just create an AWS KMS configuration and follow the instructions.
Once the tenant is created and has an active primary KMS configuration, you can use that tenant to encrypt data. If you go to the test bucket in S3 (ironcore-test-${DomainName}), you will find an object named tenant-mapping.conf. This is the mapping file that specifies which tenant controls the encryption key for each object written to the S3 proxy. The file is HOCON, a JSON superset, with the following form:
mapping = [
   { object-regex = {BUCKET_AND_KEY_REGEX}, tenant-id = {TENANT_ID} },
]
The initial mapping file has no elements in the mapping array - you will need to add a mapping entry for your test bucket. After you have created the first tenant, you can download tenant-mapping.conf from S3 and edit it; it should look something like this, if your FQDN is tryit.foo.com and you added a tenant with ID TENANT1:
mapping = [
   { object-regex = "ironcore-labs-tryit-foo-com/.*", tenant-id = "TENANT1" }
]
This specifies that everything written to the test bucket should be encrypted using TENANT1's KMS.
There are more details on the mapping file and on the internals of the S3 proxy in the How It Works section.

Update Your App

The S3 proxy is now ready to go. You should just update the configuration for your S3 client library much like you did when you went through the steps in the Checking Client Compatibility section. Provide the access key ID and secret access key, add an endpoint using the FQDN you created, enable SSL, set the client to use path-style addressing, and the client should be configured to use the new S3 proxy. If you write to the test bucket, everything should be encrypted, and the encryption key is wrapped using the KMS you set up in your AWS account.
If you write an object to the bucket, you can examine it in the S3 console and see that it is encrypted, and that the tenant ID and wrapped encryption key are stored in tags on the object. If you get the object from S3 using the proxy, it will be decrypted automatically.

Advanced Installation

We have information on our Advanced Installation page about how to install the product in additional regions, or to install a version for development, QA, or production.

Need Help?

If you encounter issues when installing or running the product, the first troubleshooting step is to check the logs that are written to CloudWatch in your AWS account. Just find the log group named ${StackName}, then examine the s3-proxy/s3-proxy/* log stream. This is the log from the S3 Proxy container. This will often provide information about the cause of an issue.
If you need further assistance or have questions, please email saas.shield.support@ironcorelabs.com.

Products

Documentation

Trust Center

Find Us