Amazon SageMaker with guardrails on the AWS Cloud

Quick Start Reference Deployment

DRAFT DOCUMENT / UNOFFICIAL GUIDANCE

This portion of the deployment guide is located at docs/partner_editable/__settings.adoc_

January 2021
Deepak Behera and Girish Chandra Tejaswi S., Brillio
Tony Bulding, AWS Quick Start team

Visit our GitHub repository for source files and to post feedback, report bugs, or submit feature ideas for this Quick Start.

This Quick Start was created by Brillio in collaboration with Amazon Web Services (AWS). Quick Starts are automated reference deployments that use AWS CloudFormation templates to deploy key technologies on AWS, following AWS best practices.

Overview

This portion of the deployment guide is located at docs/partner_editable/overview_target_and_usage.adoc

This Quick Start deployment guide provides step-by-step instructions for deploying Amazon SageMaker with guardrails on the AWS Cloud. This deployment is for users who want to use the capabilities of SageMaker with guardrails enabled for added security.

This deployment uses security guardrails on the SageMaker environment so that customers can build, train, and deploy machine learning (ML) models in a more secure environment. It uses enhanced security by using AWS PrivateLink, Amazon CloudWatch, AWS Identity and Access Management (IAM), AWS Key Management Service (AWS KMS), and other native services on AWS.

SageMaker with guardrails provides the following features:

A private network for performing secure API calls to other AWS services and restricting internet access for downloading packages.
Restricted SageMaker access to Amazon Elastic Container Registry (Amazon ECR).
Mandatory tagging for implementing resource policies and compliance when creating users and resources.
S3 bucket policies that restrict access to specific VPC endpoints.
Encryption of ML model artifacts and other system artifacts that are either in transit or at rest. Requests to the SageMaker API and console are made over a Secure Sockets Layer (SSL) connection.
Disabled root access to the SageMaker notebook instance at the time of launch.
Restricted IAM roles and policies for SageMaker execution and notebook access based on resource tags and project ID. Users can only open, start, and stop their own SageMaker notebooks.

Amazon may share user-deployment information with the AWS Partner that collaborated with AWS on the Quick Start.

Amazon SageMaker with guardrails on AWS

This portion of the deployment guide is located at docs/partner_editable/product_description.adoc

Amazon SageMaker provides an automated approach for various ML workflows. Users can manually provision SageMaker notebooks directly through the SageMaker console and create the associated S3 buckets to use as a data store for training models and SageMaker model artifacts.

Although SageMaker’s self-service provisioning capabilities are convenient for project teams, this model can result in limited security options when running SageMaker in an isolated environment. Security risks exist from downloading packages over the internet, managing training models from public endpoints, and accessing model-building data from an S3 bucket. Also, in its native form, SageMaker provides no options for storing shared resources using Amazon Elastic File System (Amazon EFS). Setting up EFS for storage is a manual process.

You can address these security concerns and improve the experience accessing a SageMaker notebook by using Brillio’s implementation of SageMaker. It provides guardrails for incorporating security mechanisms and add-on features that are not provided with SageMaker. These guardrails deploy within an AWS-managed virtual private cloud (VPC) and elastic network interfaces. Also, when using Brillio’s SageMaker product, you can provide more secure access to AWS services with VPC endpoint interfaces and S3 bucket gateways within the customer’s own VPC.

AWS costs

You are responsible for the cost of the AWS services and any paid third-party licenses used while running this Quick Start. There is no additional cost for using the Quick Start.

The AWS CloudFormation templates for Quick Starts include configuration parameters that you can customize. Some of the settings, such as the instance type, affect the cost of deployment. For cost estimates, see the pricing pages for each AWS service you use. Prices are subject to change.

After you deploy the Quick Start, create AWS Cost and Usage Reports to deliver billing metrics to an Amazon Simple Storage Service (Amazon S3) bucket in your account. These reports provide cost estimates based on usage throughout each month and aggregate the data at the end of the month. For more information, see What are AWS Cost and Usage Reports?

Software licenses

This portion of the deployment guide is located at docs/partner_editable/licenses.adoc

This Quick Start uses native AWS services. No additional licenses are required.

Architecture

This portion of the deployment guide is located at docs/partner_editable/architecture.adoc

Deploying this Quick Start for a new virtual private cloud (VPC) with default parameters builds the following SageMaker with guardrails environment in the AWS Cloud.

Figure 1. Quick Start architecture for Amazon SageMaker with guardrails on AWS

As shown in Figure 1, the Quick Start sets up the following:

AWS Lambda function (SageMakerBuild) for validating the VPC Domain Name System (DNS) and provisioning SageMaker resources.
AWS Service Catalog for triggering the SageMakerBuild function and passing parameters for creating resources.
AWS Identity and Access Management (IAM) roles, including:
- User role for accessing and launching the Service Catalog.
- Service Catalog launch constraint role for providing permission to provision resources.
- SageMaker execution role for providing limited access to the SageMaker notebook as determined by policies.
In the private resource subnet:
- Amazon SageMaker for running ML models and workflow.
- Amazon Elastic File System (Amazon EFS) for sharing common modules to SageMaker notebooks.
In the private Elastic Network Interface (ENI) subnet, interface endpoints through which SageMaker communicates with the following AWS services:
- Amazon CloudWatch for real-time monitoring of the SageMaker environment.
- Amazon Elastic Container Registry (Amazon ECR) with ECR Policy for storing the latest ML model images for future deployments.
- AWS Security Token Service (AWS STS) for providing access to an IAM role to perform operations on other AWS services.
Amazon Simple Storage Service (Amazon S3) gateway endpoint to access the S3 bucket for storing and retrieving ML data and bucket policy for restricting bucket access.
A dedicated S3 bucket used as a data store for training models and SageMaker model artifacts.
AWS PrivateLink, Amazon CloudWatch, AWS IAM, AWS Key Management Service (AWS KMS), and other native services on AWS to provide enhanced security.

Planning the deployment

Specialized knowledge

This deployment requires a moderate level of familiarity with AWS services. If you’re new to AWS, see Getting Started Resource Center and AWS Training and Certification. These sites provide materials for learning how to design, deploy, and operate your infrastructure and applications on the AWS Cloud.

This portion of the deployment guide is located at docs/partner_editable/specialized_knowledge.adoc

The following information is available to help you become familiar with the AWS services that are used in this Quick Start:

AWS account

If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Part of the sign-up process involves receiving a phone call and entering a PIN using the phone keypad.

Your AWS account is automatically signed up for all AWS services. You are charged only for the services you use.

Technical requirements

Before you launch the Quick Start, review the following information and ensure that your account is properly configured. Otherwise, deployment might fail.

Resource quotas

If necessary, request service quota increases for the following resources. You might request quota increases to avoid exceeding the default limits for any resources that are shared across multiple deployments. The Service Quotas console displays your usage and quotas for some aspects of some services. For more information, see What is Service Quotas? and AWS service quotas.

This portion of the deployment guide is located at docs/partner_editable/service_limits.adoc

Resource	This deployment uses
VPCs	1
VPC endpoints	5
Security groups	2
AWS Identity and Access Management (IAM) roles	4
S3 buckets	1
SageMaker notebook	1
ECR	1
EFS	1

Supported Regions

This portion of the deployment guide is located at docs/partner_editable/regions.adoc

This Quick Start supports the following Regions:

Code	Name	Opt-in Status
us-east-1	US East (N. Virginia)(Ohio)	Not required
us-east-2	US East (Ohio)	Not required
us-west-1	US West (N. California)	Not required
us-west-2	US West (Oregon)	Not required
ap-south-1	Asia Pacific (Mumbai)	Not required
ap-northeast-1	Asia Pacific (Tokyo)	Not required
ap-northeast-2	Asia Pacific (Seoul)	Not required
ap-southeast-1	Asia Pacific (Singapore)	Not required
ap-southeast-2	Asia Pacific (Sydney)	Not required
ca-central-1	Canada (Central)	Not required
eu-central-1	Europe (Frankfurt)	Not required
eu-west-1	Europe (Ireland)	Not required
eu-west-2	Europe (London)	Not required
eu-west-3	Europe (Paris)	Not required
eu-south-1	Europe (Milan)	Required
eu-north-1	Europe (Stockholm)	Not required
sa-east-1	South America (São Paulo)	Not required

Code

Name

Opt-in Status

us-east-1

US East (N. Virginia)(Ohio)

Not required

us-east-2

US East (Ohio)

Not required

us-west-1

US West (N. California)

Not required

us-west-2

US West (Oregon)

Not required

ap-south-1

Asia Pacific (Mumbai)

Not required

ap-northeast-1

Asia Pacific (Tokyo)

Not required

ap-northeast-2

Asia Pacific (Seoul)

Not required

ap-southeast-1

Asia Pacific (Singapore)

Not required

ap-southeast-2

Asia Pacific (Sydney)

Not required

ca-central-1

Canada (Central)

Not required

eu-central-1

Europe (Frankfurt)

Not required

eu-west-1

Europe (Ireland)

Not required

eu-west-2

Europe (London)

Not required

eu-west-3

Europe (Paris)

Not required

eu-south-1

Europe (Milan)

Required

eu-north-1

Europe (Stockholm)

Not required

sa-east-1

South America (São Paulo)

Not required

Certain Regions are available on an opt-in basis. For more information, see Managing AWS Regions.

IAM permissions

Before launching the Quick Start, you must sign in to the AWS Management Console with IAM permissions for the resources that the templates deploy. The AdministratorAccess managed policy within IAM provides sufficient permissions, although your organization may choose to use a custom policy with more restrictions. For more information, see AWS managed policies for job functions.

This portion of the deployment guide is located at docs/partner_editable/pre-reqs.adoc

Prepare your AWS account

Prepare your AWS account

If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions.
Use the Region selector in the navigation bar to choose the AWS Region where you want to deploy SageMaker with guardrails.

Amazon SageMaker is not supported in all AWS Regions. For a current list of supported Regions for SageMaker, see the Supported Regions in the Technical requirements section.

If necessary, request a service quota increase for SageMaker instances that you use. You might do this if you already have an existing deployment that uses this instance type or you exceed the default quota for this deployment.
Create a custom Key Management Service (KMS) key for encrypting data on the storage volume that’s attached to your notebook instance.

Deployment options

This portion of the deployment guide is located at docs/partner_editable/deployment_options.adoc

This Quick Start provides two deployment options:

Deploy Amazon SageMaker with guardrails into a new VPC. This option builds a new AWS environment consisting of the VPC, subnets, VPC endpoint, security groups, EFS, ECR, and other infrastructure components. It then deploys SageMaker into this new VPC.
Deploy Amazon SageMaker with guardrails into an existing VPC. This option provisions SageMaker in your existing AWS infrastructure.

The Quick Start provides separate templates for these options. It also lets you configure environment details, VPC, and SageMaker settings, as discussed later in this guide.

Deployment steps

This portion of the deployment guide is located at docs/partner_editable/deploy_steps.adoc

Sign in to your AWS account

Sign in to your AWS account at https://aws.amazon.com with an IAM user role that has the necessary permissions. For details, see Planning the deployment earlier in this guide.
Make sure that your AWS account is configured correctly, as discussed in the Technical requirements section.

Launch the Quick Start

You are responsible for the cost of the AWS services used while running this Quick Start reference deployment. There is no additional cost for using this Quick Start. For full details, see the pricing pages for each AWS service used by this Quick Start. Prices are subject to change.

Sign in to your AWS account, and choose one of the following options to launch the AWS CloudFormation template. For help with choosing an option, see Deployment options earlier in this guide.

Deploy Amazon SageMaker with guardrails into a new VPC on AWS

View template

Deploy Amazon SageMaker with guardrails into an existing VPC on AWS

View template

If you deploy this Quick Start in an existing VPC, ensure that your VPC is private with no internet gateway attachment. This deployment creates two subnets in an existing VPC. Also, ensure that the Domain Name System (DNS) hostname and DNS support is enabled on existing VPC attributes. Otherwise, the QuickStart enables it during the provisioning process.

Each deployment takes about 5 minutes to complete.

Check the AWS Region that’s displayed in the upper-right corner of the navigation bar, and change it if necessary. This Region is where the network infrastructure for SageMaker with guardrails is built. The template is launched in the us-east-1 Region by default.

On the Create stack page, keep the default setting for the template URL, and then choose Next.
On the Specify stack details page, change the stack name if needed. Review the parameters for the template. Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary. See the Parameter reference for more information.

When you finish reviewing and customizing the parameters, choose Next.

On the Configure stack options page, you can specify tags (key-value pairs) for resources in your stack and set advanced options. When you’re finished, choose Next.
On the Review page, review and confirm the template settings. Under Capabilities, select the two check boxes to acknowledge that the template creates IAM resources and might require the ability to automatically expand macros.
Choose Create stack to deploy the stack.
Monitor the status of the stack. When the status is CREATE_COMPLETE, the SageMaker with guardrails deployment is ready.
Use the values displayed in the Outputs tab for the stack, as shown in SageMaker with guardrails outputs after successful deployment, to view the created resources.

Figure 2. SageMaker with guardrails outputs after successful deployment

This portion of the deployment guide is located at docs/partner_editable/additional_info.adoc

Optionally deploy Amazon SageMaker with guardrails as a Service Catalog product

After the base infrastructure is configured by the CloudFormation template, data scientists and other users can assume the IAM role (SCEndUserrole) or group that was provided in the CloudFormation output when launching the Service Catalog and then launch SageMaker.

Be sure to specify the same environment name that is provided in the CloudFormation template.

Test the deployment

To test your deployment, see Get Started with Amazon SageMaker Notebook Instances and SDKs.

This portion of the deployment guide is located at docs/partner_editable/faq_troubleshooting.adoc

FAQ

Q. I encountered a CREATE_FAILED error when I launched the Quick Start.

A. If AWS CloudFormation fails to create the stack, relaunch the template with Rollback on failure set to Disabled. This setting is under Advanced in the AWS CloudFormation console on the Configure stack options page. With this setting, the stack’s state is retained and the instance is left running, so you can troubleshoot the issue. (For Windows, look at the log files in %ProgramFiles%\Amazon\EC2ConfigService and C:\cfn\log.)

When you set Rollback on failure to Disabled, you continue to incur AWS charges for this stack. Delete the stack when you finish troubleshooting.

For additional information, see Troubleshooting AWS CloudFormation on the AWS website.

Q. I encountered a size limitation error when I deployed the AWS CloudFormation templates.

A. Launch the Quick Start templates from the links in this guide or from another S3 bucket. If you deploy the templates from a local copy on your computer or from a location other than an S3 bucket, you might encounter template size limitations. For more information, see AWS CloudFormation quotas on the AWS website.

Parameter reference

Unless you are customizing the Quick Start templates for your own deployment projects, we recommend that you keep the default settings for the parameters labeled Quick Start S3 bucket name, Quick Start S3 bucket Region, and Quick Start S3 key prefix. Changing these parameter settings automatically updates code references to point to a new Quick Start location. For more information, see the AWS Quick Start Contributor’s Guide.

Parameters for launching into a new VPC

Table 1. Environment details
Parameter label (name)	Default value	Description
Environment name (`ENVName`)	`QuickStart`	Infrastructure naming convention for SageMaker with guardrails.

Table 2. VPC network configuration for SageMaker
Parameter label (name)	Default value	Description
VPC CIDR block (`VPCCIDR`)	`10.0.0.0/16`	CIDR block for the VPC.
Resource subnet CIDR block (`Subnet1CidrBlock`)	`10.0.1.0/24`	CIDR for subnet 1.
ENI subnet CIDR block (`Subnet2CidrBlock`)	`10.0.2.0/24`	CIDR for subnet 2.

Table 3. ECR repository details
Parameter label (name)	Default value	Description
ECR repository name (`ECRRepositoryName`)	`quickstart-repository`	ECR repository name.

Table 4. AWS Quick Start configuration
Parameter label (name)	Default value	Description
Quick Start S3 bucket name (`QSS3BucketName`)	`aws-quickstart`	Name of the S3 bucket for your copy of the Quick Start assets. Keep the default name unless you are customizing the template. Changing the name updates code references to point to a new Quick Start location. This name can include numbers, lowercase letters, uppercase letters, and hyphens, but do not start or end with a hyphen (-). See https://aws-quickstart.github.io/option1.html.
Quick Start S3 key prefix (`QSS3KeyPrefix`)	`deployment/`	S3 key prefix that is used to simulate a directory for your copy of the Quick Start assets. Keep the default prefix unless you are customizing the template. Changing this prefix updates code references to point to a new Quick Start location. This prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slashes (/). See https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html and https://aws-quickstart.github.io/option1.html.
Quick Start S3 bucket Region (`QSS3BucketRegion`)	`us-east-2`	AWS Region where the Quick Start S3 bucket (QSS3BucketName) is hosted. Keep the default Region unless you are customizing the template. Changing this Region updates code references to point to a new Quick Start location. When using your own bucket, specify the Region. See https://aws-quickstart.github.io/option1.html.

Table 5. Access to Service Catalog for launching SageMaker
Parameter label (name)	Default value	Description
(Optional) Enable the IAM group accessing the Service Catalog (`EnableIAMGroup`)	`NO`	IAM group for launching SageMaker. By default, this IAM role is enabled for launching SageMaker.

Table 6. (Optional) Enable SageMaker launch from main template
Parameter label (name)	Default value	Description
Deploy SageMaker (`SageMakerLaunch`)	`YES`	Do you want to launch SageMaker from the main template?

Table 7. SageMaker notebook configuration
Parameter label (name)	Default value	Description
Notebook instance name (`NotebookInstanceName`)	`Requires input`	SageMaker notebook instance name.
Notebook instance type (`NotebookInstanceType`)	`ml.t2.medium`	Select the instance type for the SageMaker notebook.
Default internet access (`DirectInternetAccess`)	`Disabled`	When value is Disabled (the default setting), this notebook instance can only access resources in your VPC.
Root access (`RootAccess`)	`Enabled`	Root access for the SageMaker notebook user.
Volume size for the SageMaker notebook (`VolumeSizeInGB`)	`5`	The size (in GB) of the ML storage volume to attach to the notebook instance. The default value is 5 GB.

Table 8. Push code from S3 bucket to SageMaker
Parameter label (name)	Default value	Description
Code to push from S3 (`S3CodePusher`)	`NO`	Do you want to load the code from S3 to the SageMaker notebook?
Code bucket name (`CodeBucketName`)	`quickstart-code-bucket`	S3 bucket name from which you want to push code.

Table 9. Access to SageMaker notebook
Parameter label (name)	Default value	Description
Enable IAM group access for SageMaker notebook (`IAMGroup`)	`NO`	IAM group for accessing the SageMaker notebook. By default, this IAM role is enabled for accessing the SageMaker notebook.

Table 10. Project detail
Parameter label (name)	Default value	Description
Project suffix (`ProjectName`)	`Requires input`	The suffix appended to all resources in the stack. This suffix allows multiple copies of the same stack to be created in the same account.
SageMaker project ID (`ProjectID`)	`QuickStart007`	Enter a valid project ID.

Parameters for launching into an existing VPC

Table 11. Environment details
Parameter label (name)	Default value	Description
Environment name (`ENVName`)	`QuickStart`	Infrastructure naming convention for SageMaker with guardrails.

Table 12. Existing VPC network configuration for SageMaker
Parameter label (name)	Default value	Description
VPC name (`VPCID`)	`Requires input`	Select existing VPC for SageMaker.
Resource subnet CIDR block (`Subnet1CidrBlock`)	`10.0.1.0/24`	CIDR for subnet 1.
ENI subnet CIDR block (`Subnet2CidrBlock`)	`10.0.2.0/24`	CIDR for subnet 2.

Table 13. ECR repository details
Parameter label (name)	Default value	Description
ECR repository name (`ECRRepositoryName`)	`quickstart-repository`	ECR repository name.

Table 14. AWS Quick Start configuration
Parameter label (name)	Default value	Description
Quick Start S3 bucket name (`QSS3BucketName`)	`aws-quickstart`	Name of the S3 bucket for your copy of the Quick Start assets. Keep the default name unless you are customizing the template. Changing the name updates code references to point to a new Quick Start location. This name can include numbers, lowercase letters, uppercase letters, and hyphens, but do not start or end with a hyphen (-). See https://aws-quickstart.github.io/option1.html.
Quick Start S3 key prefix (`QSS3KeyPrefix`)	`deployment/`	S3 key prefix that is used to simulate a directory for your copy of the Quick Start assets. Keep the default prefix unless you are customizing the template. Changing this prefix updates code references to point to a new Quick Start location. This prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slashes (/). See https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html and https://aws-quickstart.github.io/option1.html.
Quick Start S3 bucket Region (`QSS3BucketRegion`)	`us-east-2`	AWS Region where the Quick Start S3 bucket (QSS3BucketName) is hosted. Keep the default Region unless you are customizing the template. Changing this Region updates code references to point to a new Quick Start location. When using your own bucket, specify the Region. See https://aws-quickstart.github.io/option1.html.

Table 15. User access to Service Catalog to launch SageMaker
Parameter label (name)	Default value	Description
Enable IAM group access for Service Catalog (`EnableIAMGroup`)	`NO`	(Optional) IAM group for launching SageMaker. The IAM role is enabled by default.

Table 16. Enable SageMaker launch from main template
Parameter label (name)	Default value	Description
Deploy SageMaker (`SageMakerLaunch`)	`YES`	(Optional) Do you want to launch SageMaker from the main template?

Table 17. SageMaker notebook configuration
Parameter label (name)	Default value	Description
Notebook instance name (`NotebookInstanceName`)	`Requires input`	SageMaker notebook instance name.
Notebook instance type (`NotebookInstanceType`)	`ml.t2.medium`	Select instance type for the SageMaker notebook.
Default internet access (`DirectInternetAccess`)	`Disabled`	When value is Disabled (default), this notebook instance can only access resources in your VPC.
Root access (`RootAccess`)	`Enabled`	Root access for the SageMaker notebook user.
Volume size for the SageMaker notebook (`VolumeSizeInGB`)	`5`	The size (in GB) of the ML storage volume to attach to the notebook instance. The default value is 5 GB.

Table 18. Pushing code from S3 bucket to SageMaker
Parameter label (name)	Default value	Description
Code to push from S3 (`S3CodePusher`)	`NO`	Do you want to load the code from S3 to the SageMaker notebook?
Code bucket name (`CodeBucketName`)	`quickstart-code-bucket`	S3 bucket name from which to push code.

Table 19. Access to SageMaker notebook
Parameter label (name)	Default value	Description
Enable IAM group access for the SageMaker notebook (`IAMGroup`)	`NO`	(Optional) IAM group for accessing the SageMaker notebook. By default, the IAM role is enabled.

Table 20. Project detail
Parameter label (name)	Default value	Description
Project suffix (`ProjectName`)	`Requires input`	The suffix appended to all resources in the stack. This suffix allows multiple copies of the same stack to be created in the same account.
SageMaker project ID (`ProjectID`)	`QuickStart007`	Enter a valid project ID.

Send us feedback

To post feedback, submit feature ideas, or report bugs, use the Issues section of the GitHub repository for this Quick Start. To submit code, see the Quick Start Contributor’s Guide.

Quick Start reference deployments

See the AWS Quick Start home page.

GitHub repository

Visit our GitHub repository to download the templates and scripts for this Quick Start, to post your comments, and to share your customizations with others.

Notices

This document is provided for informational purposes only. It represents AWS’s current product offerings and practices as of the date of issue of this document, which are subject to change without notice. Customers are responsible for making their own independent assessment of the information in this document and any use of AWS’s products or services, each of which is provided “as is” without warranty of any kind, whether expressed or implied. This document does not create any warranties, representations, contractual commitments, conditions, or assurances from AWS, its affiliates, suppliers, or licensors. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers.

The software included with this paper is licensed under the Apache License, version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the accompanying "license" file. This code is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either expressed or implied. See the License for specific language governing permissions and limitations.