--- AWSTemplateFormatVersion: '2010-09-09' Description: This workload template deploys an ASG behind an ELB load balancer in two private subnets. The cluster is configured to use an S3 bucket for storage **WARNING** This template creates EC2 instances and related resources. You will be billed for the AWS resources used if you create a stack from this template. (qs-1s1eguak1) Metadata: cfn-lint: config: ignore_checks: - W9006 # temporary to get rid of warnings - W9002 # temporary to get rid of warnings - W9003 # temporary to get rid of warnings - W9901 # Not passing all the default parameters to reduce verbosity AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Network configuration Parameters: - VPCID - PrivateSubnet1ID - PrivateSubnet2ID - PublicSubnet1ID - PublicSubnet2ID - Label: default: Amazon EC2 configuration Parameters: - KeyPairName - BastionSecurityGroupID - WorkloadInstanceType - Label: default: Workload nodes configuration Parameters: - WorkloadNodesMinSize - WorkloadNodesMaxSize - WorkloadNodesDesiredCapacity - OperatorEmail - Label: default: Workload storage configuration Parameters: - S3BucketName - Label: default: AWS Quick Start configuration Parameters: - QSS3BucketName - QSS3KeyPrefix ParameterLabels: BastionSecurityGroupID: default: Bastion security group ID KeyPairName: default: SSH key name OperatorEmail: default: Operator email PrivateSubnet1ID: default: Private subnet 1 ID PrivateSubnet2ID: default: Private subnet 2 ID PublicSubnet1ID: default: Public subnet 1 ID PublicSubnet2ID: default: Public subnet 2 ID QSS3BucketName: default: Quick Start S3 bucket name QSS3KeyPrefix: default: Quick Start S3 key prefix S3BucketName: default: S3 bucket name VPCID: default: VPC ID WorkloadInstanceType: default: Workload servers instance type WorkloadNodesDesiredCapacity: default: Workload nodes desired capacity WorkloadNodesMaxSize: default: Workload nodes max size WorkloadNodesMinSize: default: Workload nodes min size Parameters: BastionSecurityGroupID: Description: ID of the bastion host security group to enable SSH connections (e.g., sg-7f16e910). Type: AWS::EC2::SecurityGroup::Id KeyPairName: Description: Name of an existing EC2 key pair. All instances will launch with this key pair. Type: AWS::EC2::KeyPair::KeyName OperatorEmail: AllowedPattern: ([a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?) ConstraintDescription: Must be a valid email address. Description: Email address that notifications of any scaling operations will be sent to. Type: String PrivateSubnet1ID: Description: ID of private subnet 1 in Availability Zone 1 for the workload (e.g., subnet-a0246dcd). Type: AWS::EC2::Subnet::Id PrivateSubnet2ID: Description: ID of private subnet 2 in Availability Zone 2 for the workload (e.g., subnet-b1f432cd). Type: AWS::EC2::Subnet::Id PublicSubnet1ID: Description: ID of public subnet 1 in Availability Zone 1 for the ELB load balancer (e.g., subnet-9bc642ac). Type: AWS::EC2::Subnet::Id PublicSubnet2ID: Description: ID of public subnet 2 in Availability Zone 2 for the ELB load balancer (e.g., subnet-e3246d8e). Type: AWS::EC2::Subnet::Id QSS3BucketName: AllowedPattern: ^[0-9a-zA-Z]+([0-9a-zA-Z-]*[0-9a-zA-Z])*$ ConstraintDescription: Quick Start bucket name can include numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen (-). Default: aws-quickstart Description: S3 bucket name for the Quick Start assets. This string can include numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen (-). Type: String QSS3KeyPrefix: AllowedPattern: ^[0-9a-zA-Z-/]*$ ConstraintDescription: Quick Start key prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slash (/). Default: quickstart-examples/ Description: S3 key prefix for the Quick Start assets. Quick Start key prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slash (/). Type: String S3BucketName: AllowedPattern: ^[a-z0-9][a-z0-9-.]*$ Default: type-unique-value-here-in-lowercase Description: Name of the S3 bucket that will be created for your workload to store data. Enter a unique name that does not include uppercase characters. Type: String VPCID: Description: ID of your existing VPC for deployment. Type: AWS::EC2::VPC::Id WorkloadInstanceType: AllowedValues: - t2.large - m4.large - m4.xlarge - m4.2xlarge - m4.4xlarge - m4.10xlarge - m3.medium - m3.large - m3.xlarge - m3.2xlarge - c4.large - c4.xlarge - c4.2xlarge - c4.4xlarge - c4.8xlarge - c3.large - c3.xlarge - c3.2xlarge - c3.4xlarge - c3.8xlarge - r3.large - r3.xlarge ConstraintDescription: Must contain valid instance type Default: m4.xlarge Description: Type of EC2 instance for the workload instances. Type: String WorkloadNodesDesiredCapacity: Default: '2' Description: Desired capacity for the workload nodes Auto Scaling group. Type: String WorkloadNodesMaxSize: Default: '4' Description: Maximum size of the Auto Scaling group. Type: String WorkloadNodesMinSize: Default: '2' Description: Minimum size of the Auto Scaling group. Type: String Rules: KeyPairsNotEmpty: Assertions: - Assert: Fn::Not: - Fn::EachMemberEquals: - Fn::RefAll: AWS::EC2::KeyPair::KeyName - '' AssertDescription: All key pair parameters must not be empty SubnetsInVPC: Assertions: - Assert: Fn::EachMemberIn: - Fn::ValueOfAll: - AWS::EC2::Subnet::Id - VpcId - Fn::RefAll: AWS::EC2::VPC::Id AssertDescription: All subnets must in the VPC Mappings: AWSAMIRegionMap: AMI: AMZNLINUXHVM: amzn-ami-hvm-2018.03.0.20190611-x86_64-gp2 ap-northeast-1: AMZNLINUXHVM: ami-02ddf94e5edc8e904 ap-northeast-2: AMZNLINUXHVM: ami-0ecd78c22823e02ef ap-south-1: AMZNLINUXHVM: ami-05695932c5299858a ap-southeast-1: AMZNLINUXHVM: ami-043afc2b8b6cfba5c ap-southeast-2: AMZNLINUXHVM: ami-01393ce9a3ca55d67 ca-central-1: AMZNLINUXHVM: ami-0fa94ecf2fef3420b eu-central-1: AMZNLINUXHVM: ami-0ba441bdd9e494102 eu-west-1: AMZNLINUXHVM: ami-0e61341fa75fcaa18 eu-west-2: AMZNLINUXHVM: ami-050b8344d77081f4b sa-east-1: AMZNLINUXHVM: ami-05b7dbc290217250d us-east-1: AMZNLINUXHVM: ami-0e2ff28bfb72a4e45 us-east-2: AMZNLINUXHVM: ami-0998bf58313ab53da us-west-1: AMZNLINUXHVM: ami-021bb9f371690f97a us-west-2: AMZNLINUXHVM: ami-079f731edfe27c29c Conditions: UsingDefaultBucket: !Equals [!Ref QSS3BucketName, 'aws-quickstart'] Resources: NotificationTopic: Type: AWS::SNS::Topic Properties: Subscription: - Endpoint: Ref: OperatorEmail Protocol: email ELBSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow access to the ELB VpcId: Ref: VPCID SecurityGroupIngress: - IpProtocol: tcp FromPort: 80 ToPort: 80 CidrIp: 0.0.0.0/0 - IpProtocol: tcp FromPort: 443 ToPort: 443 CidrIp: 0.0.0.0/0 ElasticLoadBalancer: Type: AWS::ElasticLoadBalancing::LoadBalancer Properties: Subnets: - Ref: PublicSubnet1ID - Ref: PublicSubnet2ID SecurityGroups: - Ref: ELBSecurityGroup AppCookieStickinessPolicy: - CookieName: dummy PolicyName: WorkloadCookieStickinessPolicy Listeners: - LoadBalancerPort: '80' InstancePort: '80' Protocol: HTTP PolicyNames: - WorkloadCookieStickinessPolicy CrossZone: true HealthCheck: Target: TCP:80 HealthyThreshold: '2' UnhealthyThreshold: '3' Interval: '30' Timeout: '3' SetupRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com Action: - sts:AssumeRole Path: / Policies: - PolicyDocument: Version: '2012-10-17' Statement: - Action: - s3:GetObject Resource: - !Sub - arn:${AWS::Partition}:s3:::${S3Bucket}/${QSS3KeyPrefix}* - S3Bucket: !If - UsingDefaultBucket - !Sub '${QSS3BucketName}-${AWS::Region}' - !Ref QSS3BucketName Effect: Allow PolicyName: aws-quick-start-s3-policy - PolicyName: WorkloadSetup PolicyDocument: Statement: - Effect: Allow Action: - cloudwatch:PutMetricData Resource: '*' - Effect: Allow Action: - cloudwatch:EnableAlarmActions - cloudwatch:PutMetricAlarm Resource: - !Sub arn:${AWS::Partition}:cloudwatch:${AWS::Region}:${AWS::AccountId}:alarm:* - Effect: Allow Action: ["s3:PutLifecycleConfiguration", "s3:AbortMultipartUpload", "s3:DeleteObjectTagging", "s3:DeleteAccessPoint", "s3:GetAccessPoint", "s3:GetAccessPointPolicyStatus", "s3:GetAccessPointPolicyStatusForObjectLambda", "s3:PutMetricsConfiguration", "s3:GetAccessPointConfigurationForObjectLambda", "s3:BypassGovernanceRetention", "s3:PutBucketLogging", "s3:ListAllMyBuckets", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetObjectVersionTagging", "s3:PutEncryptionConfiguration", "s3:GetBucketTagging", "s3:GetAccelerateConfiguration", "s3:PutIntelligentTieringConfiguration", "s3:GetBucketPolicy", "s3:PutJobTagging", "s3:PutBucketCORS", "s3:GetReplicationConfiguration", "s3:DeleteAccessPointForObjectLambda", "s3:GetStorageLensDashboard", "s3:ListJobs", "s3:GetMetricsConfiguration", "s3:PutAccelerateConfiguration", "s3:PutStorageLensConfigurationTagging", "s3:PutAccessPointConfigurationForObjectLambda", "s3:GetBucketObjectLockConfiguration", "s3:GetInventoryConfiguration", "s3:PutBucketWebsite", "s3:PutAnalyticsConfiguration", "s3:GetIntelligentTieringConfiguration", "s3:GetLifecycleConfiguration", "s3:ListStorageLensConfigurations", "s3:DeleteObjectVersion", "s3:GetBucketPublicAccessBlock", "s3:DeleteJobTagging", "s3:PutObjectVersionTagging", "s3:GetObjectAcl", "s3:PutBucketObjectLockConfiguration", "s3:CreateAccessPointForObjectLambda", "s3:PutBucketPolicy", "s3:GetBucketLogging", "s3:GetObjectVersionForReplication", "s3:GetObject", "s3:GetBucketLocation", "s3:DeleteAccessPointPolicyForObjectLambda", "s3:DeleteStorageLensConfigurationTagging", "s3:GetBucketPolicyStatus", "s3:RestoreObject", "s3:GetBucketOwnershipControls", "s3:PutStorageLensConfiguration", "s3:DeleteBucketWebsite", "s3:PutInventoryConfiguration", "s3:ListAccessPoints", "s3:ListMultipartUploadParts", "s3:GetObjectVersion", "s3:DeleteBucketOwnershipControls", "s3:GetBucketCORS", "s3:PutObject", "s3:PutBucketNotification", "s3:PutObjectTagging", "s3:GetEncryptionConfiguration", "s3:GetStorageLensConfiguration", "s3:GetObjectVersionTorrent", "s3:PutAccessPointPolicyForObjectLambda", "s3:PutObjectRetention", "s3:PutBucketPublicAccessBlock", "s3:PutBucketVersioning", "s3:GetAccountPublicAccessBlock", "s3:GetBucketNotification", "s3:GetBucketVersioning", "s3:DeleteStorageLensConfiguration", "s3:GetObjectVersionAcl", "s3:ReplicateTags", "s3:UpdateJobStatus", "s3:DeleteAccessPointPolicy", "s3:GetBucketAcl", "s3:GetObjectLegalHold", "s3:GetAnalyticsConfiguration", "s3:GetObjectRetention", "s3:DeleteBucketPolicy", "s3:PutObjectVersionAcl", "s3:PutAccountPublicAccessBlock", "s3:PutReplicationConfiguration", "s3:DescribeJob", "s3:GetAccessPointForObjectLambda", "s3:CreateAccessPoint", "s3:PutAccessPointPolicy", "s3:GetObjectTorrent", "s3:ListAccessPointsForObjectLambda", "s3:GetBucketRequestPayment", "s3:CreateJob", "s3:GetBucketWebsite", "s3:PutObjectAcl", "s3:PutBucketAcl", "s3:ListBucketVersions", "s3:GetJobTagging", "s3:PutBucketTagging", "s3:PutBucketRequestPayment", "s3:ReplicateDelete", "s3:DeleteObjectVersionTagging", "s3:UpdateJobPriority", "s3:PutBucketOwnershipControls", "s3:DeleteObject", "s3:DeleteBucket", "s3:ObjectOwnerOverrideToBucketOwner", "s3:PutObjectLegalHold", "s3:GetStorageLensConfigurationTagging", "s3:GetAccessPointPolicyForObjectLambda", "s3:GetObjectTagging", "s3:CreateBucket", "s3:ReplicateObject", "s3:GetAccessPointPolicy"] Resource: - Fn::Sub: arn:${AWS::Partition}:s3:::${S3Bucket} - Fn::Sub: arn:${AWS::Partition}:s3:::${S3Bucket}/* SetupRoleProfile: Type: AWS::IAM::InstanceProfile Properties: Path: / Roles: - Ref: SetupRole WorkloadSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Allow access to the Workload instances VpcId: Ref: VPCID SecurityGroupIngress: - IpProtocol: tcp FromPort: 80 ToPort: 80 SourceSecurityGroupId: Ref: ELBSecurityGroup - IpProtocol: tcp FromPort: 443 ToPort: 443 SourceSecurityGroupId: Ref: ELBSecurityGroup - IpProtocol: tcp FromPort: 22 ToPort: 22 SourceSecurityGroupId: Ref: BastionSecurityGroupID WorkloadASLaunchConfig: Type: AWS::AutoScaling::LaunchConfiguration Metadata: AWS::CloudFormation::Init: configSets: quickstart: - install - configure - cleanup install: {} configure: {} cleanup: {} Properties: KeyName: Ref: KeyPairName ImageId: Fn::FindInMap: - AWSAMIRegionMap - Ref: AWS::Region - AMZNLINUXHVM InstanceMonitoring: true IamInstanceProfile: Ref: SetupRoleProfile InstanceType: Ref: WorkloadInstanceType SecurityGroups: - Ref: WorkloadSecurityGroup UserData: Fn::Base64: Fn::Sub: | #!/bin/bash export PATH=$PATH:/usr/local/bin which pip &> /dev/null if [ $? -ne 0 ] ; then echo "PIP NOT INSTALLED" [ `which yum` ] && $(yum install -y epel-release; yum install -y python-pip) && echo "PIP INSTALLED" [ `which apt-get` ] && apt-get -y update && apt-get -y install python-pip && echo "PIP INSTALLED" fi pip install --upgrade pip &> /dev/null pip install awscli --ignore-installed six &> /dev/null easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz cfn-init --stack ${AWS::StackName} --resource WorkloadASLaunchConfig --configsets quickstart --region ${AWS::Region} # Signal the status from cfn-init cfn-signal -e $? --stack ${AWS::StackName} --resource WorkloadAutoScalingGroup --region ${AWS::Region} WorkloadScaleUpPolicy: Type: AWS::AutoScaling::ScalingPolicy Properties: AdjustmentType: ChangeInCapacity AutoScalingGroupName: Ref: WorkloadAutoScalingGroup Cooldown: '300' ScalingAdjustment: 1 WorkloadScaleDownPolicy: Type: AWS::AutoScaling::ScalingPolicy Properties: AdjustmentType: ChangeInCapacity AutoScalingGroupName: Ref: WorkloadAutoScalingGroup Cooldown: '300' ScalingAdjustment: -1 CPUAlarmHigh: Type: AWS::CloudWatch::Alarm Properties: AlarmDescription: Scale-up if CPU > 60% for 5 minutes MetricName: CPUUtilization Namespace: AWS/EC2 Statistic: Average Period: 60 EvaluationPeriods: 5 Threshold: 60 AlarmActions: - Ref: WorkloadScaleUpPolicy Dimensions: - Name: AutoScalingGroupName Value: Ref: WorkloadAutoScalingGroup ComparisonOperator: GreaterThanThreshold CPUAlarmLow: Type: AWS::CloudWatch::Alarm Properties: AlarmDescription: Scale-down if CPU < 40% for 30 minutes MetricName: CPUUtilization Namespace: AWS/EC2 Statistic: Average Period: 60 EvaluationPeriods: 30 Threshold: 40 AlarmActions: - Ref: WorkloadScaleDownPolicy Dimensions: - Name: AutoScalingGroupName Value: Ref: WorkloadAutoScalingGroup ComparisonOperator: LessThanThreshold WorkloadAutoScalingGroup: Type: AWS::AutoScaling::AutoScalingGroup Properties: VPCZoneIdentifier: - Ref: PrivateSubnet1ID - Ref: PrivateSubnet2ID Cooldown: '600' DesiredCapacity: Ref: WorkloadNodesDesiredCapacity HealthCheckGracePeriod: 600 HealthCheckType: EC2 LaunchConfigurationName: Ref: WorkloadASLaunchConfig LoadBalancerNames: - Ref: ElasticLoadBalancer MaxSize: Ref: WorkloadNodesMaxSize MinSize: Ref: WorkloadNodesMinSize NotificationConfigurations: - TopicARN: Ref: NotificationTopic NotificationTypes: - autoscaling:EC2_INSTANCE_LAUNCH - autoscaling:EC2_INSTANCE_LAUNCH_ERROR - autoscaling:EC2_INSTANCE_TERMINATE - autoscaling:EC2_INSTANCE_TERMINATE_ERROR - autoscaling:TEST_NOTIFICATION Tags: - Key: Name Value: Workload Server cluster node PropagateAtLaunch: true CreationPolicy: ResourceSignal: Count: Ref: WorkloadNodesDesiredCapacity Timeout: PT2H S3Bucket: Type: AWS::S3::Bucket Properties: BucketName: Ref: S3BucketName AccessControl: BucketOwnerFullControl Outputs: ELBDNSName: Description: ELB DNS Name Value: Fn::GetAtt: - ElasticLoadBalancer - DNSName ...