## Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. ## SPDX-License-Identifier: MIT-0 --- AWSTemplateFormatVersion: '2010-09-09' Description: 'Amazon EKS - Node Group' Parameters: KeyName: Description: The EC2 Key Pair to allow SSH access to the instances Type: AWS::EC2::KeyPair::KeyName NodeImageId: Type: AWS::EC2::Image::Id Description: AMI id for the node instances. NodeInstanceType: Description: EC2 instance type for the node instances Type: String Default: t2.medium AllowedValues: - t2.small - t2.medium - t2.large - t2.xlarge - t2.2xlarge - m3.medium - m3.large - m3.xlarge - m3.2xlarge - m4.large - m4.xlarge - m4.2xlarge - m4.4xlarge - m4.10xlarge - m5.large - m5.xlarge - m5.2xlarge - m5.4xlarge - m5.12xlarge - m5.24xlarge - c4.large - c4.xlarge - c4.2xlarge - c4.4xlarge - c4.8xlarge - c5.large - c5.xlarge - c5.2xlarge - c5.4xlarge - c5.9xlarge - c5.18xlarge - i3.large - i3.xlarge - i3.2xlarge - i3.4xlarge - i3.8xlarge - i3.16xlarge - r3.xlarge - r3.2xlarge - r3.4xlarge - r3.8xlarge - r4.large - r4.xlarge - r4.2xlarge - r4.4xlarge - r4.8xlarge - r4.16xlarge - x1.16xlarge - x1.32xlarge - p2.xlarge - p2.8xlarge - p2.16xlarge - p3.2xlarge - p3.8xlarge - p3.16xlarge ConstraintDescription: Must be a valid EC2 instance type NodeAutoScalingGroupMinSize: Type: Number Description: Minimum size of Node Group ASG. Default: 1 NodeAutoScalingGroupMaxSize: Type: Number Description: Maximum size of Node Group ASG. Default: 3 NodeVolumeSize: Type: Number Description: Node volume size Default: 20 ClusterName: Description: The cluster name provided when the cluster was created. If it is incorrect, nodes will not be able to join the cluster. Type: String BootstrapArguments: Description: Arguments to pass to the bootstrap script. See files/bootstrap.sh in https://github.com/awslabs/amazon-eks-ami Default: "" Type: String NodeGroupName: Description: Unique identifier for the Node Group. Type: String NetworkStackName: Description: The name of the eks and ec2 network stack that contains the export values Type: String Mappings: MaxPodsPerNode: c4.large: MaxPods: 29 c4.xlarge: MaxPods: 58 c4.2xlarge: MaxPods: 58 c4.4xlarge: MaxPods: 234 c4.8xlarge: MaxPods: 234 c5.large: MaxPods: 29 c5.xlarge: MaxPods: 58 c5.2xlarge: MaxPods: 58 c5.4xlarge: MaxPods: 234 c5.9xlarge: MaxPods: 234 c5.18xlarge: MaxPods: 737 i3.large: MaxPods: 29 i3.xlarge: MaxPods: 58 i3.2xlarge: MaxPods: 58 i3.4xlarge: MaxPods: 234 i3.8xlarge: MaxPods: 234 i3.16xlarge: MaxPods: 737 m3.medium: MaxPods: 12 m3.large: MaxPods: 29 m3.xlarge: MaxPods: 58 m3.2xlarge: MaxPods: 118 m4.large: MaxPods: 20 m4.xlarge: MaxPods: 58 m4.2xlarge: MaxPods: 58 m4.4xlarge: MaxPods: 234 m4.10xlarge: MaxPods: 234 m5.large: MaxPods: 29 m5.xlarge: MaxPods: 58 m5.2xlarge: MaxPods: 58 m5.4xlarge: MaxPods: 234 m5.12xlarge: MaxPods: 234 m5.24xlarge: MaxPods: 737 p2.xlarge: MaxPods: 58 p2.8xlarge: MaxPods: 234 p2.16xlarge: MaxPods: 234 p3.2xlarge: MaxPods: 58 p3.8xlarge: MaxPods: 234 p3.16xlarge: MaxPods: 234 r3.xlarge: MaxPods: 58 r3.2xlarge: MaxPods: 58 r3.4xlarge: MaxPods: 234 r3.8xlarge: MaxPods: 234 r4.large: MaxPods: 29 r4.xlarge: MaxPods: 58 r4.2xlarge: MaxPods: 58 r4.4xlarge: MaxPods: 234 r4.8xlarge: MaxPods: 234 r4.16xlarge: MaxPods: 737 t2.small: MaxPods: 8 t2.medium: MaxPods: 17 t2.large: MaxPods: 35 t2.xlarge: MaxPods: 44 t2.2xlarge: MaxPods: 44 x1.16xlarge: MaxPods: 234 x1.32xlarge: MaxPods: 234 Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: "EKS Cluster" Parameters: - ClusterName - Label: default: "Worker Node Configuration" Parameters: - NodeGroupName - NodeAutoScalingGroupMinSize - NodeAutoScalingGroupMaxSize - NodeInstanceType - NodeImageId - NodeVolumeSize - KeyName - BootstrapArguments Resources: NodeInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: InstanceProfileName: EKSNode Path: "/" Roles: - !Ref NodeInstanceRole NodeInstanceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com Action: - sts:AssumeRole Path: "/" ManagedPolicyArns: - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly - arn:aws:iam::aws:policy/AmazonEC2FullAccess Policies: - PolicyName: "spinnaker-bucket-read" PolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Resource: Fn::ImportValue: !Sub "${NetworkStackName}-eks-admin-role-arn" Action: - "sts:AssumeRole" - Effect: "Allow" Action: - "s3:PutAnalyticsConfiguration" - "s3:GetObjectVersionTagging" - "s3:CreateBucket" - "s3:ReplicateObject" - "s3:GetObjectAcl" - "s3:DeleteBucketWebsite" - "s3:PutLifecycleConfiguration" - "s3:GetObjectVersionAcl" - "s3:PutObjectTagging" - "s3:DeleteObject" - "s3:GetIpConfiguration" - "s3:DeleteObjectTagging" - "s3:GetBucketWebsite" - "s3:PutReplicationConfiguration" - "s3:DeleteObjectVersionTagging" - "s3:GetBucketNotification" - "s3:PutBucketCORS" - "s3:GetReplicationConfiguration" - "s3:ListMultipartUploadParts" - "s3:PutObject" - "s3:GetObject" - "s3:PutBucketNotification" - "s3:PutBucketLogging" - "s3:PutObjectVersionAcl" - "s3:GetAnalyticsConfiguration" - "s3:GetObjectVersionForReplication" - "s3:GetLifecycleConfiguration" - "s3:ListBucketByTags" - "s3:GetInventoryConfiguration" - "s3:GetBucketTagging" - "s3:PutAccelerateConfiguration" - "s3:DeleteObjectVersion" - "s3:GetBucketLogging" - "s3:ListBucketVersions" - "s3:ReplicateTags" - "s3:RestoreObject" - "s3:ListBucket" - "s3:GetAccelerateConfiguration" - "s3:GetBucketPolicy" - "s3:PutEncryptionConfiguration" - "s3:GetEncryptionConfiguration" - "s3:GetObjectVersionTorrent" - "s3:AbortMultipartUpload" - "s3:PutBucketTagging" - "s3:GetBucketRequestPayment" - "s3:GetObjectTagging" - "s3:GetMetricsConfiguration" - "s3:DeleteBucket" - "s3:PutBucketVersioning" - "s3:PutObjectAcl" - "s3:ListBucketMultipartUploads" - "s3:PutMetricsConfiguration" - "s3:PutObjectVersionTagging" - "s3:GetBucketVersioning" - "s3:GetBucketAcl" - "s3:PutInventoryConfiguration" - "s3:PutIpConfiguration" - "s3:GetObjectTorrent" - "s3:PutBucketWebsite" - "s3:PutBucketRequestPayment" - "s3:GetBucketCORS" - "s3:GetBucketLocation" - "s3:ReplicateDelete" - "s3:GetObjectVersion" Resource: - Fn::Sub: - "${BucketName}/*" - BucketName: Fn::ImportValue: !Sub "${NetworkStackName}-spinnaker-data-bucket" - Fn::ImportValue: !Sub "${NetworkStackName}-spinnaker-data-bucket" NodeSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Security group for all nodes in the cluster VpcId: Fn::ImportValue: !Sub "${NetworkStackName}-eks-vpc-id" Tags: - Key: Name Value: eks-node-sg - Key: !Sub "kubernetes.io/cluster/${ClusterName}" Value: 'owned' NodeSecurityGroupIngress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow node to communicate with each other GroupId: !Ref NodeSecurityGroup SourceSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: '-1' FromPort: 0 ToPort: 65535 NodeSecurityGroupFromControlPlaneIngress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow worker Kubelets and pods to receive communication from the cluster control plane GroupId: !Ref NodeSecurityGroup SourceSecurityGroupId: Fn::ImportValue: !Sub "${NetworkStackName}-eks-security-groups" IpProtocol: tcp FromPort: 1025 ToPort: 65535 ControlPlaneEgressToNodeSecurityGroup: Type: AWS::EC2::SecurityGroupEgress DependsOn: NodeSecurityGroup Properties: Description: Allow the cluster control plane to communicate with worker Kubelet and pods GroupId: Fn::ImportValue: !Sub "${NetworkStackName}-eks-security-groups" DestinationSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: tcp FromPort: 1025 ToPort: 65535 ClusterControlPlaneSecurityGroupIngress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow pods to communicate with the cluster API Server GroupId: Fn::ImportValue: !Sub "${NetworkStackName}-eks-security-groups" SourceSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: tcp ToPort: 443 FromPort: 443 NodeSecurityGroupFromControlPlaneOn443Ingress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow pods running extension API servers on port 443 to receive communication from cluster control plane GroupId: !Ref NodeSecurityGroup SourceSecurityGroupId: Fn::ImportValue: !Sub "${NetworkStackName}-eks-security-groups" IpProtocol: tcp FromPort: 443 ToPort: 443 ControlPlaneEgressToNodeSecurityGroupOn443: Type: AWS::EC2::SecurityGroupEgress DependsOn: NodeSecurityGroup Properties: Description: Allow the cluster control plane to communicate with pods running extension API servers on port 443 GroupId: Fn::ImportValue: !Sub "${NetworkStackName}-eks-security-groups" DestinationSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: tcp FromPort: 443 ToPort: 443 ClusterControlPlaneSecurityGroupIngress: Type: AWS::EC2::SecurityGroupIngress DependsOn: NodeSecurityGroup Properties: Description: Allow pods to communicate with the cluster API Server GroupId: Fn::ImportValue: !Sub "${NetworkStackName}-eks-security-groups" SourceSecurityGroupId: !Ref NodeSecurityGroup IpProtocol: tcp ToPort: 443 FromPort: 443 NodeGroup: Type: AWS::AutoScaling::AutoScalingGroup Properties: DesiredCapacity: !Ref NodeAutoScalingGroupMaxSize LaunchConfigurationName: !Ref NodeLaunchConfig MinSize: !Ref NodeAutoScalingGroupMinSize MaxSize: !Ref NodeAutoScalingGroupMaxSize VPCZoneIdentifier: Fn::Split: - "," - Fn::ImportValue: !Sub "${NetworkStackName}-eks-subnet-ids" Tags: - Key: Name Value: !Sub "${ClusterName}-${NodeGroupName}-Node" PropagateAtLaunch: 'true' - Key: !Sub 'kubernetes.io/cluster/${ClusterName}' Value: 'owned' PropagateAtLaunch: 'true' UpdatePolicy: AutoScalingRollingUpdate: MinInstancesInService: '1' MaxBatchSize: '1' NodeLaunchConfig: Type: AWS::AutoScaling::LaunchConfiguration Properties: AssociatePublicIpAddress: 'true' IamInstanceProfile: !Ref NodeInstanceProfile ImageId: !Ref NodeImageId InstanceType: !Ref NodeInstanceType KeyName: !Ref KeyName SecurityGroups: - !Ref NodeSecurityGroup BlockDeviceMappings: - DeviceName: /dev/xvda Ebs: VolumeSize: !Ref NodeVolumeSize VolumeType: gp2 DeleteOnTermination: true UserData: Fn::Base64: !Sub | #!/bin/bash set -o xtrace /etc/eks/bootstrap.sh ${ClusterName} ${BootstrapArguments} /opt/aws/bin/cfn-signal --exit-code $? \ --stack ${AWS::StackName} \ --resource NodeGroup \ --region ${AWS::Region} Outputs: NodeInstanceRole: Description: The node instance role Value: !GetAtt NodeInstanceRole.Arn