AWSTemplateFormatVersion: "2010-09-09" Description: Deploy Lenses with MSK into a new VPC. **WARNING** This template creates Amazon MSK cluster, Amazon EC2 instances and related resources. You will be billed for the AWS resources used if you create a stack from this template. (qs-1rrj29p5t) Metadata: QuickStartDocumentation: EntrypointName: Launch into a new VPC OptionalParameters: - LensesSchemaRegistryURLs - LensesConnectURLs Order: "1" AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Network configuration Parameters: - AvailabilityZones - VPCCIDR - PrivateSubnet1CIDR - PrivateSubnet2CIDR - PublicSubnet1CIDR - Label: default: Amazon MSK configuration Parameters: - MSKApacheKafkaVersion - MSKBrokerType - MSKEBSStorageVolumeSize - MSKNumberOfBrokerNodes - MSKEncryptionWithinCluster - MSKEncryptionInTransit - MSKEnhancedMonitoring - Label: default: Lenses configuration Parameters: - LensesInstanceType - LensesWebIngressCIDR - LensesStorage - LensesSchemaRegistryURLs - LensesConnectURLs - LensesCloudWatchMetrics - Label: default: Quick Start configuration Parameters: - QSS3BucketName - QSS3BucketRegion - QSS3KeyPrefix ParameterLabels: AvailabilityZones: default: Availability zones LensesCloudWatchMetrics: default: Lenses CloudWatch metrics LensesConnectURLs: default: Connect JSON LensesInstanceType: default: Lenses instance type LensesSchemaRegistryURLs: default: Schema registry JSON LensesStorage: default: Lenses storage preference LensesWebIngressCIDR: default: Inbound traffic CIDR address range MSKApacheKafkaVersion: default: Amazon MSK Apache Kafka version MSKBrokerType: default: Amazon MSK broker type MSKEBSStorageVolumeSize: default: Amazon MSK EBS storage volume size MSKEncryptionInTransit: default: Amazon MSK encryption in transit MSKEncryptionWithinCluster: default: Amazon MSK encryption within cluster MSKEnhancedMonitoring: default: Amazon MSK enhanced monitoring MSKNumberOfBrokerNodes: default: Number of Amazon MSK broker nodes PrivateSubnet1CIDR: default: Private subnet 1 CIDR PrivateSubnet2CIDR: default: Private subnet 2 CIDR PublicSubnet1CIDR: default: Public subnet 1 CIDR QSS3BucketName: default: Quick Start S3 bucket name QSS3BucketRegion: default: Quick Start S3 bucket Region QSS3KeyPrefix: default: Quick Start S3 key prefix VPCCIDR: default: CIDR block of the VPC Parameters: AvailabilityZones: Type: "List" Description: List of Availability Zones to use for the subnets in the VPC. LensesCloudWatchMetrics: Type: String Default: "No" AllowedValues: - "Yes" - "No" Description: Select *Yes* to forward Lenses metrics to CloudWatch. LensesConnectURLs: Type: String Default: "" Description: Connect nodes. For more information, see https://docs.lenses.io/install_setup/configuration/lenses-config.html#kafka-connect[Configuration^]. LensesInstanceType: Type: String Default: t2.large AllowedValues: - t2.large - t2.xlarge - t3.medium - t3.large - t3.xlarge - m4.large - m4.xlarge - m4.2xlarge - m4.4xlarge - m4.10xlarge - m4.16xlarge - m5.large - m5.xlarge - m5.2xlarge - m5.4xlarge - c4.large - c4.xlarge - c4.2xlarge - c4.4xlarge - c4.8xlarge - c5.large - c5.xlarge - c5.2xlarge - c5.4xlarge - c5.9xlarge - r4.large - r4.xlarge - r4.2xlarge - r4.4xlarge - r4.8xlarge - r4.16xlarge Description: Lenses EC2 instance type. For more information, see https://aws.amazon.com/ec2/instance-types/[Amazon EC2 Instance Types^]. LensesSchemaRegistryURLs: Type: String Default: "" Description: URLs for Schema Registry instances that can be used to register or look up schemas. For more information, see https://docs.lenses.io/4.1/configuration/schema-registry/[Schema Registry^]. LensesStorage: Type: String Default: local AllowedValues: - local Description: Storage mode. LensesWebIngressCIDR: Type: String AllowedPattern: (\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2}) ConstraintDescription: Entry must be a valid IP range in CIDR format `x.x.x.x/x`. Description: Enter the inbound IP address range in CIDR format `x.x.x.x/x`. To allow one specific IP address, set to `x.x.x.x/32`. To allow all IP addresses, enter `0.0.0.0/0`. MaxLength: "18" MinLength: "9" MSKApacheKafkaVersion: Type: String Default: "2.6.1" AllowedValues: - "2.2.1" - "2.3.1" - "2.4.1.1" - "2.5.1" - "2.6.0" - "2.6.1" - "2.6.2" - "2.7.0" - "2.8.0" Description: Apache Kafka version on the broker nodes. MSKBrokerType: Type: String Default: kafka.m5.large AllowedValues: - kafka.t3.small - kafka.m5.large - kafka.m5.xlarge - kafka.m5.2xlarge - kafka.m5.4xlarge - kafka.m5.8xlarge - kafka.m5.12xlarge - kafka.m5.16xlarge - kafka.m5.24xlarge Description: Select a Amazon MSK broker type. For more information, see https://aws.amazon.com/msk/pricing/[Amazon MSK pricing]. MSKEBSStorageVolumeSize: Type: Number Default: 100 Description: EBS storage volume per broker, in GiB. After you create the cluster, you can not decrease the storage volume. MaxValue: 16384 MinValue: 1 MSKEncryptionInTransit: Type: String Default: TLS AllowedValues: - TLS Description: Encryption in transit MSKEncryptionWithinCluster: Type: String Default: true Description: Encryption within the cluster MSKEnhancedMonitoring: Type: String Default: PER_TOPIC_PER_BROKER AllowedValues: - DEFAULT - PER_BROKER - PER_TOPIC_PER_BROKER Description: Specifies the level of monitoring for the MSK cluster. MSKNumberOfBrokerNodes: Type: Number Default: 2 Description: The number of broker nodes you want in the Amazon MSK cluster. PrivateSubnet1CIDR: Type: String Default: "10.0.0.0/19" AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$ ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28. Description: CIDR block for private subnet 1, located in Availability Zone 1. PrivateSubnet2CIDR: Type: String Default: "10.0.32.0/19" AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$ ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28. Description: CIDR block for private subnet 2, located in Availability Zone 2. PublicSubnet1CIDR: Type: String Default: "10.0.128.0/20" AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$ ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28. Description: CIDR Block for the public DMZ subnet 1, located in Availability Zone 1 QSS3BucketName: AllowedPattern: '^[0-9a-zA-Z]+([0-9a-zA-Z-]*[0-9a-zA-Z])*$' ConstraintDescription: The Quick Start bucket name can include numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen (-). Default: aws-quickstart Description: Name of the S3 bucket for your copy of the Quick Start assets. Keep the default name unless you are customizing the template. Changing the name updates code references to point to a new Quick Start location. This name can include numbers, lowercase letters, uppercase letters, and hyphens, but do not start or end with a hyphen (-). See https://aws-quickstart.github.io/option1.html. Type: String QSS3BucketRegion: Default: 'us-east-1' Description: 'AWS Region where the Quick Start S3 bucket (QSS3BucketName) is hosted. Keep the default Region unless you are customizing the template. Changing this Region updates code references to point to a new Quick Start location. When using your own bucket, specify the Region. See https://aws-quickstart.github.io/option1.html.' Type: String QSS3KeyPrefix: AllowedPattern: '^[0-9a-zA-Z-/]*$' ConstraintDescription: The Quick Start S3 key prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slashes (/). The prefix should end with a forward slash (/). Default: quickstart-lenses/ Description: S3 key prefix that is used to simulate a directory for your copy of the Quick Start assets. Keep the default prefix unless you are customizing the template. Changing this prefix updates code references to point to a new Quick Start location. This prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slashes (/). End with a forward slash. See https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html and https://aws-quickstart.github.io/option1.html. Type: String VPCCIDR: Type: String Default: "10.0.0.0/16" AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$ ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/16-28. Description: CIDR Block for the VPC. Conditions: UsingDefaultBucket: !Equals [!Ref QSS3BucketName, 'aws-quickstart'] Resources: VPCStack: Type: "AWS::CloudFormation::Stack" Properties: Parameters: AvailabilityZones: !Join - "," - Ref: AvailabilityZones NumberOfAZs: "2" PrivateSubnet1ACIDR: !Ref PrivateSubnet1CIDR PrivateSubnet2ACIDR: !Ref PrivateSubnet2CIDR PublicSubnet1CIDR: !Ref PublicSubnet1CIDR VPCCIDR: !Ref VPCCIDR TemplateURL: !Sub - "https://${S3Bucket}.s3.${S3Region}.${AWS::URLSuffix}/${QSS3KeyPrefix}submodules/quickstart-aws-vpc/templates/aws-vpc.template.yaml" - S3Region: !If [UsingDefaultBucket, !Ref 'AWS::Region', !Ref QSS3BucketRegion] S3Bucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName] MSKStack: Type: "AWS::CloudFormation::Stack" Properties: Parameters: ApacheKafkaVersion: !Ref MSKApacheKafkaVersion BrokerType: !Ref MSKBrokerType ClusterName: !Join - "-" - - quickstart-cluster - "Fn::Select": - 4 - "Fn::Split": - "-" - "Fn::Select": - 2 - "Fn::Split": - / - Ref: "AWS::StackId" EBSStorageVolumeSize: !Ref MSKEBSStorageVolumeSize EncryptionInTransit: !Ref MSKEncryptionInTransit EncryptionWithinCluster: !Ref MSKEncryptionWithinCluster EnhancedMonitoring: !Ref MSKEnhancedMonitoring NumberOfBrokerNodes: !Ref MSKNumberOfBrokerNodes PrivateSubnetOneID: !GetAtt VPCStack.Outputs.PrivateSubnet1AID PrivateSubnetTwoID: !GetAtt VPCStack.Outputs.PrivateSubnet2AID VPCID: !GetAtt VPCStack.Outputs.VPCID TemplateURL: !Sub - "https://${S3Bucket}.s3.${S3Region}.${AWS::URLSuffix}/${QSS3KeyPrefix}templates/msk.template.yaml" - S3Region: !If [UsingDefaultBucket, !Ref 'AWS::Region', !Ref QSS3BucketRegion] S3Bucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName] LensesStack: Type: "AWS::CloudFormation::Stack" Properties: Parameters: CloudWatchMetrics: !Ref LensesCloudWatchMetrics ConnectURLs: !Ref LensesConnectURLs InstanceType: !Ref LensesInstanceType LensesLocation: !Ref LensesWebIngressCIDR LensesStorage: !Ref LensesStorage MSKClusterARN: !GetAtt MSKStack.Outputs.MSKClusterArn MSKSecretDecryptionKMSKey: "" MSKSecretName: "" MSKSecurityGroup: !GetAtt MSKStack.Outputs.MSKSecurityGroupID SchemaRegistryURLs: !Ref LensesSchemaRegistryURLs StoragePostgresDatabase: "" StoragePostgresHostname: "" StoragePostgresPassword: "" StoragePostgresUsername: "" SubnetId: !GetAtt VPCStack.Outputs.PublicSubnet1ID VPC: !GetAtt VPCStack.Outputs.VPCID TemplateURL: !Sub - "https://${S3Bucket}.s3.${S3Region}.${AWS::URLSuffix}/${QSS3KeyPrefix}templates/lenses-io.template.yaml" - S3Region: !If [UsingDefaultBucket, !Ref 'AWS::Region', !Ref QSS3BucketRegion] S3Bucket: !If [UsingDefaultBucket, !Sub '${QSS3BucketName}-${AWS::Region}', !Ref QSS3BucketName] Outputs: FQDN: Description: Fully qualified domain name (FQDN) for the Lenses EC2 instance. Value: !GetAtt 'LensesStack.Outputs.FQDN' Postdeployment: Description: See the deployment guide for post-deployment steps. Value: https://aws.amazon.com/quickstart/?quickstart-all.sort-by=item.additionalFields.sortDate&quickstart-all.sort-order=desc&awsm.page-quickstart-all=5