--- AWSTemplateFormatVersion: '2010-09-09' Description: >- (WWPS-GLS-WF-CROMWELL-AIO) Creates the complete set of resources needed to run genomics workflows using Cromwell on AWS Batch Mappings: TagMap: default: architecture: "genomics-workflows" solution: "cromwell" tags: - Key: "architecture" Value: "genomics-workflows" - Key: "solution" Value: "cromwell" Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: "VPC" Parameters: - Namespace - VpcId - SubnetIds - ServerSubnetID - DBSubnetIDs - Label: default: "Data Storage" Parameters: - S3DataBucketName - ExistingDataBucket - Label: default: "AWS Batch" Parameters: - DefaultCEMinvCpus - DefaultCEMaxvCpus - PriorityCEMinvCpus - PriorityCEMaxvCpus - Label: default: "Cromwell Resources" Parameters: - CromwellVersion - CromwellVersionSpecified - CromwellJarUrl - S3DataBucketARNs - DBUsername - DBPassword ParameterLabels: Namespace: default: Namespace VpcId: default: VPC ID SubnetIds: default: VPC Subnet IDs ServerSubnetID: default: "Server Subnet ID" S3DataBucketName: default: S3 Data Bucket Name ExistingDataBucket: default: Existing Data Bucket? DefaultCEMinvCpus: default: Default Min vCPU DefaultCEMaxvCpus: default: Default Max vCPU PriorityCEMinvCpus: default: High Priority Min vCPU PriorityCEMaxvCpus: default: High Priority Max vCPU S3DataBucketARNs: default: "S3 Open Data Bucket ARNs" CromwellVersion: default: "Cromwell Version" CromwellVersionSpecified: default: "Cromwell Version Specified" DBSubnetIDs: default: "Database Subnet IDs" DBUsername: default: "Cromwell Database Username" DBPassword: default: "Cromwell Database Password" TemplateRootUrl: default: Template Root URL Parameters: Namespace: Type: String Description: Namespace (e.g. project name) to use to label resources. Between 4-20 letters, digits, and '-', starting with a letter. Default: "" VpcId: Type: AWS::EC2::VPC::Id Description: 'The VPC to create security groups and deploy AWS Batch to. NOTE: Must be the same VPC as the provided subnet IDs.' SubnetIds: Type: List Description: 'Subnets you want your batch compute environment to launch in. We recommend private subnets. NOTE: Must be from the VPC provided.' ServerSubnetID: Type: AWS::EC2::Subnet::Id Description: Subnet for the Cromwell server. For public access, use a public subnet. S3DataBucketName: Description: >- A S3 bucket name for storing analysis results The bucket name must respect the S3 bucket naming conventions (can contain lowercase letters, numbers, periods and hyphens). If left blank a unique bucket name will be generated. Type: String Default: "" AllowedPattern: "((?=^.{3,63}$)(?!^(\\d+\\.)+\\d+$)(^(([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])\\.)*([a-z0-9]|[a-z0-9][a-z0-9\\-]*[a-z0-9])$)|(^.{0}$))" ConstraintDescription: "Must respect S3 bucket naming conventions" ExistingDataBucket: Description: Does this data bucket already exist? Type: String AllowedValues: - "Yes" - "No" Default: "No" DefaultCEMinvCpus: Description: The minimum number of CPUs for the default Batch Compute Environment Type: Number Default: 0 DefaultCEMaxvCpus: Description: The maximum number of CPUs for the default Batch Compute Environment Type: Number Default: 100 PriorityCEMinvCpus: Description: The minimum number of CPUs for the high-priority Batch Compute Environment Type: Number Default: 0 PriorityCEMaxvCpus: Description: The maximum number of CPUs for the high-priority Batch Compute Environment Type: Number Default: 100 CromwellVersion: Type: String Description: >- Version of Cromwell to install. "latest" will retrieve the currently released version of Cromwell from Github. AllowedValues: - latest - specify ... Default: latest CromwellVersionSpecified: Type: String Description: >- Specific version of Cromwell to install. Must match a released version number. For example, 52, 52.1, etc. The minimum supported version is 52. Ignored if "Cromwell Version" is set to "latest". Default: "" CromwellJarUrl: Type: String Description: >- URL to a pre-built cromwell-*.jar file. Example: https://mycicdserver.com/build/cromwell-XX-SNAP.jar. If this is specifed, CromwellVersion is ignored. Default: "" DBSubnetIDs: Type: List Description: Minimum 2 subnets for the Cromwell Database. Private subnets are recommended. DBUsername: Type: String Description: >- The master username for the Aurora MySQL RDS cluster that will be used as Cromwell's database Default: cromwell MaxLength: 16 MinLength: 1 AllowedPattern: "^[a-zA-Z]{1}[a-zA-Z0-9]{0,15}$" ConstraintDescription: >- DBUsername must have 1 to 16 alphanumeric characters. First character must be a letter DBPassword: Type: String NoEcho: true Description: >- The master password for the Aurora MySQL RDS cluster that will be used as Cromwell's database MinLength: 8 AllowedPattern: '^[ -~][^"@/]{7,}$' ConstraintDescription: >- DBPassword must have at least 8 printable ASCII characters. Can't contain any of the following: / (slash), "(double quote) and @ (at sign). ArtifactBucketName: Type: String Default: aws-genomics-workflows Description: >- S3 Bucket where distrubution artifacts and additions scripts are stored ArtifactBucketPrefix: Type: String Default: artifacts # dist: pin_version Description: >- Prefix in ArtifactBucketName where distribution artifacts and additions scripts are stored S3DataBucketARNs: Description: Open datasets on AWS S3 for workflow inputs Type: CommaDelimitedList Default: "arn:aws:s3:::gatk-test-data/*,arn:aws:s3:::broad-references/*" TemplateRootUrl: Type: String Description: >- Root URL for where nested templates are stored Default: https://aws-genomics-workflows.s3.amazonaws.com/templates ConstraintDescription: >- Must be a valid S3 URL AllowedPattern: "https://[a-z0-9-./]{0,}s3([a-z0-9-.]+)*\\.amazonaws\\.com/[a-z0-9-./]{3,}" Conditions: NoNamespace: !Equals [ !Ref Namespace, "" ] Resources: GenomicsWorkflowStack: Type: AWS::CloudFormation::Stack Properties: TemplateURL: !Sub ${TemplateRootUrl}/gwfcore/gwfcore-root.template.yaml Parameters: Namespace: Fn::If: - NoNamespace - !Sub ${AWS::StackName} - !Ref Namespace VpcId: !Ref VpcId SubnetIds: !Join [",", !Ref SubnetIds ] S3BucketName: !Ref S3DataBucketName ExistingBucket: !Ref ExistingDataBucket DefaultCEMinvCpus: !Ref DefaultCEMinvCpus DefaultCEMaxvCpus: !Ref DefaultCEMaxvCpus PriorityCEMinvCpus: !Ref PriorityCEMinvCpus PriorityCEMaxvCpus: !Ref PriorityCEMaxvCpus ArtifactBucketName: !Ref ArtifactBucketName ArtifactBucketPrefix: !Ref ArtifactBucketPrefix TemplateRootUrl: !Ref TemplateRootUrl Tags: !FindInMap ["TagMap", "default", "tags"] CromwellStack: Type: AWS::CloudFormation::Stack DependsOn: GenomicsWorkflowStack Properties: TemplateURL: !Sub ${TemplateRootUrl}/cromwell/cromwell-resources.template.yaml Parameters: Namespace: Fn::If: - NoNamespace - !Sub ${AWS::StackName}-cromwell - !Sub ${Namespace}-cromwell GWFCoreNamespace: Fn::If: - NoNamespace - !Sub ${AWS::StackName} - !Ref Namespace VpcId: !Ref VpcId ServerSubnetID: !Ref ServerSubnetID CromwellVersion: !Ref CromwellVersion CromwellVersionSpecified: !Ref CromwellVersionSpecified CromwellJarUrl: !Ref CromwellJarUrl S3DataBucketARNs: !Join [",", !Ref S3DataBucketARNs] DBUsername: !Ref DBUsername DBPassword: !Ref DBPassword DBSubnetIDs: !Join [",", !Ref DBSubnetIDs] Tags: !FindInMap ["TagMap", "default", "tags"] Outputs: CromwellVersion: Value: !GetAtt CromwellStack.Outputs.CromwellVersion Export: Name: !Sub "${Namespace}-CromwellVersion" EC2Instance: Value: !GetAtt CromwellStack.Outputs.EC2Instance Export: Name: !Sub "${Namespace}-EC2Instance" Description: >- The EC2 Instance ID of your Cromwell Server PublicIp: Value: !GetAtt CromwellStack.Outputs.PublicIp Export: Name: !Sub "${Namespace}-PublicIp" Description: >- Cromwell server public IP address HostName: Value: !GetAtt CromwellStack.Outputs.HostName Export: Name: !Sub "${Namespace}-HostName" Description: >- Cromwell server public DNS name CloudWatchLogStream: Value: !GetAtt CromwellStack.Outputs.CloudWatchLogStream Export: Name: !Sub "${Namespace}-CloudWatchLogStream" Description: >- Cromwell server log stream VpcId: Description: >- The VPC created for your Nextflow stack. Value: !Ref VpcId S3DataBucket: Value: !GetAtt 'GenomicsWorkflowStack.Outputs.S3BucketName' Export: Name: !Sub "${Namespace}-DataBucket" Description: >- S3 bucket for storing genomics workflow input and output data BatchDefaultQueue: Value: !GetAtt 'GenomicsWorkflowStack.Outputs.DefaultJobQueueArn' Export: Name: !Sub "${Namespace}-DefaultJobQueue" Description: >- The default AWS Batch job queue for workflow jobs, based on EC2 SPOT instances BatchPriorityQueue: Value: !GetAtt 'GenomicsWorkflowStack.Outputs.PriorityJobQueueArn' Export: Name: !Sub "${Namespace}-PriorityJobQueue" Description: >- AWS Batch job queue for high priority workflow jobs, based on EC2 On-Demand instances ...