AWSTemplateFormatVersion : '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: > This template deploys a CodePipeline with its required resources. The following stages are predefined in this template: - Source - UpdatePipeline - UnitTests - BuildAndPackage (MainGitBranch only) - DeployTest (MainGitBranch only) - IntegrationTests - DeployProd (MainGitBranch only) **WARNING** You will be billed for the AWS resources used if you create a stack from this template. # To deploy this template and connect to the main git branch, run this against the leading account: # `sam deploy --config-file pipeline/samconfig-pipeline.toml --config-env pipeline`. Metadata: cfn-lint: config: ignore_checks: - I3011 - I3013 Parameters: ProjectSubfolder: Type: String Default: "{{cookiecutter.monorepo_project_subfolder}}" CodeCommitRepositoryName: Type: String Default: "{{cookiecutter.codecommit_repository_name}}" MainGitBranch: Type: String Default: "{{cookiecutter.main_git_branch}}" MonorepoSsmPrefix: Type: String Default: "{{cookiecutter.monorepo_ssm_prefix}}" CodeBuildImage: Type: String Default: aws/codebuild/amazonlinux2-x86_64-standard:5.0 # Stage 1 Parameters TestConfigEnvName: Type: String Default: "{{cookiecutter.testing_stage_name}}" TestingPipelineExecutionRole: Type: String Default: "{{cookiecutter.testing_pipeline_execution_role}}" TestingCloudFormationExecutionRole: Type: String Default: "{{cookiecutter.testing_cloudformation_execution_role}}" # Stage 2 Parameters ProdConfigEnvName: Type: String Default: "{{cookiecutter.prod_stage_name}}" ProdPipelineExecutionRole: Type: String Default: "{{cookiecutter.prod_pipeline_execution_role}}" ProdCloudFormationExecutionExeRole: Type: String Default: "{{cookiecutter.prod_cloudformation_execution_role}}" Resources: # ____ _ _ _ # | _ \(_)_ __ ___| (_)_ __ ___ # | |_) | | '_ \ / _ | | | '_ \ / _ \ # | __/| | |_) | __| | | | | | __/ # |_| |_| .__/ \___|_|_|_| |_|\___| # |_| Pipeline: Type: AWS::CodePipeline::Pipeline Properties: ArtifactStore: Location: !Ref PipelineArtifactsBucket Type: S3 RoleArn: !GetAtt CodePipelineExecutionRole.Arn RestartExecutionOnUpdate: false Stages: - Name: Source Actions: - Name: SourceCodeRepo ActionTypeId: Version: "1" Category: Source Owner: AWS Provider: CodeCommit Configuration: BranchName: !Ref MainGitBranch RepositoryName: !Ref CodeCommitRepositoryName PollForSourceChanges: false OutputArtifacts: - Name: SourceCodeAsZip RunOrder: 1 - Name: UpdatePipeline Actions: - Name: CreateChangeSet ActionTypeId: Category: Deploy Owner: AWS Provider: CloudFormation Version: "1" Configuration: ActionMode: CHANGE_SET_REPLACE RoleArn: !GetAtt PipelineStackCloudFormationExecutionRole.Arn StackName: !Ref AWS::StackName ChangeSetName: !Sub ${AWS::StackName}-ChangeSet TemplatePath: !Sub "SourceCodeAsZip::${ProjectSubfolder}/codepipeline.yaml" Capabilities: CAPABILITY_NAMED_IAM InputArtifacts: - Name: SourceCodeAsZip RunOrder: 1 - Name: ExecuteChangeSet ActionTypeId: Category: Deploy Owner: AWS Provider: CloudFormation Version: "1" Configuration: ActionMode: CHANGE_SET_EXECUTE RoleArn: !GetAtt PipelineStackCloudFormationExecutionRole.Arn StackName: !Ref AWS::StackName ChangeSetName: !Sub ${AWS::StackName}-ChangeSet OutputArtifacts: - Name: !Sub ${AWS::StackName}ChangeSet RunOrder: 2 - Name: UnitTest Actions: - Name: UnitTest ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: "1" Configuration: ProjectName: !Ref CodeBuildProjectUnitTest InputArtifacts: - Name: SourceCodeAsZip - Name: BuildAndPackage Actions: - Name: CodeBuild ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: "1" Configuration: ProjectName: !Ref CodeBuildProjectBuildAndPackage InputArtifacts: - Name: SourceCodeAsZip OutputArtifacts: - Name: BuildArtifactAsZip - Name: DeployTest Actions: - Name: DeployTest ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: "1" Configuration: ProjectName: !Ref CodeBuildProjectDeploy EnvironmentVariables: !Sub | [ {"name": "ENV_CONFIG_NAME", "value": "${TestConfigEnvName}"}, {"name": "ENV_PIPELINE_EXECUTION_ROLE", "value": "${TestingPipelineExecutionRole}"}, {"name": "ENV_CLOUDFORMATION_EXECUTION_ROLE", "value": "${TestingCloudFormationExecutionRole}"} ] InputArtifacts: - Name: BuildArtifactAsZip RunOrder: 1 - Name: IntegrationTest ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: "1" Configuration: ProjectName: !Ref CodeBuildProjectIntegrationTest EnvironmentVariables: !Sub | [ {"name": "ENV_PIPELINE_EXECUTION_ROLE", "value": "${TestingPipelineExecutionRole}"} ] InputArtifacts: - Name: SourceCodeAsZip RunOrder: 2 - Name: DeployProd Actions: # uncomment this to have a manual approval step before deployment to production # - Name: ManualApproval # ActionTypeId: # Category: Approval # Owner: AWS # Provider: Manual # Version: "1" # RunOrder: 1 - Name: DeployProd ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: "1" RunOrder: 2 # keeping run order as 2 in case manual approval is enabled Configuration: ProjectName: !Ref CodeBuildProjectDeploy EnvironmentVariables: !Sub | [ {"name": "ENV_CONFIG_NAME", "value": "${ProdConfigEnvName}"}, {"name": "ENV_PIPELINE_EXECUTION_ROLE", "value": "${ProdPipelineExecutionRole}"}, {"name": "ENV_CLOUDFORMATION_EXECUTION_ROLE", "value": "${ProdCloudFormationExecutionExeRole}"} ] InputArtifacts: - Name: BuildArtifactAsZip PipelineArtifactsBucket: Type: AWS::S3::Bucket DeletionPolicy: Retain UpdateReplacePolicy: Retain Properties: VersioningConfiguration: Status: Enabled LoggingConfiguration: DestinationBucketName: !Ref PipelineArtifactsLoggingBucket LogFilePrefix: "artifacts-logs" BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: AES256 PipelineArtifactsBucketPolicy: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref PipelineArtifactsBucket PolicyDocument: Statement: - Effect: Deny Action: s3:* Principal: "*" Resource: - !Sub "${PipelineArtifactsBucket.Arn}/*" - !GetAtt PipelineArtifactsBucket.Arn Condition: Bool: aws:SecureTransport: false - Effect: Allow Action: s3:* Principal: AWS: - !GetAtt CodePipelineExecutionRole.Arn Resource: - !Sub arn:${AWS::Partition}:s3:::${PipelineArtifactsBucket} - !Sub arn:${AWS::Partition}:s3:::${PipelineArtifactsBucket}/* PipelineArtifactsLoggingBucket: Type: AWS::S3::Bucket DeletionPolicy: Retain UpdateReplacePolicy: Retain Properties: AccessControl: "LogDeliveryWrite" OwnershipControls: Rules: - ObjectOwnership: ObjectWriter VersioningConfiguration: Status: Enabled BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: AES256 PipelineArtifactsLoggingBucketPolicy: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref PipelineArtifactsLoggingBucket PolicyDocument: Statement: - Effect: Deny Action: s3:* Principal: "*" Resource: - !Sub "${PipelineArtifactsLoggingBucket.Arn}/*" - !GetAtt PipelineArtifactsLoggingBucket.Arn Condition: Bool: aws:SecureTransport: false CodePipelineExecutionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - sts:AssumeRole Principal: Service: - codepipeline.amazonaws.com Policies: - PolicyName: CodePipelineAccess PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - "iam:PassRole" Resource: "*" - PolicyName: CodeCommitAccess PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - codecommit:CancelUploadArchive - codecommit:GetBranch - codecommit:GetCommit - codecommit:GetUploadArchiveStatus - codecommit:UploadArchive Resource: !Sub "arn:${AWS::Partition}:codecommit:${AWS::Region}:${AWS::AccountId}:${CodeCommitRepositoryName}" - PolicyName: CodePipelineCodeAndS3Bucket PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - s3:GetBucketAcl - s3:GetBucketLocation Resource: - !GetAtt PipelineArtifactsBucket.Arn - Effect: Allow Action: - s3:GetObject - s3:GetObjectVersion - s3:PutObject Resource: Fn::Sub: ${PipelineArtifactsBucket.Arn}/* - PolicyName: CodePipelineCodeBuildAndCloudformationAccess PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - codebuild:StartBuild - codebuild:BatchGetBuilds Resource: - !GetAtt CodeBuildProjectBuildAndPackage.Arn - !GetAtt CodeBuildProjectUnitTest.Arn - !GetAtt CodeBuildProjectIntegrationTest.Arn - !GetAtt CodeBuildProjectDeploy.Arn - Effect: Allow Action: - cloudformation:CreateStack - cloudformation:DescribeStacks - cloudformation:DeleteStack - cloudformation:UpdateStack - cloudformation:CreateChangeSet - cloudformation:ExecuteChangeSet - cloudformation:DeleteChangeSet - cloudformation:DescribeChangeSet - cloudformation:SetStackPolicy - cloudformation:SetStackPolicy - cloudformation:ValidateTemplate Resource: - !Sub "arn:${AWS::Partition}:cloudformation:${AWS::Region}:${AWS::AccountId}:stack/${AWS::StackName}/*" # PipelineStackCloudFormationExecutionRole is used for the pipeline to self mutate PipelineStackCloudFormationExecutionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: Effect: Allow Action: sts:AssumeRole Principal: Service: cloudformation.amazonaws.com Policies: - PolicyName: GrantCloudFormationFullAccess PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: "*" Resource: "*" # ____ _ ____ _ _ _ # / ___|___ __| | ___| __ ) _ _(_| | __| | # | | / _ \ / _` |/ _ | _ \| | | | | |/ _` | # | |__| (_) | (_| | __| |_) | |_| | | | (_| | # \____\___/ \__,_|\___|____/ \__,_|_|_|\__,_| CodeBuildServiceRole: Type: AWS::IAM::Role Properties: Tags: - Key: Role Value: aws-sam-pipeline-codebuild-service-role AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: sts:AssumeRole Principal: Service: - codebuild.amazonaws.com Policies: - PolicyName: CodeBuildLogs PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - logs:CreateLogGroup - logs:CreateLogStream - logs:PutLogEvents Resource: - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/codebuild/*" - PolicyName: CodeBuildArtifactsBucket PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: - s3:GetObject - s3:GetObjectVersion - s3:PutObject Resource: - !Sub "arn:${AWS::Partition}:s3:::${PipelineArtifactsBucket}/*" - PolicyName: AssumeStagePipExecutionRoles PolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Action: sts:AssumeRole Resource: "*" Condition: StringEquals: aws:ResourceTag/Role: pipeline-execution-role CodeBuildProjectUnitTest: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Environment: Type: LINUX_CONTAINER ComputeType: BUILD_GENERAL1_SMALL Image: !Ref CodeBuildImage EnvironmentVariables: - Name: PROJECT_SUBFOLDER Value: !Ref ProjectSubfolder ServiceRole: !GetAtt CodeBuildServiceRole.Arn Source: Type: CODEPIPELINE BuildSpec: !Sub "${ProjectSubfolder}/pipeline/buildspec_unit_test.yml" CodeBuildProjectBuildAndPackage: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Environment: Type: LINUX_CONTAINER ComputeType: BUILD_GENERAL1_SMALL Image: !Ref CodeBuildImage PrivilegedMode: true EnvironmentVariables: - Name: PROJECT_SUBFOLDER Value: !Ref ProjectSubfolder - Name: TESTING_PIPELINE_EXECUTION_ROLE Value: !Ref TestingPipelineExecutionRole - Name: TESTING_ENV_CONFIG_NAME Value: !Ref TestConfigEnvName - Name: PROD_PIPELINE_EXECUTION_ROLE Value: !Ref ProdPipelineExecutionRole - Name: PROD_ENV_CONFIG_NAME Value: !Ref ProdConfigEnvName ServiceRole: !GetAtt CodeBuildServiceRole.Arn Source: Type: CODEPIPELINE BuildSpec: !Sub "${ProjectSubfolder}/pipeline/buildspec_build_package.yml" CodeBuildProjectIntegrationTest: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Environment: Type: LINUX_CONTAINER ComputeType: BUILD_GENERAL1_SMALL Image: !Ref CodeBuildImage EnvironmentVariables: - Name: PROJECT_SUBFOLDER Value: !Ref ProjectSubfolder ServiceRole: !GetAtt CodeBuildServiceRole.Arn Source: Type: CODEPIPELINE BuildSpec: !Sub "${ProjectSubfolder}/pipeline/buildspec_integration_test.yml" CodeBuildProjectDeploy: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Environment: Type: LINUX_CONTAINER ComputeType: BUILD_GENERAL1_SMALL Image: !Ref CodeBuildImage EnvironmentVariables: - Name: PROJECT_SUBFOLDER Value: !Ref ProjectSubfolder ServiceRole: !GetAtt CodeBuildServiceRole.Arn Source: Type: CODEPIPELINE BuildSpec: !Sub "${ProjectSubfolder}/pipeline/buildspec_deploy.yml" # _____ __________ # / \ ____ ____ ____\______ \ ____ ______ ____ # / \ / \ / _ \ / \ / _ \| _// __ \\____ \ / _ \ # / Y ( <_> ) | ( <_> ) | \ ___/| |_> > <_> ) # \____|__ /\____/|___| /\____/|____|_ /\___ > __/ \____/ # \/ \/ \/ \/|__| MonorepoTriggerFunction: Type: AWS::Serverless::Function Properties: InlineCode: | import os, boto3 from aws_lambda_powertools.logging import Logger from aws_lambda_powertools.utilities import parameters logger = Logger() cc = boto3.client('codecommit') cp = boto3.client('codepipeline') ssm = boto3.client('ssm') ssm_prefix = os.environ.get('SSM_PREFIX', '') app_dir = os.environ.get('APP_DIR_IN_MONOREPO') pipeline_name = os.environ.get('PIPELINE_NAME') @logger.inject_lambda_context def lambda_handler(event, _): commit_id = event['detail']['commitId'] branch_name = event['detail']['referenceName'] repository = event['detail']['repositoryName'] paths = get_modified_files_since_last_run(repository, commit_id, branch_name) toplevel_dirs = get_unique_toplevel_dirs(paths) logger.info({'paths': paths, 'toplevel_dirs': toplevel_dirs, 'pipeline_name': pipeline_name}) if app_dir in toplevel_dirs: start_codepipeline(pipeline_name) else: logger.info('Not triggering Pipeline %s. No changes under App dir %s', pipeline_name, app_dir) update_last_commit(repository, commit_id, branch_name) def get_unique_toplevel_dirs(modified_files): toplevel_dirs = set(filter(lambda a: len(a) > 1, [file.split('/')[0] for file in modified_files])) logger.info('toplevel dirs: %s', toplevel_dirs) return list(toplevel_dirs) def start_codepipeline(codepipeline_name): try: cp.start_pipeline_execution(name=codepipeline_name) logger.info(f'Started CodePipeline {codepipeline_name}.') except cp.exceptions.PipelineNotFoundException: logger.info(f'Could not find CodePipeline {codepipeline_name}.') return False return True def param_name_last_commit(repository, branch_name): return os.path.join('/', ssm_prefix, repository, branch_name, app_dir, 'LastCommit') def get_last_commit(repository, commit_id, branch_name): param_name = param_name_last_commit(repository, branch_name) try: return parameters.get_parameter(param_name, force_fetch=True) except parameters.GetParameterError: logger.info('not found ssm parameter %s', param_name) commit = cc.get_commit(repositoryName=repository, commitId=commit_id)['commit'] parent = commit['parents'][0] if commit['parents'] else None return parent def update_last_commit(repository, commit_id, branch_name): ssm.put_parameter(Name=param_name_last_commit(repository, branch_name), Description='Keep track of the last commit already triggered', Value=commit_id, Type='String', Overwrite=True) def get_modified_files_since_last_run(repo_name, base_commit, branch_name): last_commit = get_last_commit(repo_name, base_commit, branch_name) diff = cc.get_differences(repositoryName=repo_name, beforeCommitSpecifier=last_commit, afterCommitSpecifier=base_commit ).get('differences', None) logger.info(f"last_commit: '{last_commit}' - base_commit: '{base_commit}'") logger.info(f"diff: {diff}") before_blob_paths = {d.get('beforeBlob', {}).get('path') for d in diff} after_blob_paths = {d.get('afterBlob', {}).get('path') for d in diff} all_modifications = before_blob_paths.union(after_blob_paths) return list(filter(lambda f: f is not None, all_modifications)) Handler: index.lambda_handler Runtime: python3.9 Timeout: 30 Layers: - !Sub arn:${AWS::Partition}:lambda:${AWS::Region}:017000801446:layer:AWSLambdaPowertoolsPython:22 Architectures: - x86_64 Environment: Variables: SSM_PREFIX: !Ref MonorepoSsmPrefix PIPELINE_NAME: !Ref Pipeline APP_DIR_IN_MONOREPO: !Ref ProjectSubfolder Policies: - CodeCommitReadPolicy: RepositoryName: !Ref CodeCommitRepositoryName - SSMParameterReadPolicy: ParameterName: !Sub "${MonorepoSsmPrefix}/*" - Statement: - Effect: Allow Action: ssm:PutParameter Resource: !Sub "arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${MonorepoSsmPrefix}/*" - Statement: - Effect: Allow Action: codepipeline:StartPipelineExecution Resource: !Sub "arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:*" Events: EBRule: Type: EventBridgeRule Properties: DeadLetterConfig: Type: SQS Pattern: source: [aws.codecommit] detail-type: ['CodeCommit Repository State Change'] resources: [!Sub "arn:${AWS::Partition}:codecommit:${AWS::Region}:${AWS::AccountId}:${CodeCommitRepositoryName}"] detail: event: - referenceCreated - referenceUpdated referenceType: - branch referenceName: - !Ref MainGitBranch Outputs: CodePipelineUrl: Value: !Sub "https://${AWS::Region}.console.aws.amazon.com/codesuite/codepipeline/pipelines/${Pipeline}/view?region=${AWS::Region}" CodePipelineExecutionRoleArn: Value: !GetAtt CodePipelineExecutionRole.Arn CodeBuildServiceRoleArn: Value: !GetAtt CodeBuildServiceRole.Arn PipelineArtifactsBucketName: Value: !Ref PipelineArtifactsBucket PipelineArtifactsLoggingBucketName: Value: !Ref PipelineArtifactsLoggingBucket