# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: MIT-0 # # AWS infrastructure to copy RDS DB Snapshots off to a separate DR account. # This template is for the CONSUMER of the snapshots ie. the DR account. # AWSTemplateFormatVersion: "2010-09-09" Description: "For receiving (DR) copies of DB Snapshots from another account" Parameters: CodeBucket: Description: The S3Bucket that stores the Lambdas Default: draco Type: String CodePrefix: Description: The S3 prefix within the bucket (may or may not have a trailing slash!) AllowedPattern: ^[0-9a-zA-Z-/]*$ Default: draco/ Type: String DeploymentTimestamp: Description: The time of deploy used to trigger the fetch of the latest code Default: 2000-01-01T00:00:00 Type: String SourceAcct: Description: The account that shares the snapshot copies to be DRd Default: 111111111111 Type: String DrTagKey: Description: The tag key that identifies snapshot transit copies made for DR Default: DRACO Type: String DrTagValue: Description: The tag value that identifies snapshot transit copies made for DR Default: SnapshotCopy Type: String Resources: DracoRegionalBucket: Type: AWS::S3::Bucket Properties: BucketName: !Sub "draco-${AWS::AccountId}-${AWS::Region}" VersioningConfiguration: Status: Enabled DracoBucketPolicy: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref DracoRegionalBucket PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: AWS: !Ref SourceAcct Action: - s3:GetObject - s3:GetObjectVersion Resource: - !Sub 'arn:aws:s3:::${DracoRegionalBucket}/${CodePrefix}*' CopyObjects: Type: Custom::CopyObjects Properties: DeploymentTimestamp: !Ref DeploymentTimestamp ServiceToken: !GetAtt CopyObjectsFunction.Arn DestBucket: !Ref DracoRegionalBucket SourceBucket: !Ref CodeBucket PhysicalResourceId: DracoFiles Prefix: !Ref CodePrefix Objects: - consumer.zip - consumer.yaml - wait4copy.zip - wait4copy.yaml - producer.zip - producer.yaml CopyObjectsRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: lambda.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole Path: / Policies: - PolicyName: lambda-copier PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Action: - s3:GetObject Resource: - !Sub 'arn:aws:s3:::${CodeBucket}/${CodePrefix}*' - Effect: Allow Action: - s3:GetObject - s3:PutObject - s3:DeleteObject Resource: - !Sub 'arn:aws:s3:::${DracoRegionalBucket}/${CodePrefix}*' - Effect: Allow Action: - s3:ListBucket Resource: - !Sub 'arn:aws:s3:::${DracoRegionalBucket}' # Copies objects from a source S3 bucket to a destination if they don't already exist. # It returns the S3 object versions. The main use for this is as the S3ObjectVersion # parameter in Lambda functions defined elsewhere in the template, allowing the code to # be updated, or not depending whether a new version has been uploaded. CopyObjectsFunction: Type: AWS::Lambda::Function Properties: Description: Copies objects from a source S3 bucket to a destination Handler: index.handler Runtime: python3.7 Role: !GetAtt CopyObjectsRole.Arn Timeout: 240 Code: ZipFile: | import boto3 import botocore import cfnresponse import logging import threading from datetime import datetime logging.getLogger().setLevel(logging.INFO) def copy_objects(src_bucket, dst_bucket, prefix, objects): s3 = boto3.client('s3') versions = {} etags = {} for o in objects: try: rsp = s3.head_object(Bucket=dst_bucket, Key=prefix+o) versions[o] = rsp['VersionId'] etags[o] = rsp['ETag'] except botocore.exceptions.ClientError as error: if src_bucket == dst_bucket: raise error versions[o] = None etags[o] = '-' logging.debug(f'{dst_bucket}/{prefix+o}: version={versions[o]}, etag={etags[o]}') if src_bucket != dst_bucket: try: crsp = s3.copy_object(CopySource=f'{src_bucket}/{prefix+o}', Bucket=dst_bucket, Key=prefix+o, CopySourceIfNoneMatch=etags[o]) versions[o] = crsp['VersionId'] logging.debug(f'{dst_bucket}/{prefix+o}: copied version={versions[o]}') except botocore.exceptions.ClientError as error: logging.debug(f'{dst_bucket}/{prefix+o}: up-to-date') if error.response['Error']['Code'] != 'PreconditionFailed': raise error return versions def timeout(event, context): logging.error('Execution is about to time out, sending failure response to CloudFormation') cfnresponse.send(event, context, cfnresponse.FAILED, {}, None) def handler(event, context): # make sure we send a failure to CloudFormation if the function # is going to timeout timer = threading.Timer((context.get_remaining_time_in_millis() / 1000.00) - 0.5, timeout, args=[event, context]) timer.start() status = cfnresponse.SUCCESS responseData = {} try: src_bucket = event['ResourceProperties']['SourceBucket'] dst_bucket = event['ResourceProperties']['DestBucket'] physical_resource_id = event['ResourceProperties']['PhysicalResourceId'] prefix = event['ResourceProperties']['Prefix'] objects = event['ResourceProperties']['Objects'] if event['RequestType'] != 'Delete': # Leave the objects in the bucket versions = copy_objects(src_bucket, dst_bucket, prefix, objects) responseData = versions logging.info(f'Bucket: {src_bucket} has versions: {versions}') except Exception as e: logging.error(f'Exception: {e}', exc_info=True) status = cfnresponse.FAILED finally: timer.cancel() cfnresponse.send(event, context, status, responseData, physical_resource_id) LambdaExecutionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: lambda.amazonaws.com Action: - sts:AssumeRole Policies: - PolicyName: CloudwatchLogs PolicyDocument: Statement: - Effect: Allow Action: - logs:CreateLogGroup - logs:CreateLogStream - logs:PutLogEvents - logs:GetLogEvents Resource: - !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:* - PolicyName: ManipulateSnapshots PolicyDocument: Statement: - Effect: Allow Action: - rds:CopyDBSnapshot - rds:CopyDBClusterSnapshot - rds:DeleteDBSnapshot - rds:DeleteDBClusterSnapshot - rds:ModifyDBSnapshotAttribute - rds:DescribeDBSnapshots - rds:DescribeDBClusterSnapshots - rds:ListTagsForResource - ec2:CopySnapshot - ec2:CreateTags - ec2:DescribeTags - ec2:DescribeSnapshots - ec2:DeleteSnapshot Resource: '*' - PolicyName: AllowPublish2ProducerTopic PolicyDocument: Statement: - Effect: Allow Action: - sns:Publish Resource: !Sub "arn:aws:sns:${AWS::Region}:${SourceAcct}:DracoProducer" - PolicyName: KeyManagement PolicyDocument: Statement: - Effect: Allow Action: - kms:CreateAlias - kms:CreateKey - kms:DescribeKey - kms:EnableKeyRotation - kms:ListAliases - kms:TagResource Resource: '*' - PolicyName: KeyUsage PolicyDocument: Statement: - Effect: Allow Action: - kms:CreateGrant - kms:Decrypt - kms:DescribeKey - kms:Encrypt - kms:GenerateDataKey* - kms:ListGrants - kms:ReEncrypt* - kms:RevokeGrants Resource: '*' - PolicyName: SaveKeys PolicyDocument: Statement: - Effect: Allow Action: - s3:GetObject* - s3:PutObject Resource: - !Sub 'arn:aws:s3:::${DracoRegionalBucket}/keys*' - PolicyName: AllowStepfunctions PolicyDocument: Statement: - Effect: Allow Action: - states:StartExecution Resource: !Sub "arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:wait4copy-${AWS::AccountId}" SnapshotTopicPolicy: Type: AWS::SNS::TopicPolicy Properties: Topics: - !Ref SnapshotTopic PolicyDocument: Version: '2012-10-17' Id: SetupSnapshotTopic Statement: - Sid: Defaults Effect: Allow Principal: AWS: "*" Action: - sns:GetTopicAttributes - sns:SetTopicAttributes - sns:AddPermission - sns:RemovePermission - sns:DeleteTopic - sns:Subscribe - sns:ListSubscriptionsByTopic - sns:Publish - sns:Receive Resource: !Ref SnapshotTopic Condition: StringEquals: AWS:SourceOwner: !Ref SourceAcct - Sid: AllowSourcePublish Effect: Allow Principal: AWS: !Ref SourceAcct Action: - sns:Publish Resource: !Ref SnapshotTopic SnapshotTopic: Type: AWS::SNS::Topic Properties: TopicName: DracoConsumer DisplayName: Topic for Snapshot Processing Subscription: - Endpoint: !GetAtt SnapshotHandlerLambda.Arn Protocol: lambda LambdaInvokePermission: Type: AWS::Lambda::Permission Properties: Action: lambda:InvokeFunction Principal: sns.amazonaws.com SourceArn: !Ref SnapshotTopic FunctionName: !GetAtt SnapshotHandlerLambda.Arn SnapshotHandlerLambda: Type: AWS::Lambda::Function Properties: Description: Code that copies the shared snapshot for DR Environment: Variables: PRODUCER_TOPIC_ARN: !Sub "arn:aws:sns:${AWS::Region}:${SourceAcct}:DracoProducer" STATE_MACHINE_ARN: !Sub "arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:wait4copy-${AWS::AccountId}" DRY_RUN: true TAG_KEY: !Ref DrTagKey TAG_VALUE: !Ref DrTagValue Handler: consumer.handler Runtime: nodejs12.x Timeout: 60 Role: !GetAtt LambdaExecutionRole.Arn Code: S3Bucket: !Ref DracoRegionalBucket S3Key: !Sub "${CodePrefix}consumer.zip" S3ObjectVersion: !GetAtt CopyObjects.consumer.zip DracoLogGroup: Type: AWS::Logs::LogGroup Properties: LogGroupName: !Sub "/aws/lambda/${SnapshotHandlerLambda}" RetentionInDays: 90 CloudWatchErrorMetric: Type: AWS::Logs::MetricFilter Properties: FilterPattern: "ERROR" LogGroupName: !Ref DracoLogGroup MetricTransformations: - MetricValue: "1" DefaultValue: 0 MetricNamespace: "Draco/SnapshotHandler" MetricName: "ErrorCount" CloudWatchWarnMetric: Type: AWS::Logs::MetricFilter Properties: FilterPattern: "WARN" LogGroupName: !Ref DracoLogGroup MetricTransformations: - MetricValue: "1" DefaultValue: 0 MetricNamespace: "Draco/SnapshotHandler" MetricName: "WarnCount" Wait4Copy: Type: AWS::CloudFormation::Stack Properties: TemplateURL: !Sub "https://s3.amazonaws.com/${CodeBucket}/draco/wait4copy.yaml" Parameters: CodeBucket: !Ref DracoRegionalBucket CodeKey: !Sub "${CodePrefix}wait4copy.zip" CodeVersion: !GetAtt CopyObjects.wait4copy.zip NotifyTopicArn: !Ref SnapshotTopic Outputs: DracoRegionalBucket: Value: !Ref DracoRegionalBucket DrTopicArn: Value: !Ref SnapshotTopic DrLambdaRoleArn: Value: !GetAtt LambdaExecutionRole.Arn # vim: sts=2 et sw=2: