AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: > GraphLoader Sample SAM Template for GraphLoader Parameters: NeptuneLoadFromS3IAMRoleArn: Type: String NeptuneLoaderEndpoint: Type: String NeptuneSubnet: Type: String NeptuneSecurityGroup: Type: String LoggingBucketName: Type: String VPC: Type: String PrivateSubnet1: Type: String PrivateSubnet2: Type: String PowertoolsLayer: Type: String MasterKey: Type: String VPCCidrBlock: Type: String # More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst Globals: Function: Timeout: 3 Tracing: Active Runtime: python3.9 Architectures: - x86_64 RuntimeManagementConfig: UpdateRuntimeOn: Auto Api: TracingEnabled: True Resources: GraphLoadStateMachine: Type: AWS::Serverless::StateMachine # More info about State Machine Resource: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-statemachine.html Properties: DefinitionUri: statemachine/graph_load.asl.json DefinitionSubstitutions: GraphLoadFunctionArn: !GetAtt GraphLoadFunction.Arn CheckGraphLoadStatusFunctionArn: !GetAtt CheckGraphLoadStatusFunction.Arn Policies: # Find out more about SAM policy templates: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-policy-templates.html - LambdaInvokePolicy: FunctionName: !Ref GraphLoadFunction - LambdaInvokePolicy: FunctionName: !Ref CheckGraphLoadStatusFunction GraphLoadTriggerSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Security group for the graph load function # GroupName: GraphLoadSecurityGroup VpcId: !Ref VPC SecurityGroupEgress: - IpProtocol: "tcp" Description: Allow outbound traffic to VPC FromPort: "443" ToPort: "443" CidrIp: !Ref VPCCidrBlock GraphLoadStateMachineTrigger: Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction Properties: CodeUri: functions/graph_load_state_machine_trigger Handler: app.lambda_handler Runtime: python3.9 Timeout: 15 ReservedConcurrentExecutions: 1 KmsKeyArn: !Ref MasterKey DeadLetterQueue: TargetArn: !GetAtt GraphLoadTriggerDeadLetterLoadQueue.Arn Type: SQS Events: GraphLoadEvent: Type: SQS Properties: Queue: !GetAtt GraphLoadQueue.Arn BatchSize: 10 Environment: Variables: GRAPH_LOAD_STATE_MACHINE_ARN: !Ref GraphLoadStateMachine Policies: - Statement: - Sid: KmsInlinePolicy Effect: Allow Action: - "kms:GenerateDataKey" - "kms:Decrypt" - "kms:DescribeKey" Resource: !Ref MasterKey - Sid: SqsInlinePolicy Effect: Allow Action: - "sqs:DeleteMessage" - "sqs:ReceiveMessage" - "sqs:SendMessage" - "sqs:GetQueueAttributes" - "sqs:ChangeMessageVisibility" - "sqs:SetQueueAttributes" Resource: !GetAtt GraphLoadQueue.Arn - Sid: StateMachineInlinePolicy Effect: Allow Action: - "states:CreateActivity" - "states:CreateStateMachine" - "states:DeleteActivity" - "states:DeleteStateMachine" - "states:DescribeActivity" - "states:DescribeExecution" - "states:DescribeStateMachine" - "states:GetActivityTask" - "states:GetExecutionHistory" - "states:ListActivities" - "states:ListExecutions" - "states:ListStateMachines" - "states:SendTaskFailure" - "states:SendTaskHeartbeat" - "states:SendTaskSuccess" - "states:StartExecution" - "states:StopExecution" Resource: !Ref GraphLoadStateMachine VpcConfig: SecurityGroupIds: - !Ref GraphLoadTriggerSecurityGroup SubnetIds: - !Ref PrivateSubnet1 - !Ref PrivateSubnet2 Layers: - !Ref PowertoolsLayer Metadata: cfn_nag: rules_to_suppress: - id: W89 reason: False positive. This is being deployed in a VPC by specifying the VpcConfig parameters. GraphLoadStagingBucket: Type: AWS::S3::Bucket Properties: BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: AES256 PublicAccessBlockConfiguration: BlockPublicAcls: True BlockPublicPolicy: True IgnorePublicAcls: True RestrictPublicBuckets: True VersioningConfiguration: Status: Enabled LoggingConfiguration: DestinationBucketName: !Ref LoggingBucketName LogFilePrefix: GraphETL/Video/GraphLoad GraphLoadStagingBucketPolicy: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref GraphLoadStagingBucket PolicyDocument: Version: 2012-10-17 Statement: - Action: - 's3:GetObject' Effect: Deny Resource: !Join - '' - - 'arn:aws:s3:::' - !Ref GraphLoadStagingBucket - /* Principal: '*' Condition: StringNotEquals: 'aws:PrincipalAccount': !Ref AWS::AccountId GraphLoadQueue: Type: AWS::SQS::Queue Properties: KmsMasterKeyId: !Ref MasterKey GraphLoadTriggerDeadLetterLoadQueue: Type: AWS::SQS::Queue Properties: KmsMasterKeyId: !Ref MasterKey GraphLoadFunction: Type: AWS::Serverless::Function Properties: CodeUri: functions/load_to_graph/ Handler: app.lambda_handler Timeout: 60 ReservedConcurrentExecutions: 1 KmsKeyArn: !Ref MasterKey Environment: Variables: NEPTUNE_LOADER_ENDPOINT: !Ref NeptuneLoaderEndpoint NEPTUNE_LOAD_ROLE_ARN: !Ref NeptuneLoadFromS3IAMRoleArn GRAPH_LOAD_STAGING_BUCKET: !Ref GraphLoadStagingBucket VpcConfig: SecurityGroupIds: - !Ref NeptuneSecurityGroup SubnetIds: - !Ref NeptuneSubnet Policies: - AmazonS3ReadOnlyAccess Layers: - !Ref PowertoolsLayer Metadata: cfn_nag: rules_to_suppress: - id: W89 reason: False positive. This is being deployed in a VPC by specifying the VpcConfig parameters. CheckGraphLoadStatusFunction: Type: AWS::Serverless::Function Properties: CodeUri: functions/check_load_status/ Handler: app.lambda_handler Timeout: 60 ReservedConcurrentExecutions: 1 KmsKeyArn: !Ref MasterKey Environment: Variables: NEPTUNE_LOADER_ENDPOINT: !Ref NeptuneLoaderEndpoint NEPTUNE_LOAD_ROLE_ARN: !Ref NeptuneLoadFromS3IAMRoleArn VpcConfig: SecurityGroupIds: - !Ref NeptuneSecurityGroup SubnetIds: - !Ref NeptuneSubnet Layers: - !Ref PowertoolsLayer Metadata: cfn_nag: rules_to_suppress: - id: W89 reason: False positive. This is being deployed in a VPC by specifying the VpcConfig parameters. Outputs: GraphLoadStagingBucket: Description: "Bucket for staging graph load files" Value: !Ref GraphLoadStagingBucket GraphLoadQueueName: Description: "Queue for triggering graph loading (name)" Value: !GetAtt GraphLoadQueue.QueueName GraphLoadQueueArn: Description: "Queue for triggering graph loading (arn)" Value: !GetAtt GraphLoadQueue.Arn