Description: SIEM on Amazon OpenSearch Service v2.10.0 Metadata: AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Initial Deployment Parameters Parameters: - AllowedSourceIpAddresses - Label: default: Basic Configuration Parameters: - DeploymentTarget - DomainOrCollectionName - SnsEmail - ReservedConcurrency - Label: default: Log Enrichment - optional Parameters: - GeoLite2LicenseKey - OtxApiKey - EnableTor - EnableAbuseCh - IocDownloadInterval - Label: default: Advanced Configuration - optional Parameters: - VpcEndpointId - CreateS3VpcEndpoint - CreateSqsVpcEndpoint - Label: default: Control Tower Integration - optional Parameters: - ControlTowerLogBucketNameList - ControlTowerSqsForLogBuckets - ControlTowerRoleArnForEsLoader - Label: default: Security Lake Integration - optional Parameters: - SecurityLakeSubscriberSqs - SecurityLakeRoleArn - SecurityLakeExternalId - Label: default: Other parameters Parameters: [] Mappings: RegionMap: af-south-1: ElbV2AccountId: '098369216593' LambdaArch: arm64 ap-east-1: ElbV2AccountId: '754344448648' LambdaArch: arm64 ap-northeast-1: ElbV2AccountId: '582318560864' LambdaArch: arm64 ap-northeast-2: ElbV2AccountId: '600734575887' LambdaArch: arm64 ap-northeast-3: ElbV2AccountId: '383597477331' LambdaArch: arm64 ap-south-1: ElbV2AccountId: '718504428378' LambdaArch: arm64 ap-southeast-1: ElbV2AccountId: '114774131450' LambdaArch: arm64 ap-southeast-2: ElbV2AccountId: '783225319266' LambdaArch: arm64 ap-southeast-3: ElbV2AccountId: '589379963580' LambdaArch: arm64 ca-central-1: ElbV2AccountId: '985666609251' LambdaArch: arm64 cn-north-1: ElbV2AccountId: '638102146993' LambdaArch: x86_64 cn-northwest-1: ElbV2AccountId: '037604701340' LambdaArch: x86_64 eu-central-1: ElbV2AccountId: '054676820928' LambdaArch: arm64 eu-north-1: ElbV2AccountId: '897822967062' LambdaArch: arm64 eu-south-1: ElbV2AccountId: '635631232127' LambdaArch: arm64 eu-west-1: ElbV2AccountId: '156460612806' LambdaArch: arm64 eu-west-2: ElbV2AccountId: '652711504416' LambdaArch: arm64 eu-west-3: ElbV2AccountId: '009996457667' LambdaArch: arm64 me-south-1: ElbV2AccountId: '076674570225' LambdaArch: arm64 sa-east-1: ElbV2AccountId: '507241528517' LambdaArch: arm64 us-east-1: ElbV2AccountId: '127311923021' LambdaArch: arm64 us-east-2: ElbV2AccountId: '033677994240' LambdaArch: arm64 us-gov-east-1: ElbV2AccountId: '190560391635' LambdaArch: x86_64 us-gov-west-1: ElbV2AccountId: '048591011584' LambdaArch: x86_64 us-iso-east-1: ElbV2AccountId: '770363063475' LambdaArch: x86_64 us-iso-west-1: ElbV2AccountId: '121062877647' LambdaArch: x86_64 us-isob-east-1: ElbV2AccountId: '740734521339' LambdaArch: x86_64 us-west-1: ElbV2AccountId: '027434742980' LambdaArch: arm64 us-west-2: ElbV2AccountId: '797873946194' LambdaArch: arm64 ap-south-2: ElbV2AccountId: '999999999999' LambdaArch: x86_64 ap-southeast-4: ElbV2AccountId: '999999999999' LambdaArch: x86_64 eu-central-2: ElbV2AccountId: '999999999999' LambdaArch: x86_64 eu-south-2: ElbV2AccountId: '999999999999' LambdaArch: x86_64 me-central-1: ElbV2AccountId: '999999999999' LambdaArch: x86_64 Parameters: DeploymentTarget: Type: String Default: opensearch_managed_cluster AllowedValues: - opensearch_managed_cluster - opensearch_serverless Description: Where would you like to deploy the SIEM solution? Amazon OpenSearch managed cluster or serverless? Serverless is experimental option DomainOrCollectionName: Type: String Default: aes-siem AllowedPattern: ^[0-9a-zA-Z_-]* Description: Amazon OpenSearch Service Domain name or OpenSearch Serverless Collection name VpcEndpointId: Type: String Default: '' AllowedPattern: (^vpce-[0-9a-z]*|aos-[0-9a-z]*|) Description: (Optional) Specify VPC Endpoint for OpenSearch managed cluster or OpenSearch Serverless. This should be manually created before deployment. If you specify VPC Endpoint, a few lambda functions and other resources will be deploy into VPC AllowedSourceIpAddresses: Type: String Default: 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 AllowedPattern: ^[0-9./\s]* Description: Space-delimited list of CIDR blocks. This parameter applies only during the initial deployment SnsEmail: Type: String Default: '' AllowedPattern: ^([0-9a-zA-Z@_\-\+\.]*|) Description: (Optional) Input your email as SNS topic, where Amazon OpenSearch Service will send alerts to GeoLite2LicenseKey: Type: String Default: '' AllowedPattern: ^([0-9a-zA-Z]{6}_[0-9a-zA-Z]{29}_mmk|[0-9a-zA-Z]{16}|)$ Description: (Optional) If you wolud like to enrich geoip locaiton such as IP address's country, get a license key from MaxMind and input the key. The license is a string of 16 or 40 digits MaxLength: 40 ReservedConcurrency: Type: Number Default: 10 Description: Input lambda reserved concurrency for es-loader. Increase this value if there are steady logs delay despite withou errors OtxApiKey: Type: String Default: '' AllowedPattern: ^([0-9a-f,x]{64}|)$ Description: '(Optional) If you wolud like to download IoC from AlienVault OTX, please enter OTX API Key. See details: https://otx.alienvault.com' MaxLength: 64 EnableTor: Type: String Default: 'false' AllowedValues: - 'true' - 'false' Description: 'Would you like to download TOR IoC? See details: https://check.torproject.org/api/bulk' EnableAbuseCh: Type: String Default: 'false' AllowedValues: - 'true' - 'false' Description: 'Would you like to download IoC from abuse.ch? See details: https://feodotracker.abuse.ch/blocklist/' IocDownloadInterval: Type: Number Default: 720 Description: Specify interval in minute to download IoC, default is 720 miniutes ( = 12 hours ).min is 30 minutes. max is 10080 minutes ( = 7 days ). MaxValue: 10080 MinValue: 30 CreateS3VpcEndpoint: Type: String Default: 'true' AllowedValues: - 'true' - 'false' Description: Create new SQS VPC Endpoint with SIEM solution. If you use existing VPC and already have S3 VPC Endpoint, select false CreateSqsVpcEndpoint: Type: String Default: 'true' AllowedValues: - 'true' - 'false' Description: Create new S3 VPC Endpoint with SIEM solution. If you use existing VPC and already have S3 VPC Endpoint, select false ControlTowerLogBucketNameList: Type: String Default: '' AllowedPattern: ^[-0-9a-z.\s,]*$ Description: Specify S3 log bucket names in the Log Archive account. Comma separated list. (e.g., aws-controltower-logs-123456789012-ap-northeast-1, aws-controltower-s3-access-logs-123456789012-ap-northeast-1 ) ControlTowerRoleArnForEsLoader: Type: String Default: '' AllowedPattern: ^(arn:aws.*:iam::[0-9]{12}:role/.*|)$ Description: Specify IAM Role ARN to be assumed by aes-siem-es-loader. (e.g., arn:aws-us-gov:iam::123456789012:role/ct-role-for-siem ) ControlTowerSqsForLogBuckets: Type: String Default: '' AllowedPattern: ^(arn:aws[0-9a-zA-Z:/_-]*|)$ Description: Specify SQS ARN for S3 log buckets in Log Archive Account. (e.g., arn:aws-us-gov:sqs:ap-northeast-1:12345678902:aes-siem-ct ) SecurityLakeRoleArn: Type: String Default: '' AllowedPattern: ^(arn:aws.*:iam::[0-9]{12}:role/AmazonSecurityLake-[0-9a-f-]*|)$ Description: Specify IAM Role ARN to be assumed by aes-siem-es-loader. (e.g., arn:aws-us-gov:iam::123456789012:role/AmazonSecurityLake-00001111-2222-3333-5555-666677778888 ) SecurityLakeExternalId: Type: String Default: '' AllowedPattern: ^([0-9a-zA-Z]*|)$ Description: Specify Security Lake external ID for cross account. (e.g., externalid123 ) SecurityLakeSubscriberSqs: Type: String Default: '' AllowedPattern: ^(arn:aws.*:sqs:.*:[0-9]{12}:AmazonSecurityLake-[0-9a-f-]*-Main-Queue|)$ Description: Specify SQS ARN of Security Lake Subscriber. (e.g., arn:aws-us-gov:sqs:us-east-1:12345678902:AmazonSecurityLake-00001111-2222-3333-5555-666677778888-Main-Queue ) Conditions: isGlobalRegion: !Not - !Or - !Equals - !Ref 'AWS::Region' - cn-north-1 - !Equals - !Ref 'AWS::Region' - cn-northwest-1 - !Equals - !Ref 'AWS::Region' - us-gov-east-1 - !Equals - !Ref 'AWS::Region' - us-gov-west-1 HasLambdaArchitecturesProp: !Not - !Or - !Equals - !Ref 'AWS::Region' - ap-south-2 - !Equals - !Ref 'AWS::Region' - ap-southeast-4 - !Equals - !Ref 'AWS::Region' - eu-central-2 - !Equals - !Ref 'AWS::Region' - eu-south-2 - !Equals - !Ref 'AWS::Region' - me-central-1 IsServerless: !Equals - !Ref 'DeploymentTarget' - opensearch_serverless IsManagedCluster: !Equals - !Ref 'DeploymentTarget' - opensearch_managed_cluster hasVpce: !Not - !Equals - !Ref 'VpcEndpointId' - '' IsInVpc: !Or - !Equals - false - true - !Condition 'hasVpce' HasGeoipLicense: !Not - !Equals - !Ref 'GeoLite2LicenseKey' - '' EnableIOC: !Or - !Not - !Equals - !Ref 'OtxApiKey' - '' - !Equals - !Ref 'EnableTor' - 'true' - !Equals - !Ref 'EnableAbuseCh' - 'true' HasSnsEmail: !Not - !Equals - !Ref 'SnsEmail' - '' SqsVpceIsRequired: !And - !Equals - !Ref 'CreateS3VpcEndpoint' - 'true' - !Condition 'IsInVpc' S3VpceIsRequired: !And - !Equals - !Ref 'CreateSqsVpcEndpoint' - 'true' - !Condition 'IsInVpc' IsControlTowerAcccess: !And - !Not - !Equals - !Ref 'ControlTowerLogBucketNameList' - '' - !Not - !Equals - !Ref 'ControlTowerRoleArnForEsLoader' - '' - !Not - !Equals - !Ref 'ControlTowerSqsForLogBuckets' - '' IsSecurityLakeAcccess: !And - !Not - !Equals - !Ref 'SecurityLakeExternalId' - '' - !Not - !Equals - !Ref 'SecurityLakeRoleArn' - '' - !Not - !Equals - !Ref 'SecurityLakeSubscriberSqs' - '' Resources: LambdaResourceValidatorServiceRole8BE4D6B0: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' LambdaResourceValidatorE0EF57CE: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/deploy_es.zip' Role: !GetAtt 'LambdaResourceValidatorServiceRole8BE4D6B0.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / resource validator for deployment Environment: Variables: ACCOUNT_ID: !Ref 'AWS::AccountId' AOS_SUBNET_IDS: '' DEPLOYMENT_TARGET: !Ref 'DeploymentTarget' DOMAIN_OR_COLLECTION_NAME: !Ref 'DomainOrCollectionName' S3_LOG: !Sub 'aes-siem-${AWS::AccountId}-log' S3_SNAPSHOT: !Sub 'aes-siem-${AWS::AccountId}-snapshot' SOLUTION_PREFIX: aes-siem VPCE_ID: !Ref 'VpcEndpointId' FunctionName: aes-siem-resource-validator Handler: index.resource_validator_handler MemorySize: 128 ReservedConcurrentExecutions: 1 Runtime: python3.9 Timeout: 30 DependsOn: - LambdaResourceValidatorServiceRole8BE4D6B0 LambdaResourceValidatorCurrentVersion1C9DB5C1386d40e64504275b8d67bc70d088c8cc: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaResourceValidatorE0EF57CE' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain aessiempolicyforvpcvalidation724B105E: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - ec2:CreateNetworkInterface - ec2:DescribeNetworkInterfaces - ec2:DeleteNetworkInterface - ec2:AssignPrivateIpAddresses - ec2:UnassignPrivateIpAddresses Effect: Allow Resource: '*' - Action: - aoss:BatchGetCollection - aoss:BatchGetVpcEndpoint - ec2:DescribeRouteTables - ec2:DescribeSubnets - ec2:DescribeVpcEndpoints - ec2:DescribeVpcs - es:DescribeVpcEndpoints - iam:GetRole Effect: Allow Resource: '*' - Action: s3:GetBucketPolicy Effect: Allow Resource: !Sub 'arn:aws-us-gov:s3:::aes-siem-${AWS::AccountId}-log' Sid: ToGetBucektPolicy - Action: s3:PutObject Effect: Allow Resource: !Sub 'arn:aws-us-gov:s3:::aes-siem-${AWS::AccountId}-snapshot/*' Sid: ToUploadPolicy Version: '2012-10-17' PolicyName: aes-siem-policy-for-vpc-validation Roles: - !Ref 'LambdaResourceValidatorServiceRole8BE4D6B0' ExecCustomResourceValidator: Type: AWS::CloudFormation::CustomResource Properties: ServiceToken: !GetAtt 'LambdaResourceValidatorE0EF57CE.Arn' ConfigVersion: 2.10.0 vpce: !Ref 'VpcEndpointId' DeploymentTarget: !Ref 'DeploymentTarget' DeletionPolicy: Retain KmsAesSiemLog44B26597: Type: AWS::KMS::Key Properties: KeyPolicy: Statement: - Action: kms:* Effect: Allow Principal: AWS: !Sub 'arn:${AWS::Partition}:iam::${AWS::AccountId}:root' Resource: '*' - Action: kms:GenerateDataKey Effect: Allow Principal: Service: guardduty.amazonaws.com Resource: '*' Sid: Allow GuardDuty to use the key - Action: - kms:Encrypt - kms:Decrypt - kms:ReEncrypt* - kms:GenerateDataKey* - kms:DescribeKey Effect: Allow Principal: Service: delivery.logs.amazonaws.com Resource: '*' Sid: Allow VPC Flow Logs to use the key - Action: - kms:DescribeKey - kms:ReEncryptFrom Effect: Allow Principal: AWS: !Sub 'arn:${AWS::Partition}:iam::${AWS::AccountId}:root' Resource: '*' Sid: Allow principals in the account to decrypt log files - Action: - kms:Decrypt - kms:DescribeKey - kms:Encrypt - kms:GenerateDataKey* - kms:ReEncrypt* Condition: ForAnyValue:StringEquals: aws:CalledVia: athena.amazonaws.com Effect: Allow Principal: AWS: !Sub 'arn:${AWS::Partition}:iam::${AWS::AccountId}:root' Resource: '*' Sid: Allow Athena to query s3 objects with this key - Action: kms:DescribeKey Effect: Allow Principal: Service: cloudtrail.amazonaws.com Resource: '*' Sid: Allow CloudTrail to describe key - Action: kms:GenerateDataKey* Condition: StringLike: kms:EncryptionContext:aws:cloudtrail:arn: - !Sub 'arn:aws-us-gov:cloudtrail:*:${AWS::AccountId}:trail/*' Effect: Allow Principal: Service: cloudtrail.amazonaws.com Resource: '*' Sid: Allow CloudTrail to encrypt logs - Action: - kms:Decrypt - kms:GenerateDataKey Effect: Allow Principal: Service: events.amazonaws.com Resource: '*' Version: '2012-10-17' Description: CMK for SIEM solution UpdateReplacePolicy: Retain DeletionPolicy: Retain KmsAesSiemLogAliasE0A4C571: Type: AWS::KMS::Alias Properties: AliasName: alias/aes-siem-key TargetKeyId: !GetAtt 'KmsAesSiemLog44B26597.Arn' UpdateReplacePolicy: Retain DeletionPolicy: Retain S3BucketForGeoip04B5F171: Type: AWS::S3::Bucket Properties: BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: AES256 BucketName: !Sub 'aes-siem-${AWS::AccountId}-geo' LifecycleConfiguration: Rules: - ExpirationInDays: 8 Id: delete-ioc-temp-files Prefix: IOC/tmp/ Status: Enabled PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true UpdateReplacePolicy: Retain DeletionPolicy: Retain S3BucketForGeoipPolicy854C0CB1: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref 'S3BucketForGeoip04B5F171' PolicyDocument: Statement: - Action: s3:* Condition: Bool: aws:SecureTransport: 'false' Effect: Deny Principal: AWS: '*' Resource: - !GetAtt 'S3BucketForGeoip04B5F171.Arn' - !Sub '${S3BucketForGeoip04B5F171.Arn}/*' Version: '2012-10-17' S3BucketForLog20898FE4: Type: AWS::S3::Bucket Properties: BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: AES256 BucketName: !Sub 'aes-siem-${AWS::AccountId}-log' PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true VersioningConfiguration: Status: Enabled DependsOn: - ExecCustomResourceValidator UpdateReplacePolicy: Retain DeletionPolicy: Retain S3BucketForLogPolicy546D5712: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref 'S3BucketForLog20898FE4' PolicyDocument: Statement: - Action: s3:* Condition: Bool: aws:SecureTransport: 'false' Effect: Deny Principal: AWS: '*' Resource: - !GetAtt 'S3BucketForLog20898FE4.Arn' - !Sub '${S3BucketForLog20898FE4.Arn}/*' - Action: s3:PutObject Effect: Allow Principal: AWS: !Sub - arn:${AWS::Partition}:iam::${Param1}:root - Param1: !FindInMap - RegionMap - !Ref 'AWS::Region' - ElbV2AccountId Resource: - !Sub '${S3BucketForLog20898FE4.Arn}/AWSLogs/${AWS::AccountId}/*' - !Sub '${S3BucketForLog20898FE4.Arn}/*/AWSLogs/${AWS::AccountId}/*' Sid: ALB,CLB Policy - Action: - s3:GetBucketAcl - s3:ListBucket - s3:PutObject Condition: StringEquals: aws:SourceAccount: - !Ref 'AWS::AccountId' Effect: Allow Principal: Service: delivery.logs.amazonaws.com Resource: - !GetAtt 'S3BucketForLog20898FE4.Arn' - !Sub '${S3BucketForLog20898FE4.Arn}/*' Sid: AWSLogDelivery For NLB,R53Resolver,Flowlogs - Action: - s3:GetBucketAcl - s3:ListBucket Effect: Allow Principal: Service: - cloudtrail.amazonaws.com - config.amazonaws.com Resource: !GetAtt 'S3BucketForLog20898FE4.Arn' Sid: AWSLogDeliveryAclCheck For Cloudtrail, Config - Action: s3:PutObject Condition: StringEquals: aws:SourceAccount: - !Ref 'AWS::AccountId' Effect: Allow Principal: Service: - cloudtrail.amazonaws.com - config.amazonaws.com Resource: !Sub '${S3BucketForLog20898FE4.Arn}/*/*' Sid: AWSLogDeliveryWrite For CloudTrail, Config - Action: - s3:GetBucketLocation - s3:PutObject Condition: StringEquals: aws:SourceAccount: - !Ref 'AWS::AccountId' Effect: Allow Principal: Service: guardduty.amazonaws.com Resource: - !GetAtt 'S3BucketForLog20898FE4.Arn' - !Sub '${S3BucketForLog20898FE4.Arn}/*' Sid: Allow GuardDuty to put objects Version: '2012-10-17' DependsOn: - ExecCustomResourceValidator S3BucketForLogNotificationsAEE88E1E: Type: Custom::S3BucketNotifications Properties: ServiceToken: !GetAtt 'BucketNotificationsHandler050a0587b7544547bf325f094a3db8347ECC3691.Arn' BucketName: !Ref 'S3BucketForLog20898FE4' NotificationConfiguration: LambdaFunctionConfigurations: - Events: - s3:ObjectCreated:* LambdaFunctionArn: !GetAtt 'LambdaEsLoader4B1E2DD9.Arn' Managed: true DependsOn: - ExecCustomResourceValidator - S3BucketForLogAllowBucketNotificationsToaessiemLambdaEsLoaderEBF3B9FB7766EAA3 S3BucketForLogAllowBucketNotificationsToaessiemLambdaEsLoaderEBF3B9FB7766EAA3: Type: AWS::Lambda::Permission Properties: Action: lambda:InvokeFunction FunctionName: !GetAtt 'LambdaEsLoader4B1E2DD9.Arn' Principal: s3.amazonaws.com SourceAccount: !Ref 'AWS::AccountId' SourceArn: !GetAtt 'S3BucketForLog20898FE4.Arn' DependsOn: - ExecCustomResourceValidator S3BucketForSnapshot40E67D36: Type: AWS::S3::Bucket Properties: BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: AES256 BucketName: !Sub 'aes-siem-${AWS::AccountId}-snapshot' PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true UpdateReplacePolicy: Retain DeletionPolicy: Retain S3BucketForSnapshotPolicy3DEBD2C0: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref 'S3BucketForSnapshot40E67D36' PolicyDocument: Statement: - Action: s3:* Condition: Bool: aws:SecureTransport: 'false' Effect: Deny Principal: AWS: '*' Resource: - !GetAtt 'S3BucketForSnapshot40E67D36.Arn' - !Sub '${S3BucketForSnapshot40E67D36.Arn}/*' - Action: - s3:PutObject - s3:GetObject - s3:DeleteObject Effect: Allow Principal: AWS: !GetAtt 'AesSiemSnapshotRoleF313ED39.Arn' Resource: !Sub '${S3BucketForSnapshot40E67D36.Arn}/*' Sid: Allow OpenSearch Service to store snapshot Version: '2012-10-17' AesSiemSnapshotRoleF313ED39: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: opensearchservice.amazonaws.com Version: '2012-10-17' Policies: - PolicyDocument: Statement: - Action: s3:ListBucket Effect: Allow Resource: !GetAtt 'S3BucketForSnapshot40E67D36.Arn' - Action: - s3:GetObject - s3:PutObject - s3:DeleteObject Effect: Allow Resource: !Sub '${S3BucketForSnapshot40E67D36.Arn}/*' Version: '2012-10-17' PolicyName: s3access RoleName: aes-siem-snapshot-role AesSiemSnsRole64262F46: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: opensearchservice.amazonaws.com Version: '2012-10-17' RoleName: aes-siem-sns-role AesSiemSnsRoleDefaultPolicy7B8095B5: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - kms:Decrypt - kms:GenerateDataKey Effect: Allow Resource: !GetAtt 'KmsAesSiemLog44B26597.Arn' - Action: sns:Publish Effect: Allow Resource: !Ref 'SnsTopic2C1570A4' Version: '2012-10-17' PolicyName: AesSiemSnsRoleDefaultPolicy7B8095B5 Roles: - !Ref 'AesSiemSnsRole64262F46' AesSiemEsLoaderEC2RoleFE3F9F00: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: ec2.amazonaws.com Version: '2012-10-17' RoleName: aes-siem-es-loader-for-ec2 AesSiemEsLoaderEC2RoleDefaultPolicyAF44001A: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - sqs:GetQueue* - sqs:ListQueues* - sqs:ReceiveMessage* - sqs:DeleteMessage* Effect: Allow Resource: !GetAtt 'AesSiemDlq1CD8439D.Arn' - Action: kms:Decrypt Effect: Allow Resource: !GetAtt 'KmsAesSiemLog44B26597.Arn' - Action: - s3:GetObject* - s3:GetBucket* - s3:List* Effect: Allow Resource: - !GetAtt 'S3BucketForGeoip04B5F171.Arn' - !Sub '${S3BucketForGeoip04B5F171.Arn}/*' - Action: - s3:GetObject* - s3:GetBucket* - s3:List* Effect: Allow Resource: - !GetAtt 'S3BucketForLog20898FE4.Arn' - !Sub '${S3BucketForLog20898FE4.Arn}/*' Version: '2012-10-17' PolicyName: AesSiemEsLoaderEC2RoleDefaultPolicyAF44001A Roles: - !Ref 'AesSiemEsLoaderEC2RoleFE3F9F00' AesSiemEsLoaderEC2InstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Roles: - !Ref 'AesSiemEsLoaderEC2RoleFE3F9F00' InstanceProfileName: !Ref 'AesSiemEsLoaderEC2RoleFE3F9F00' AesSiemVpcNoinboundSecurityGroup58555CE0: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: aes-siem/AesSiemVpcNoinboundSecurityGroup VpcId: !GetAtt 'ExecCustomResourceValidator.vpc_id' UpdateReplacePolicy: Retain DeletionPolicy: Retain Condition: IsInVpc AesSiemVpcSecurityGroup2nd: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: aes-siem/AesSiemVpcSecurityGroup2nd GroupName: aes-siem-vpc-sg2 SecurityGroupIngress: - CidrIp: !GetAtt 'ExecCustomResourceValidator.cidr_block0' FromPort: 443 IpProtocol: tcp ToPort: 443 - CidrIp: !GetAtt 'ExecCustomResourceValidator.cidr_block1' FromPort: 443 IpProtocol: tcp ToPort: 443 - CidrIp: !GetAtt 'ExecCustomResourceValidator.cidr_block2' FromPort: 443 IpProtocol: tcp ToPort: 443 - CidrIp: !GetAtt 'ExecCustomResourceValidator.cidr_block3' FromPort: 443 IpProtocol: tcp ToPort: 443 VpcId: !GetAtt 'ExecCustomResourceValidator.vpc_id' Condition: IsInVpc VpcAesSiemSQSEndpoint8BFF7847: Type: AWS::EC2::VPCEndpoint Properties: ServiceName: !Sub 'com.amazonaws.${AWS::Region}.sqs' VpcId: !GetAtt 'ExecCustomResourceValidator.vpc_id' PrivateDnsEnabled: true SecurityGroupIds: - !GetAtt 'AesSiemVpcSecurityGroup2nd.GroupId' SubnetIds: !GetAtt 'ExecCustomResourceValidator.subnets' VpcEndpointType: Interface Condition: SqsVpceIsRequired VpcAesSiemS3Endpoint003F70DF: Type: AWS::EC2::VPCEndpoint Properties: ServiceName: !Sub 'com.amazonaws.${AWS::Region}.s3' VpcId: !GetAtt 'ExecCustomResourceValidator.vpc_id' RouteTableIds: !GetAtt 'ExecCustomResourceValidator.route_table_ids' VpcEndpointType: Gateway Condition: S3VpceIsRequired AesSiemDlq1CD8439D: Type: AWS::SQS::Queue Properties: KmsDataKeyReusePeriodSeconds: 86400 KmsMasterKeyId: alias/aws/sqs MessageRetentionPeriod: 1209600 QueueName: aes-siem-dlq UpdateReplacePolicy: Delete DeletionPolicy: Delete AesSiemSqsSplitLogs0191F431: Type: AWS::SQS::Queue Properties: KmsDataKeyReusePeriodSeconds: 86400 KmsMasterKeyId: alias/aws/sqs MessageRetentionPeriod: 1209600 QueueName: aes-siem-sqs-splitted-logs RedrivePolicy: deadLetterTargetArn: !GetAtt 'AesSiemDlq1CD8439D.Arn' maxReceiveCount: 20 VisibilityTimeout: 600 UpdateReplacePolicy: Delete DeletionPolicy: Delete LambdaEsLoaderServiceRoleFFD43869: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' LambdaEsLoaderServiceRoleDefaultPolicyB7A386B3: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - ec2:CreateNetworkInterface - ec2:DescribeNetworkInterfaces - ec2:DeleteNetworkInterface - ec2:AssignPrivateIpAddresses - ec2:UnassignPrivateIpAddresses Effect: Allow Resource: '*' - Action: sqs:SendMessage Effect: Allow Resource: !GetAtt 'AesSiemDlq1CD8439D.Arn' - Action: - sqs:SendMessage - sqs:ReceiveMessage - sqs:DeleteMessage - sqs:GetQueueAttributes Effect: Allow Resource: !GetAtt 'AesSiemDlq1CD8439D.Arn' - Action: - sqs:SendMessage - sqs:ReceiveMessage - sqs:DeleteMessage - sqs:GetQueueAttributes Effect: Allow Resource: !GetAtt 'AesSiemSqsSplitLogs0191F431.Arn' - Action: - sqs:ReceiveMessage - sqs:ChangeMessageVisibility - sqs:GetQueueUrl - sqs:DeleteMessage - sqs:GetQueueAttributes Effect: Allow Resource: !GetAtt 'AesSiemSqsSplitLogs0191F431.Arn' - Action: kms:Decrypt Effect: Allow Resource: !GetAtt 'KmsAesSiemLog44B26597.Arn' - Action: - s3:GetObject* - s3:GetBucket* - s3:List* Effect: Allow Resource: - !GetAtt 'S3BucketForGeoip04B5F171.Arn' - !Sub '${S3BucketForGeoip04B5F171.Arn}/*' - Action: - s3:GetObject* - s3:GetBucket* - s3:List* Effect: Allow Resource: - !GetAtt 'S3BucketForLog20898FE4.Arn' - !Sub '${S3BucketForLog20898FE4.Arn}/*' Version: '2012-10-17' PolicyName: LambdaEsLoaderServiceRoleDefaultPolicyB7A386B3 Roles: - !Ref 'LambdaEsLoaderServiceRoleFFD43869' LambdaEsLoader4B1E2DD9: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/es_loader.zip' Role: !GetAtt 'LambdaEsLoaderServiceRoleFFD43869.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' DeadLetterConfig: TargetArn: !GetAtt 'AesSiemDlq1CD8439D.Arn' Description: SIEM on Amazon OpenSearch Service v2.10.0 / es-loader Environment: Variables: AOSS_TYPE: !GetAtt 'AesSiemDomainDeployedR2.aoss_type' CONTROL_TOWER_LOG_BUCKETS: !Ref 'ControlTowerLogBucketNameList' CONTROL_TOWER_ROLE_ARN: !Ref 'ControlTowerRoleArnForEsLoader' CONTROL_TOWER_ROLE_SESSION_NAME: aes-siem-es-loader ENDPOINT: !GetAtt 'AesSiemDomainDeployedR2.endpoint' GEOIP_BUCKET: !Sub 'aes-siem-${AWS::AccountId}-geo' LOG_LEVEL: info POWERTOOLS_LOGGER_LOG_EVENT: 'false' POWERTOOLS_METRICS_NAMESPACE: SIEM POWERTOOLS_SERVICE_NAME: es-loader SECURITY_LAKE_EXTERNAL_ID: !Ref 'SecurityLakeExternalId' SECURITY_LAKE_ROLE_ARN: !Ref 'SecurityLakeRoleArn' SECURITY_LAKE_ROLE_SESSION_NAME: aes-siem-es-loader SQS_SPLITTED_LOGS_URL: !Ref 'AesSiemSqsSplitLogs0191F431' FunctionName: aes-siem-es-loader Handler: index.lambda_handler MemorySize: 2048 ReservedConcurrentExecutions: !Ref 'ReservedConcurrency' Runtime: python3.8 Timeout: 600 VpcConfig: SubnetIds: !GetAtt 'ExecCustomResourceValidator.subnets' SecurityGroupIds: !If - IsInVpc - - !GetAtt 'AesSiemVpcNoinboundSecurityGroup58555CE0.GroupId' - [] DependsOn: - LambdaEsLoaderServiceRoleDefaultPolicyB7A386B3 - LambdaEsLoaderServiceRoleFFD43869 LambdaEsLoaderCurrentVersion9DFE69550483136678b0fb8ed271a79421bcb99a: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaEsLoader4B1E2DD9' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain LambdaEsLoaderSqsEventSourceaessiemAesSiemSqsSplitLogs506AFFA6A7D8B2E9: Type: AWS::Lambda::EventSourceMapping Properties: FunctionName: !Ref 'LambdaEsLoader4B1E2DD9' BatchSize: 1 EventSourceArn: !GetAtt 'AesSiemSqsSplitLogs0191F431.Arn' LambdaAddPandasLayerRoleDCA80237: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' Policies: - PolicyDocument: Statement: - Action: - lambda:UpdateFunctionConfiguration - lambda:GetFunction Effect: Allow Resource: !GetAtt 'LambdaEsLoader4B1E2DD9.Arn' - Action: lambda:PublishLayerVersion Effect: Allow Resource: - arn:aws-us-gov:lambda:*:*:layer:AWSDataWrangler-* - arn:aws-us-gov:lambda:*:*:layer:AWSSDKPandas-* - Action: - lambda:ListLayers - lambda:GetLayerVersion Effect: Allow Resource: '*' - Action: - s3:Get* - s3:List* Effect: Allow Resource: '*' Version: '2012-10-17' PolicyName: add-pandas-layer-policy LambdaAddPandasLayerRoleDefaultPolicy6A268C17: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - s3:GetObject* - s3:GetBucket* - s3:List* - s3:DeleteObject* - s3:PutObject - s3:PutObjectLegalHold - s3:PutObjectRetention - s3:PutObjectTagging - s3:PutObjectVersionTagging - s3:Abort* Effect: Allow Resource: - !GetAtt 'S3BucketForGeoip04B5F171.Arn' - !Sub '${S3BucketForGeoip04B5F171.Arn}/*' Version: '2012-10-17' PolicyName: LambdaAddPandasLayerRoleDefaultPolicy6A268C17 Roles: - !Ref 'LambdaAddPandasLayerRoleDCA80237' LambdaAddPandasLayer8F3F6957: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/add_pandas_layer.zip' Role: !GetAtt 'LambdaAddPandasLayerRoleDCA80237.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / add-pandas-layer Environment: Variables: GEOIP_BUCKET: !Sub 'aes-siem-${AWS::AccountId}-geo' FunctionName: aes-siem-add-pandas-layer Handler: lambda_function.lambda_handler MemorySize: 128 ReservedConcurrentExecutions: 1 Runtime: python3.9 Timeout: 300 DependsOn: - LambdaAddPandasLayerRoleDefaultPolicy6A268C17 - LambdaAddPandasLayerRoleDCA80237 LambdaAddPandasLayerCurrentVersion52D075434f01b1fd65646a928c8322ec5ef2c616: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaAddPandasLayer8F3F6957' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain ExecLambdaAddPandasLayer: Type: AWS::CloudFormation::CustomResource Properties: ServiceToken: !GetAtt 'LambdaAddPandasLayer8F3F6957.Arn' ConfigVersion: 2.10.0 DependsOn: - LambdaAddPandasLayerRoleDefaultPolicy6A268C17 - LambdaAddPandasLayerRoleDCA80237 - LambdaEsLoaderCurrentVersion9DFE69550483136678b0fb8ed271a79421bcb99a - LambdaEsLoader4B1E2DD9 - LambdaEsLoaderServiceRoleDefaultPolicyB7A386B3 - LambdaEsLoaderServiceRoleFFD43869 - LambdaEsLoaderSqsEventSourceaessiemAesSiemSqsSplitLogs506AFFA6A7D8B2E9 DeletionPolicy: Retain LambdaEsLoaderStopperServiceRole83AABC1A: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' LambdaEsLoaderStopperServiceRoleDefaultPolicyCC98CC06: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: lambda:PutFunctionConcurrency Effect: Allow Resource: !GetAtt 'LambdaEsLoader4B1E2DD9.Arn' - Action: sns:Publish Effect: Allow Resource: !Ref 'SnsTopic2C1570A4' Version: '2012-10-17' PolicyName: LambdaEsLoaderStopperServiceRoleDefaultPolicyCC98CC06 Roles: - !Ref 'LambdaEsLoaderStopperServiceRole83AABC1A' LambdaEsLoaderStopper35C1D57B: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/es_loader_stopper.zip' Role: !GetAtt 'LambdaEsLoaderStopperServiceRole83AABC1A.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / es-loader-stopper Environment: Variables: AES_SIEM_ALERT_TOPIC_ARN: !Ref 'SnsTopic2C1570A4' ES_LOADER_FUNCTION_ARN: !GetAtt 'LambdaEsLoader4B1E2DD9.Arn' ES_LOADER_RESERVED_CONCURRENCY: !Ref 'ReservedConcurrency' FunctionName: aes-siem-es-loader-stopper Handler: index.lambda_handler MemorySize: 128 ReservedConcurrentExecutions: 1 Runtime: python3.9 Timeout: 300 DependsOn: - LambdaEsLoaderStopperServiceRoleDefaultPolicyCC98CC06 - LambdaEsLoaderStopperServiceRole83AABC1A LambdaEsLoaderStopperCurrentVersion312AA850c573dcca153a704a50da1e59f91d4e7b: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaEsLoaderStopper35C1D57B' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain LambdaMetricsExporterServiceRoleDDE0BD95: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' LambdaMetricsExporterServiceRoleDefaultPolicy5F0A7AC4: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - ec2:CreateNetworkInterface - ec2:DescribeNetworkInterfaces - ec2:DeleteNetworkInterface - ec2:AssignPrivateIpAddresses - ec2:UnassignPrivateIpAddresses Effect: Allow Resource: '*' Sid: ForVpcAccess - Action: - kms:Encrypt - kms:ReEncrypt* - kms:GenerateDataKey* Effect: Allow Resource: !GetAtt 'KmsAesSiemLog44B26597.Arn' - Action: - s3:DeleteObject* - s3:PutObject - s3:PutObjectLegalHold - s3:PutObjectRetention - s3:PutObjectTagging - s3:PutObjectVersionTagging - s3:Abort* Effect: Allow Resource: - !GetAtt 'S3BucketForLog20898FE4.Arn' - !Sub '${S3BucketForLog20898FE4.Arn}/*' Version: '2012-10-17' PolicyName: LambdaMetricsExporterServiceRoleDefaultPolicy5F0A7AC4 Roles: - !Ref 'LambdaMetricsExporterServiceRoleDDE0BD95' LambdaMetricsExporter2737F589: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/index_metrics_exporter.zip' Role: !GetAtt 'LambdaMetricsExporterServiceRoleDDE0BD95.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / index-metrics-exporter Environment: Variables: COLLECTION_NAME: !Ref 'DomainOrCollectionName' ENDPOINT: !GetAtt 'AesSiemDomainDeployedR2.endpoint' LOG_BUCKET: !Sub 'aes-siem-${AWS::AccountId}-log' PERIOD_HOUR: '1' FunctionName: aes-siem-index-metrics-exporter Handler: index.lambda_handler MemorySize: 256 ReservedConcurrentExecutions: 1 Runtime: python3.9 Timeout: 300 VpcConfig: SubnetIds: !GetAtt 'ExecCustomResourceValidator.subnets' SecurityGroupIds: !If - IsInVpc - - !GetAtt 'AesSiemVpcNoinboundSecurityGroup58555CE0.GroupId' - [] DependsOn: - LambdaMetricsExporterServiceRoleDefaultPolicy5F0A7AC4 - LambdaMetricsExporterServiceRoleDDE0BD95 LambdaMetricsExporterCurrentVersion79B0413F645e22a550159959c60a5f88fd6ed2b2: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaMetricsExporter2737F589' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain EventBridgeRuleLambdaMetricsExporterE956E7F2: Type: AWS::Events::Rule Properties: ScheduleExpression: rate(1 hour) State: ENABLED Targets: - Arn: !GetAtt 'LambdaMetricsExporter2737F589.Arn' Id: Target0 EventBridgeRuleLambdaMetricsExporterAllowEventRuleaessiemLambdaMetricsExporterCD618C3523DA2D6A: Type: AWS::Lambda::Permission Properties: Action: lambda:InvokeFunction FunctionName: !GetAtt 'LambdaMetricsExporter2737F589.Arn' Principal: events.amazonaws.com SourceArn: !GetAtt 'EventBridgeRuleLambdaMetricsExporterE956E7F2.Arn' LambdaGeoipDownloaderServiceRoleE37FB908: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' LambdaGeoipDownloaderServiceRoleDefaultPolicyE7B8AE65: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - s3:GetObject* - s3:GetBucket* - s3:List* - s3:DeleteObject* - s3:PutObject - s3:PutObjectLegalHold - s3:PutObjectRetention - s3:PutObjectTagging - s3:PutObjectVersionTagging - s3:Abort* Effect: Allow Resource: - !GetAtt 'S3BucketForGeoip04B5F171.Arn' - !Sub '${S3BucketForGeoip04B5F171.Arn}/*' Version: '2012-10-17' PolicyName: LambdaGeoipDownloaderServiceRoleDefaultPolicyE7B8AE65 Roles: - !Ref 'LambdaGeoipDownloaderServiceRoleE37FB908' LambdaGeoipDownloaderA5EFF97E: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/geoip_downloader.zip' Role: !GetAtt 'LambdaGeoipDownloaderServiceRoleE37FB908.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / geoip-downloader Environment: Variables: license_key: !Ref 'GeoLite2LicenseKey' s3bucket_name: !Sub 'aes-siem-${AWS::AccountId}-geo' FunctionName: aes-siem-geoip-downloader Handler: index.lambda_handler MemorySize: 320 ReservedConcurrentExecutions: 1 Runtime: python3.9 Timeout: 300 DependsOn: - LambdaGeoipDownloaderServiceRoleDefaultPolicyE7B8AE65 - LambdaGeoipDownloaderServiceRoleE37FB908 LambdaGeoipDownloaderCurrentVersion7F1CD34F168d0c32bbb5f780439e5136bb51456c: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaGeoipDownloaderA5EFF97E' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain ExecLambdaGeoipDownloader: Type: AWS::CloudFormation::CustomResource Properties: ServiceToken: !GetAtt 'LambdaGeoipDownloaderA5EFF97E.Arn' License: !Ref 'GeoLite2LicenseKey' DeletionPolicy: Retain EventBridgeRuleLambdaGeoipDownloaderAFA87072: Type: AWS::Events::Rule Properties: ScheduleExpression: rate(12 hours) State: !If - HasGeoipLicense - ENABLED - DISABLED Targets: - Arn: !GetAtt 'LambdaGeoipDownloaderA5EFF97E.Arn' Id: Target0 EventBridgeRuleLambdaGeoipDownloaderAllowEventRuleaessiemLambdaGeoipDownloaderC2D21D002CAED82D: Type: AWS::Lambda::Permission Properties: Action: lambda:InvokeFunction FunctionName: !GetAtt 'LambdaGeoipDownloaderA5EFF97E.Arn' Principal: events.amazonaws.com SourceArn: !GetAtt 'EventBridgeRuleLambdaGeoipDownloaderAFA87072.Arn' LambdaIocPlanServiceRole6CDE7C5D: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' LambdaIocPlanServiceRoleDefaultPolicyBAE5ADEE: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - s3:GetObject* - s3:GetBucket* - s3:List* - s3:DeleteObject* - s3:PutObject - s3:PutObjectLegalHold - s3:PutObjectRetention - s3:PutObjectTagging - s3:PutObjectVersionTagging - s3:Abort* Effect: Allow Resource: - !GetAtt 'S3BucketForGeoip04B5F171.Arn' - !Sub '${S3BucketForGeoip04B5F171.Arn}/*' Version: '2012-10-17' PolicyName: LambdaIocPlanServiceRoleDefaultPolicyBAE5ADEE Roles: - !Ref 'LambdaIocPlanServiceRole6CDE7C5D' LambdaIocPlan6E369BFB: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/ioc_database.zip' Role: !GetAtt 'LambdaIocPlanServiceRole6CDE7C5D.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / ioc-plan Environment: Variables: ABUSE_CH: !Ref 'EnableAbuseCh' GEOIP_BUCKET: !Sub 'aes-siem-${AWS::AccountId}-geo' LOG_LEVEL: INFO OTX_API_KEY: !Ref 'OtxApiKey' TOR: !Ref 'EnableTor' FunctionName: aes-siem-ioc-plan Handler: lambda_function.plan MemorySize: 128 ReservedConcurrentExecutions: 1 Runtime: python3.9 Timeout: 300 DependsOn: - LambdaIocPlanServiceRoleDefaultPolicyBAE5ADEE - LambdaIocPlanServiceRole6CDE7C5D LambdaIocPlanCurrentVersion1BAA7B4C3cd4c3ab065e3d10f76917db06a8ca5e: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaIocPlan6E369BFB' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain LambdaIocDownloadServiceRoleEB9001C6: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' LambdaIocDownloadServiceRoleDefaultPolicy9EDF8942: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - s3:ListAllMyBuckets - s3:PutFunctionConcurrency Effect: Allow Resource: '*' - Action: - s3:GetObject* - s3:GetBucket* - s3:List* - s3:DeleteObject* - s3:PutObject - s3:PutObjectLegalHold - s3:PutObjectRetention - s3:PutObjectTagging - s3:PutObjectVersionTagging - s3:Abort* Effect: Allow Resource: - !GetAtt 'S3BucketForGeoip04B5F171.Arn' - !Sub '${S3BucketForGeoip04B5F171.Arn}/*' Version: '2012-10-17' PolicyName: LambdaIocDownloadServiceRoleDefaultPolicy9EDF8942 Roles: - !Ref 'LambdaIocDownloadServiceRoleEB9001C6' LambdaIocDownload5519716E: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/ioc_database.zip' Role: !GetAtt 'LambdaIocDownloadServiceRoleEB9001C6.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / ioc-download Environment: Variables: GEOIP_BUCKET: !Sub 'aes-siem-${AWS::AccountId}-geo' LOG_LEVEL: INFO OTX_API_KEY: !Ref 'OtxApiKey' FunctionName: aes-siem-ioc-download Handler: lambda_function.download MemorySize: 384 Runtime: python3.9 Timeout: 900 DependsOn: - LambdaIocDownloadServiceRoleDefaultPolicy9EDF8942 - LambdaIocDownloadServiceRoleEB9001C6 LambdaIocDownloadCurrentVersion48773E6B25d00c408dbf835f063b066d2f8b4258: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaIocDownload5519716E' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain LambdaIocCreatedbServiceRole555565C0: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' LambdaIocCreatedbServiceRoleDefaultPolicyA06805A4: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - s3:GetObject* - s3:GetBucket* - s3:List* - s3:DeleteObject* - s3:PutObject - s3:PutObjectLegalHold - s3:PutObjectRetention - s3:PutObjectTagging - s3:PutObjectVersionTagging - s3:Abort* Effect: Allow Resource: - !GetAtt 'S3BucketForGeoip04B5F171.Arn' - !Sub '${S3BucketForGeoip04B5F171.Arn}/*' Version: '2012-10-17' PolicyName: LambdaIocCreatedbServiceRoleDefaultPolicyA06805A4 Roles: - !Ref 'LambdaIocCreatedbServiceRole555565C0' LambdaIocCreatedb04F35777: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/ioc_database.zip' Role: !GetAtt 'LambdaIocCreatedbServiceRole555565C0.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / ioc-createdb Environment: Variables: GEOIP_BUCKET: !Sub 'aes-siem-${AWS::AccountId}-geo' LOG_LEVEL: INFO FunctionName: aes-siem-ioc-createdb Handler: lambda_function.createdb MemorySize: 1024 Runtime: python3.9 Timeout: 900 DependsOn: - LambdaIocCreatedbServiceRoleDefaultPolicyA06805A4 - LambdaIocCreatedbServiceRole555565C0 LambdaIocCreatedbCurrentVersionD9D04E9Cc6a582bf5661dc7434b1a73c2714ec23: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaIocCreatedb04F35777' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain IocStateMachineLogGroupC1AB417E: Type: AWS::Logs::LogGroup Properties: LogGroupName: /aws/vendedlogs/states/aes-siem-ioc-logs RetentionInDays: 30 UpdateReplacePolicy: Delete DeletionPolicy: Delete IocStateMachineRole095AC46F: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: states.amazonaws.com Version: '2012-10-17' IocStateMachineRoleDefaultPolicy2B85C920: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - logs:CreateLogDelivery - logs:GetLogDelivery - logs:UpdateLogDelivery - logs:DeleteLogDelivery - logs:ListLogDeliveries - logs:PutResourcePolicy - logs:DescribeResourcePolicies - logs:DescribeLogGroups Effect: Allow Resource: '*' - Action: lambda:InvokeFunction Effect: Allow Resource: - !GetAtt 'LambdaIocPlan6E369BFB.Arn' - !Sub '${LambdaIocPlan6E369BFB.Arn}:*' - Action: lambda:InvokeFunction Effect: Allow Resource: - !GetAtt 'LambdaIocCreatedb04F35777.Arn' - !Sub '${LambdaIocCreatedb04F35777.Arn}:*' - Action: lambda:InvokeFunction Effect: Allow Resource: - !GetAtt 'LambdaIocDownload5519716E.Arn' - !Sub '${LambdaIocDownload5519716E.Arn}:*' Version: '2012-10-17' PolicyName: IocStateMachineRoleDefaultPolicy2B85C920 Roles: - !Ref 'IocStateMachineRole095AC46F' IocStateMachine01BE6E56: Type: AWS::StepFunctions::StateMachine Properties: RoleArn: !GetAtt 'IocStateMachineRole095AC46F.Arn' DefinitionString: !Sub '{"StartAt":"IocPlan","States":{"IocPlan":{"Next":"need to download?","Retry":[{"ErrorEquals":["Lambda.ServiceException","Lambda.AWSLambdaException","Lambda.SdkClientException"],"IntervalSeconds":2,"MaxAttempts":6,"BackoffRate":2}],"Type":"Task","OutputPath":"$.Payload","Resource":"arn:${AWS::Partition}:states:::lambda:invoke","Parameters":{"FunctionName":"${LambdaIocPlan6E369BFB.Arn}","Payload":""}},"need to download?":{"Type":"Choice","Choices":[{"Variable":"$.mapped[0].ioc","IsPresent":false,"Next":"SkipDownload"}],"Default":"MapDownload"},"MapDownload":{"Type":"Map","Next":"IocCreatedb","Parameters":{"mapped.$":"$$.Map.Item.Value"},"Iterator":{"StartAt":"IocDownload","States":{"IocDownload":{"End":true,"Retry":[{"ErrorEquals":["Lambda.ServiceException","Lambda.AWSLambdaException","Lambda.SdkClientException"],"IntervalSeconds":2,"MaxAttempts":6,"BackoffRate":2}],"Catch":[{"ErrorEquals":["States.Timeout","States.TaskFailed"],"ResultPath":"$.catcher","Next":"IgnoreTimeout"}],"Type":"Task","TimeoutSeconds":899,"OutputPath":"$.Payload","Resource":"arn:${AWS::Partition}:states:::lambda:invoke","Parameters":{"FunctionName":"${LambdaIocDownload5519716E.Arn}","Payload.$":"$"}},"IgnoreTimeout":{"Type":"Pass","End":true}}},"ItemsPath":"$.mapped","MaxConcurrency":4},"IocCreatedb":{"End":true,"Retry":[{"ErrorEquals":["Lambda.ServiceException","Lambda.AWSLambdaException","Lambda.SdkClientException"],"IntervalSeconds":2,"MaxAttempts":6,"BackoffRate":2}],"Type":"Task","Resource":"arn:${AWS::Partition}:states:::lambda:invoke","Parameters":{"FunctionName":"${LambdaIocCreatedb04F35777.Arn}","Payload.$":"$"}},"SkipDownload":{"Type":"Pass","End":true}},"TimeoutSeconds":3600}' LoggingConfiguration: Destinations: - CloudWatchLogsLogGroup: LogGroupArn: !GetAtt 'IocStateMachineLogGroupC1AB417E.Arn' Level: ALL StateMachineName: aes-siem-ioc-state-machine DependsOn: - IocStateMachineRoleDefaultPolicy2B85C920 - IocStateMachineRole095AC46F UpdateReplacePolicy: Delete DeletionPolicy: Delete IocStateMachineEventsRoleC1BF1919: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: events.amazonaws.com Version: '2012-10-17' IocStateMachineEventsRoleDefaultPolicyC412ACCE: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: states:StartExecution Effect: Allow Resource: !Ref 'IocStateMachine01BE6E56' Version: '2012-10-17' PolicyName: IocStateMachineEventsRoleDefaultPolicyC412ACCE Roles: - !Ref 'IocStateMachineEventsRoleC1BF1919' EventBridgeRuleStepFunctionsIoc6CE991F5: Type: AWS::Events::Rule Properties: ScheduleExpression: !Sub 'rate(${IocDownloadInterval} minutes)' State: !If - EnableIOC - ENABLED - DISABLED Targets: - Arn: !Ref 'IocStateMachine01BE6E56' Id: Target0 RoleArn: !GetAtt 'IocStateMachineEventsRoleC1BF1919.Arn' AesSiemDeployRoleForLambda654D64F2: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' Policies: - PolicyDocument: Statement: - Action: - logs:PutResourcePolicy - logs:DescribeLogGroups - logs:DescribeLogStreams Effect: Allow Resource: !Sub 'arn:aws-us-gov:logs:${AWS::Region}:${AWS::AccountId}:*' - Action: - logs:CreateLogGroup - logs:CreateLogStream - logs:PutLogEvents - logs:PutRetentionPolicy Effect: Allow Resource: - !Sub 'arn:aws-us-gov:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/aes/domains/${DomainOrCollectionName}/*' - !Sub 'arn:aws-us-gov:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/OpenSearchService/domains/${DomainOrCollectionName}/*' - !Sub 'arn:aws-us-gov:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/aes-siem-*' Version: '2012-10-17' PolicyName: cwl_loggroup - PolicyDocument: Statement: - Action: - ec2:DescribeVpcEndpoints - es:CreateDomain - es:DescribeDomain - es:ESHttp* - es:UpdateDomainConfig - iam:GetRole Effect: Allow Resource: '*' - Action: iam:CreateServiceLinkedRole Effect: Allow Resource: - !Sub 'arn:aws-us-gov:iam::${AWS::AccountId}:role/aws-service-role/observability.aoss.amazonaws.com/AWSServiceRoleForAmazonOpenSearchServerless' - !Sub 'arn:aws-us-gov:iam::${AWS::AccountId}:role/aws-service-role/opensearchservice.amazonaws.com/AWSServiceRoleForAmazonOpenSearchService' - !Sub 'arn:aws-us-gov:iam::${AWS::AccountId}:role/aws-service-role/es.amazonaws.com/AWSServiceRoleForAmazonElasticsearchService' - Action: - aoss:APIAccessAll - aoss:DashboardsAccessAll - aoss:BatchGetCollection - aoss:BatchGetVpcEndpoint - aoss:CreateCollection - aoss:CreateSecurityPolicy - aoss:GetSecurityPolicy - aoss:UpdateSecurityPolicy Effect: Allow Resource: '*' Version: '2012-10-17' PolicyName: opensearch_deployment - PolicyDocument: Statement: - Action: - events:DeleteRule - events:ListRules - events:PutRule - events:PutTargets - events:RemoveTargets - lambda:AddPermission - lambda:RemovePermission Effect: Allow Resource: '*' Version: '2012-10-17' PolicyName: crhelper - PolicyDocument: Statement: - Action: iam:PassRole Effect: Allow Resource: !GetAtt 'AesSiemSnapshotRoleF313ED39.Arn' Version: '2012-10-17' PolicyName: assume_snapshotrole - PolicyDocument: Statement: - Action: s3:ListBucket Effect: Allow Resource: !GetAtt 'S3BucketForSnapshot40E67D36.Arn' - Action: - s3:GetObject - s3:PutObject Effect: Allow Resource: !Sub '${S3BucketForSnapshot40E67D36.Arn}/*' Version: '2012-10-17' PolicyName: s3access RoleName: aes-siem-deploy-role-for-lambda AesSiemDeployRoleForLambdaDefaultPolicy4129DE4C: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: - ec2:CreateNetworkInterface - ec2:DescribeNetworkInterfaces - ec2:DeleteNetworkInterface - ec2:AssignPrivateIpAddresses - ec2:UnassignPrivateIpAddresses Effect: Allow Resource: '*' Version: '2012-10-17' PolicyName: AesSiemDeployRoleForLambdaDefaultPolicy4129DE4C Roles: - !Ref 'AesSiemDeployRoleForLambda654D64F2' AossDataAccessPolicyForDeployment: Type: AWS::OpenSearchServerless::AccessPolicy Properties: Name: siem-data-access-for-deploymet Policy: !Sub '[{"Description":"For SIEM Deployment","Principal":["${AesSiemDeployRoleForLambda654D64F2.Arn}"],"Rules":[{"Resource":["collection/${DomainOrCollectionName}"],"Permission":["aoss:CreateCollectionItems","aoss:UpdateCollectionItems"],"ResourceType":"collection"},{"Resource":["index/${DomainOrCollectionName}/*"],"Permission":["aoss:*"],"ResourceType":"index"}]},{"Description":"For SIEM Deployment2","Principal":["${AesSiemDeployRoleForLambda654D64F2.Arn}"],"Rules":[{"Resource":["collection/*"],"Permission":["aoss:DescribeCollectionItems"],"ResourceType":"collection"}]}]' Type: data Description: Created By SIEM Solution. DO NOT EDIT Condition: IsServerless AossDataAccessPolicyForLoader: Type: AWS::OpenSearchServerless::AccessPolicy Properties: Name: siem-data-access-for-loader Policy: !Sub '[{"Description":"For SIEM Loader","Principal":["${LambdaEsLoaderServiceRoleFFD43869.Arn}"],"Rules":[{"Resource":["index/${DomainOrCollectionName}/log-*","index/${DomainOrCollectionName}/metrics-*"],"Permission":["aoss:CreateIndex","aoss:UpdateIndex","aoss:ReadDocument","aoss:WriteDocument"],"ResourceType":"index"}]},{"Description":"For SIEM metrics exporter","Principal":["${LambdaMetricsExporterServiceRoleDDE0BD95.Arn}"],"Rules":[{"Resource":["index/${DomainOrCollectionName}/*"],"Permission":["aoss:DescribeIndex"],"ResourceType":"index"}]}]' Type: data Description: Created By SIEM Solution. DO NOT EDIT Condition: IsServerless LambdaDeployAES636B5079: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/deploy_es.zip' Role: !GetAtt 'AesSiemDeployRoleForLambda654D64F2.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / opensearch domain deployment Environment: Variables: ACCOUNT_ID: !Ref 'AWS::AccountId' ALLOWED_SOURCE_ADDRESSES: !Ref 'AllowedSourceIpAddresses' DEPLOYMENT_TARGET: !Ref 'DeploymentTarget' DOMAIN_OR_COLLECTION_NAME: !Ref 'DomainOrCollectionName' ROLE_AOS_ADMIN: !GetAtt 'AesSiemDeployRoleForLambda654D64F2.Arn' S3_SNAPSHOT: !Ref 'S3BucketForSnapshot40E67D36' SOLUTION_PREFIX: aes-siem VPCE_ID: !Ref 'VpcEndpointId' FunctionName: aes-siem-deploy-aes Handler: index.aes_domain_handler MemorySize: 128 ReservedConcurrentExecutions: 1 Runtime: python3.9 Timeout: 300 DependsOn: - AesSiemDeployRoleForLambdaDefaultPolicy4129DE4C - AesSiemDeployRoleForLambda654D64F2 LambdaDeployAESCurrentVersion047915F1ce3da89df1ffa0da78c111b3a75f8916: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaDeployAES636B5079' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain AesSiemDomainDeployedR2: Type: AWS::CloudFormation::CustomResource Properties: ServiceToken: !GetAtt 'LambdaDeployAES636B5079.Arn' ConfigVersion: 2.10.0 Target: !Ref 'DeploymentTarget' Name: !Ref 'DomainOrCollectionName' vpce: !Ref 'VpcEndpointId' DependsOn: - AesSiemDeployRoleForLambdaDefaultPolicy4129DE4C - AesSiemDeployRoleForLambda654D64F2 LambdaConfigureAESA0471961: Type: AWS::Lambda::Function Properties: Code: S3Bucket: '%%BUCKET_NAME%%' S3Key: '%%SOLUTION_NAME%%/%%VERSION%%/assets/deploy_es.zip' Role: !GetAtt 'AesSiemDeployRoleForLambda654D64F2.Arn' Architectures: !If - HasLambdaArchitecturesProp - - !FindInMap - RegionMap - !Ref 'AWS::Region' - LambdaArch - !Ref 'AWS::NoValue' Description: SIEM on Amazon OpenSearch Service v2.10.0 / opensearch configuration Environment: Variables: ACCOUNT_ID: !Ref 'AWS::AccountId' DOMAIN_OR_COLLECTION_NAME: !Ref 'DomainOrCollectionName' ENDPOINT: !GetAtt 'AesSiemDomainDeployedR2.endpoint' ROLE_AOS_ADMIN: !GetAtt 'AesSiemDeployRoleForLambda654D64F2.Arn' ROLE_ES_LOADER: !GetAtt 'LambdaEsLoaderServiceRoleFFD43869.Arn' ROLE_METRICS_EXPORTER: !GetAtt 'LambdaMetricsExporterServiceRoleDDE0BD95.Arn' ROLE_SNAPSHOT: !GetAtt 'AesSiemSnapshotRoleF313ED39.Arn' S3_SNAPSHOT: !Ref 'S3BucketForSnapshot40E67D36' SOLUTION_PREFIX: aes-siem FunctionName: aes-siem-configure-aes Handler: index.aes_config_handler MemorySize: 128 ReservedConcurrentExecutions: 1 Runtime: python3.9 Timeout: 600 VpcConfig: SubnetIds: !GetAtt 'ExecCustomResourceValidator.subnets' SecurityGroupIds: !If - IsInVpc - - !GetAtt 'AesSiemVpcNoinboundSecurityGroup58555CE0.GroupId' - [] DependsOn: - AesSiemDeployRoleForLambdaDefaultPolicy4129DE4C - AesSiemDeployRoleForLambda654D64F2 LambdaConfigureAESCurrentVersion65114768b51be324d3f92d1e341a6842ba118176: Type: AWS::Lambda::Version Properties: FunctionName: !Ref 'LambdaConfigureAESA0471961' Description: 2.10.0 UpdateReplacePolicy: Retain DeletionPolicy: Retain AesSiemDomainConfiguredR2: Type: AWS::CloudFormation::CustomResource Properties: ServiceToken: !GetAtt 'LambdaConfigureAESA0471961.Arn' ConfigVersion: 2.10.0 Target: !Ref 'DeploymentTarget' Name: !Ref 'DomainOrCollectionName' DependsOn: - AesSiemDomainDeployedR2 DeletionPolicy: Retain aessiempolicytoloadentriestoesE6506021: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: es:ESHttp* Effect: Allow Resource: !Sub 'arn:aws-us-gov:es:${AWS::Region}:${AWS::AccountId}:domain/${DomainOrCollectionName}/*' - Action: aoss:APIAccessAll Effect: Allow Resource: !Sub - arn:aws-us-gov:aoss:${AWS::Region}:${AWS::AccountId}:collection/${Param1} - Param1: !Select - 0 - !Split - . - !GetAtt 'AesSiemDomainDeployedR2.endpoint' Version: '2012-10-17' PolicyName: aes-siem-policy-to-load-entries-to-es Roles: - !Ref 'LambdaEsLoaderServiceRoleFFD43869' - !Ref 'AesSiemEsLoaderEC2RoleFE3F9F00' - !Ref 'LambdaMetricsExporterServiceRoleDDE0BD95' aessiempolicytogetparametersbypathB5BFC6F0: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: ssm:GetParametersByPath Effect: Allow Resource: !Sub 'arn:aws-us-gov:ssm:*:${AWS::AccountId}:parameter/siem/exclude-logs/*' Version: '2012-10-17' PolicyName: aes-siem-policy-to-get-parameters-by-path Roles: - !Ref 'LambdaEsLoaderServiceRoleFFD43869' BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Principal: Service: lambda.amazonaws.com Version: '2012-10-17' ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: s3:PutBucketNotification Effect: Allow Resource: '*' Version: '2012-10-17' PolicyName: BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36 Roles: - !Ref 'BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC' BucketNotificationsHandler050a0587b7544547bf325f094a3db8347ECC3691: Type: AWS::Lambda::Function Properties: Description: AWS CloudFormation handler for "Custom::S3BucketNotifications" resources (@aws-cdk/aws-s3) Code: ZipFile: | import boto3 # type: ignore import json import logging import urllib.request s3 = boto3.client("s3") EVENTBRIDGE_CONFIGURATION = 'EventBridgeConfiguration' CONFIGURATION_TYPES = ["TopicConfigurations", "QueueConfigurations", "LambdaFunctionConfigurations"] def handler(event: dict, context): response_status = "SUCCESS" error_message = "" try: props = event["ResourceProperties"] bucket = props["BucketName"] notification_configuration = props["NotificationConfiguration"] request_type = event["RequestType"] managed = props.get('Managed', 'true').lower() == 'true' stack_id = event['StackId'] if managed: config = handle_managed(request_type, notification_configuration) else: config = handle_unmanaged(bucket, stack_id, request_type, notification_configuration) put_bucket_notification_configuration(bucket, config) except Exception as e: logging.exception("Failed to put bucket notification configuration") response_status = "FAILED" error_message = f"Error: {str(e)}. " finally: submit_response(event, context, response_status, error_message) def handle_managed(request_type, notification_configuration): if request_type == 'Delete': return {} return notification_configuration def handle_unmanaged(bucket, stack_id, request_type, notification_configuration): external_notifications = find_external_notifications(bucket, stack_id) if request_type == 'Delete': return external_notifications def with_id(notification): notification['Id'] = f"{stack_id}-{hash(json.dumps(notification, sort_keys=True))}" return notification notifications = {} for t in CONFIGURATION_TYPES: external = external_notifications.get(t, []) incoming = [with_id(n) for n in notification_configuration.get(t, [])] notifications[t] = external + incoming if EVENTBRIDGE_CONFIGURATION in notification_configuration: notifications[EVENTBRIDGE_CONFIGURATION] = notification_configuration[EVENTBRIDGE_CONFIGURATION] elif EVENTBRIDGE_CONFIGURATION in external_notifications: notifications[EVENTBRIDGE_CONFIGURATION] = external_notifications[EVENTBRIDGE_CONFIGURATION] return notifications def find_external_notifications(bucket, stack_id): existing_notifications = get_bucket_notification_configuration(bucket) external_notifications = {} for t in CONFIGURATION_TYPES: external_notifications[t] = [n for n in existing_notifications.get(t, []) if not n['Id'].startswith(f"{stack_id}-")] if EVENTBRIDGE_CONFIGURATION in existing_notifications: external_notifications[EVENTBRIDGE_CONFIGURATION] = existing_notifications[EVENTBRIDGE_CONFIGURATION] return external_notifications def get_bucket_notification_configuration(bucket): return s3.get_bucket_notification_configuration(Bucket=bucket) def put_bucket_notification_configuration(bucket, notification_configuration): s3.put_bucket_notification_configuration(Bucket=bucket, NotificationConfiguration=notification_configuration) def submit_response(event: dict, context, response_status: str, error_message: str): response_body = json.dumps( { "Status": response_status, "Reason": f"{error_message}See the details in CloudWatch Log Stream: {context.log_stream_name}", "PhysicalResourceId": event.get("PhysicalResourceId") or event["LogicalResourceId"], "StackId": event["StackId"], "RequestId": event["RequestId"], "LogicalResourceId": event["LogicalResourceId"], "NoEcho": False, } ).encode("utf-8") headers = {"content-type": "", "content-length": str(len(response_body))} try: req = urllib.request.Request(url=event["ResponseURL"], headers=headers, data=response_body, method="PUT") with urllib.request.urlopen(req) as response: print(response.read().decode("utf-8")) print("Status code: " + response.reason) except Exception as e: print("send(..) failed executing request.urlopen(..): " + str(e)) Handler: index.handler Role: !GetAtt 'BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC.Arn' Runtime: python3.9 Timeout: 300 DependsOn: - BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36 - BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC SnsTopic2C1570A4: Type: AWS::SNS::Topic Properties: DisplayName: AES SIEM KmsMasterKeyId: !GetAtt 'KmsAesSiemLog44B26597.Arn' TopicName: aes-siem-alert SnsTopicPolicy520DD923: Type: AWS::SNS::TopicPolicy Properties: PolicyDocument: Statement: - Action: sns:Publish Effect: Allow Principal: Service: events.amazonaws.com Resource: !Ref 'SnsTopic2C1570A4' Sid: '0' Version: '2012-10-17' Topics: - !Ref 'SnsTopic2C1570A4' SnsTopicTokenSubscriptionD7374A11: Type: AWS::SNS::Subscription Properties: Protocol: email TopicArn: !Ref 'SnsTopic2C1570A4' Endpoint: !Ref 'SnsEmail' Condition: HasSnsEmail EventBridgeRuleAosNotificationsC282068B: Type: AWS::Events::Rule Properties: EventPattern: resources: - !Sub 'arn:aws-us-gov:es:${AWS::Region}:${AWS::AccountId}:domain/${DomainOrCollectionName}' source: - aws.es State: ENABLED Targets: - Arn: !Ref 'SnsTopic2C1570A4' Id: Target0 Condition: HasSnsEmail accesstocontroltowerlogbucketsDFC3958F: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Resource: !Ref 'ControlTowerRoleArnForEsLoader' - Action: - sqs:ReceiveMessage - sqs:ChangeMessageVisibility - sqs:GetQueueUrl - sqs:DeleteMessage - sqs:GetQueueAttributes Effect: Allow Resource: !Sub - arn:aws-us-gov:sqs:*:${Param1}:* - Param1: !Select - 4 - !Split - ':' - !Ref 'ControlTowerRoleArnForEsLoader' Version: '2012-10-17' PolicyName: access_to_control_tower Roles: - !Ref 'LambdaEsLoaderServiceRoleFFD43869' Condition: IsControlTowerAcccess EventSourceMappingForCT63FDE1BA: Type: AWS::Lambda::EventSourceMapping Properties: FunctionName: !Ref 'LambdaEsLoader4B1E2DD9' EventSourceArn: !Ref 'ControlTowerSqsForLogBuckets' Condition: IsControlTowerAcccess accesstosecuritylakelogbuckets32CCAD02: Type: AWS::IAM::Policy Properties: PolicyDocument: Statement: - Action: sts:AssumeRole Effect: Allow Resource: !Ref 'SecurityLakeRoleArn' - Action: - sqs:ReceiveMessage - sqs:ChangeMessageVisibility - sqs:GetQueueUrl - sqs:DeleteMessage - sqs:GetQueueAttributes Effect: Allow Resource: !Sub - arn:aws-us-gov:sqs:*:${Param1}:* - Param1: !Select - 4 - !Split - ':' - !Ref 'SecurityLakeRoleArn' Version: '2012-10-17' PolicyName: access_to_security_lake Roles: - !Ref 'LambdaEsLoaderServiceRoleFFD43869' Condition: IsSecurityLakeAcccess EventSourceMappingForCT230540116: Type: AWS::Lambda::EventSourceMapping Properties: FunctionName: !Ref 'LambdaEsLoader4B1E2DD9' EventSourceArn: !Ref 'SecurityLakeSubscriberSqs' Condition: IsSecurityLakeAcccess TotalFreeStorageSpaceRemainsLowAlarm3888040E: Type: AWS::CloudWatch::Alarm Properties: ComparisonOperator: LessThanOrEqualToThreshold EvaluationPeriods: 30 AlarmDescription: Triggered when total free space for the cluster remains less 200MB for 30 minutes. Dimensions: - Name: ClientId Value: !Ref 'AWS::AccountId' - Name: DomainName Value: !Ref 'DomainOrCollectionName' MetricName: FreeStorageSpace Namespace: AWS/ES Period: 60 Statistic: Sum Threshold: 200 EsLoaderStopperRule2C1D9F30: Type: AWS::Events::Rule Properties: EventPattern: detail-type: - CloudWatch Alarm State Change resources: - !GetAtt 'TotalFreeStorageSpaceRemainsLowAlarm3888040E.Arn' source: - aws.cloudwatch State: ENABLED Targets: - Arn: !GetAtt 'LambdaEsLoaderStopper35C1D57B.Arn' Id: Target0 EsLoaderStopperRuleAllowEventRuleaessiemLambdaEsLoaderStopper325F7AA593386C47: Type: AWS::Lambda::Permission Properties: Action: lambda:InvokeFunction FunctionName: !GetAtt 'LambdaEsLoaderStopper35C1D57B.Arn' Principal: events.amazonaws.com SourceArn: !GetAtt 'EsLoaderStopperRule2C1D9F30.Arn' SIEMDashboard35199390: Type: AWS::CloudWatch::Dashboard Properties: DashboardBody: !Sub '{"widgets":[{"type":"text","width":24,"height":1,"x":0,"y":0,"properties":{"markdown":"# CloudWatch Alarm"}},{"type":"metric","width":6,"height":6,"x":0,"y":1,"properties":{"view":"timeSeries","title":"${TotalFreeStorageSpaceRemainsLowAlarm3888040E}","region":"${AWS::Region}","annotations":{"alarms":["${TotalFreeStorageSpaceRemainsLowAlarm3888040E.Arn}"]},"yAxis":{}}},{"type":"text","width":24,"height":1,"x":0,"y":7,"properties":{"markdown":"# Lambda Function: ${LambdaEsLoader4B1E2DD9}"}},{"type":"metric","width":12,"height":4,"x":0,"y":8,"properties":{"view":"timeSeries","title":"Error count and success rate (%)","region":"${AWS::Region}","metrics":[["AWS/Lambda","Errors","FunctionName","${LambdaEsLoader4B1E2DD9}",{"color":"#d13212","label":"Errors (Count)","stat":"Sum","id":"errors"}],[{"label":"Success rate (%)","color":"#2ca02c","expression":"100 - 100 * errors / MAX([errors, invocations])","yAxis":"right"}],["AWS/Lambda","Invocations","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Sum","visible":false,"id":"invocations"}]],"yAxis":{"left":{"showUnits":false},"right":{"max":100,"showUnits":false}},"period":60}},{"type":"metric","width":12,"height":4,"x":12,"y":8,"properties":{"view":"timeSeries","title":"Invocations (Count)","region":"${AWS::Region}","metrics":[["AWS/Lambda","Invocations","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"},"period":60}},{"type":"metric","width":12,"height":4,"x":0,"y":12,"properties":{"view":"timeSeries","title":"Duration (Milliseconds)","region":"${AWS::Region}","metrics":[["AWS/Lambda","Duration","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Minimum"}],["AWS/Lambda","Duration","FunctionName","${LambdaEsLoader4B1E2DD9}"],["AWS/Lambda","Duration","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Maximum"}]],"yAxis":{"left":{"showUnits":false}},"period":60}},{"type":"metric","width":12,"height":4,"x":12,"y":12,"properties":{"view":"timeSeries","title":"Throttles (Count)","region":"${AWS::Region}","metrics":[["AWS/Lambda","Throttles","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"},"period":60}},{"type":"log","width":12,"height":4,"x":0,"y":16,"properties":{"view":"table","title":"Longest 5 invocations","region":"${AWS::Region}","query":"SOURCE ''/aws/lambda/${LambdaEsLoader4B1E2DD9}'' | fields @timestamp, @duration, @requestId\n | sort @duration desc\n | head 5"}},{"type":"metric","width":12,"height":4,"x":12,"y":16,"properties":{"view":"timeSeries","title":"ConcurrentExecutions (Count)","region":"${AWS::Region}","metrics":[["AWS/Lambda","ConcurrentExecutions","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Maximum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"},"period":60}},{"type":"text","width":24,"height":1,"x":0,"y":20,"properties":{"markdown":"# OpenSearch Service: ${DomainOrCollectionName} domain"}},{"type":"metric","width":12,"height":4,"x":0,"y":21,"properties":{"view":"timeSeries","title":"Data Node CPUUtilization (Cluster Max Percentage)","region":"${AWS::Region}","metrics":[["AWS/ES","CPUUtilization","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"stat":"Maximum"}]],"yAxis":{"left":{"max":100,"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"metric","width":12,"height":4,"x":12,"y":21,"properties":{"view":"timeSeries","title":"Data Node JVMMemoryPressure (Cluster Max Percentage)","region":"${AWS::Region}","metrics":[["AWS/ES","JVMMemoryPressure","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}"]],"yAxis":{"left":{"max":100,"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"metric","width":12,"height":4,"x":0,"y":25,"properties":{"view":"timeSeries","title":"HTTP requests by error response code (Cluster Total Count)","region":"${AWS::Region}","metrics":[["AWS/ES","4xx","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"stat":"Sum"}],["AWS/ES","5xx","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":25,"properties":{"view":"timeSeries","title":"Active Shards Count","region":"${AWS::Region}","metrics":[["AWS/ES","Shards.active","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}"],["AWS/ES","Shards.activePrimary","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}"]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":0,"y":29,"properties":{"view":"timeSeries","title":"Cluster DiskThroughputThrottle","region":"${AWS::Region}","metrics":[["AWS/ES","ThroughputThrottle","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"Cluster Disk ThroughputThrottle","stat":"Maximum"}]],"annotations":{"horizontal":[{"value":1,"yAxis":"left"}]},"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":29,"properties":{"view":"timeSeries","title":"ClusterIndexWritesBlocked (Cluster Max Count)","region":"${AWS::Region}","metrics":[["AWS/ES","ClusterIndexWritesBlocked","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}"]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"text","width":12,"height":1,"x":0,"y":33,"properties":{"markdown":"# Read / Search"}},{"type":"text","width":12,"height":1,"x":12,"y":33,"properties":{"markdown":"# Write / Indexing"}},{"type":"metric","width":12,"height":4,"x":0,"y":34,"properties":{"view":"timeSeries","title":"EBS Read Throughput / IOPS","region":"${AWS::Region}","metrics":[["AWS/ES","ReadThroughput","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"ReadThroughput (Bytes/Second)","stat":"Maximum"}],["AWS/ES","ReadIOPS","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"ReadIOPS (Count/Second)","stat":"Maximum","yAxis":"right"}]],"yAxis":{"left":{"showUnits":false},"right":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":34,"properties":{"view":"timeSeries","title":"EBS Write Throughput / IOPS","region":"${AWS::Region}","metrics":[["AWS/ES","WriteThroughput","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"WriteThroughput (Bytes/Second)","stat":"Maximum"}],["AWS/ES","WriteIOPS","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"WriteIOPS (Count/Second)","stat":"Maximum","yAxis":"right"}]],"yAxis":{"left":{"showUnits":false},"right":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":0,"y":38,"properties":{"view":"timeSeries","title":"EBS Read Latency / Disk Queue","region":"${AWS::Region}","metrics":[["AWS/ES","ReadLatency","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"ReadLatency (Seconds)","stat":"Maximum"}],["AWS/ES","DiskQueueDepth","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"DiskQueueDepth (Count)","stat":"Maximum","yAxis":"right"}]],"yAxis":{"left":{"showUnits":false},"right":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":38,"properties":{"view":"timeSeries","title":"EBS Write Latency / Disk Queue","region":"${AWS::Region}","metrics":[["AWS/ES","WriteLatency","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"WriteLatency (Seconds)","stat":"Maximum"}],["AWS/ES","DiskQueueDepth","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"DiskQueueDepth (Count)","stat":"Maximum","yAxis":"right"}]],"yAxis":{"left":{"showUnits":false},"right":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":0,"y":42,"properties":{"view":"timeSeries","title":"Search Rate / Latency (Node Average)","region":"${AWS::Region}","metrics":[["AWS/ES","SearchRate","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"SearchRate (Count)"}],["AWS/ES","SearchLatency","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"SearchLatency (Milliseconds)","yAxis":"right"}]],"yAxis":{"left":{"showUnits":false},"right":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":42,"properties":{"view":"timeSeries","title":"Indexing Rate / Latency (Node Average)","region":"${AWS::Region}","metrics":[["AWS/ES","IndexingRate","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"IndexingRate (Count)"}],["AWS/ES","IndexingLatency","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"label":"IndexingLatency (Milliseconds)","yAxis":"right"}]],"yAxis":{"left":{"showUnits":false},"right":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":0,"y":46,"properties":{"view":"timeSeries","title":"ThreadpoolReadQueue (Node Average Count)","region":"${AWS::Region}","metrics":[["AWS/ES","ThreadpoolSearchQueue","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}"]],"annotations":{"horizontal":[{"value":1000,"yAxis":"left"}]},"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"metric","width":12,"height":4,"x":12,"y":46,"properties":{"view":"timeSeries","title":"ThreadpoolWriteQueue (Node Average Count)","region":"${AWS::Region}","metrics":[["AWS/ES","ThreadpoolWriteQueue","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}"]],"annotations":{"horizontal":[{"value":10000,"yAxis":"left"}]},"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"metric","width":12,"height":4,"x":0,"y":50,"properties":{"view":"timeSeries","title":"Threadpool Search Rejected Count (Node Total Count)","region":"${AWS::Region}","metrics":[["AWS/ES","ThreadpoolSearchRejected","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":50,"properties":{"view":"timeSeries","title":"Threadpool Indexing Rejected Count (Node Total Count)","region":"${AWS::Region}","metrics":[["AWS/ES","ThreadpoolWriteRejected","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"stat":"Sum"}],["AWS/ES","CoordinatingWriteRejected","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"stat":"Sum"}],["AWS/ES","PrimaryWriteRejected","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"stat":"Sum"}],["AWS/ES","ReplicaWriteRejected","ClientId","${AWS::AccountId}","DomainName","${DomainOrCollectionName}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"text","width":24,"height":1,"x":0,"y":54,"properties":{"markdown":"# SQS"}},{"type":"metric","width":12,"height":4,"x":0,"y":55,"properties":{"view":"timeSeries","title":"${AesSiemSqsSplitLogs0191F431.QueueName}: NumberOfMessagesReceived (Count)","region":"${AWS::Region}","metrics":[["AWS/SQS","NumberOfMessagesReceived","QueueName","${AesSiemSqsSplitLogs0191F431.QueueName}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"metric","width":12,"height":4,"x":12,"y":55,"properties":{"view":"timeSeries","title":"${AesSiemDlq1CD8439D.QueueName}: ApproximateNumberOfMessagesVisible (Count)","region":"${AWS::Region}","metrics":[["AWS/SQS","ApproximateNumberOfMessagesVisible","QueueName","${AesSiemDlq1CD8439D.QueueName}",{"stat":"Maximum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"text","width":24,"height":1,"x":0,"y":59,"properties":{"markdown":"# Lambda Function Logs: ${LambdaEsLoader4B1E2DD9}"}},{"type":"log","width":24,"height":6,"x":0,"y":60,"properties":{"view":"table","title":"CRITICAL Logs","region":"${AWS::Region}","query":"SOURCE ''/aws/lambda/${LambdaEsLoader4B1E2DD9}'' | fields @timestamp, message, s3_key\n | filter level == \"CRITICAL\"\n | sort @timestamp desc\n | limit 100"}},{"type":"log","width":24,"height":6,"x":0,"y":66,"properties":{"view":"table","title":"ERROR Logs","region":"${AWS::Region}","query":"SOURCE ''/aws/lambda/${LambdaEsLoader4B1E2DD9}'' | fields @timestamp, message, s3_key\n | filter level == \"ERROR\"\n | sort @timestamp desc\n | limit 100"}},{"type":"text","width":12,"height":3,"x":0,"y":72,"properties":{"markdown":"## Sample query\nTo investigate critical/error log with CloudWatch Logs Insights\n\n```\nfields @timestamp, @message\n| filter s3_key == \"copy s3_key and paste here\"\nOR @requestId == \"copy function_request_id and paste here\"```"}},{"type":"log","width":24,"height":6,"x":0,"y":75,"properties":{"view":"table","title":"Exception Logs","region":"${AWS::Region}","query":"SOURCE ''/aws/lambda/${LambdaEsLoader4B1E2DD9}'' | fields @timestamp, @message\n | filter @message =~ /^\\[ERROR]/\n | filter @message not like /No active exception to reraise/\n # exclude raise without Exception\n | sort @timestamp desc\n | limit 100"}}]}' DashboardName: SIEM Condition: IsManagedCluster SIEMDashboardServerlessC64B983C: Type: AWS::CloudWatch::Dashboard Properties: DashboardBody: !Sub - '{"widgets":[{"type":"text","width":24,"height":1,"x":0,"y":0,"properties":{"markdown":"# Lambda Function: ${LambdaEsLoader4B1E2DD9}"}},{"type":"metric","width":12,"height":4,"x":0,"y":1,"properties":{"view":"timeSeries","title":"Error count and success rate (%)","region":"${AWS::Region}","metrics":[["AWS/Lambda","Errors","FunctionName","${LambdaEsLoader4B1E2DD9}",{"color":"#d13212","label":"Errors (Count)","stat":"Sum","id":"errors"}],[{"label":"Success rate (%)","color":"#2ca02c","expression":"100 - 100 * errors / MAX([errors, invocations])","yAxis":"right"}],["AWS/Lambda","Invocations","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Sum","visible":false,"id":"invocations"}]],"yAxis":{"left":{"showUnits":false},"right":{"max":100,"showUnits":false}},"period":60}},{"type":"metric","width":12,"height":4,"x":12,"y":1,"properties":{"view":"timeSeries","title":"Invocations (Count)","region":"${AWS::Region}","metrics":[["AWS/Lambda","Invocations","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"},"period":60}},{"type":"metric","width":12,"height":4,"x":0,"y":5,"properties":{"view":"timeSeries","title":"Duration (Milliseconds)","region":"${AWS::Region}","metrics":[["AWS/Lambda","Duration","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Minimum"}],["AWS/Lambda","Duration","FunctionName","${LambdaEsLoader4B1E2DD9}"],["AWS/Lambda","Duration","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Maximum"}]],"yAxis":{"left":{"showUnits":false}},"period":60}},{"type":"metric","width":12,"height":4,"x":12,"y":5,"properties":{"view":"timeSeries","title":"Throttles (Count)","region":"${AWS::Region}","metrics":[["AWS/Lambda","Throttles","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"},"period":60}},{"type":"log","width":12,"height":4,"x":0,"y":9,"properties":{"view":"table","title":"Longest 5 invocations","region":"${AWS::Region}","query":"SOURCE ''/aws/lambda/${LambdaEsLoader4B1E2DD9}'' | fields @timestamp, @duration, @requestId\n | sort @duration desc\n | head 5"}},{"type":"metric","width":12,"height":4,"x":12,"y":9,"properties":{"view":"timeSeries","title":"ConcurrentExecutions (Count)","region":"${AWS::Region}","metrics":[["AWS/Lambda","ConcurrentExecutions","FunctionName","${LambdaEsLoader4B1E2DD9}",{"stat":"Maximum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"},"period":60}},{"type":"text","width":24,"height":1,"x":0,"y":13,"properties":{"markdown":"# OpenSearch Serverless: ${DomainOrCollectionName} collection"}},{"type":"metric","width":12,"height":4,"x":0,"y":14,"properties":{"view":"timeSeries","title":"HTTP requests by response code 2xs, 3xx","region":"${AWS::Region}","metrics":[["AWS/AOSS","2xx","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60,"stat":"Sum"}],["AWS/AOSS","3xx","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"text","width":12,"height":1,"x":12,"y":14,"properties":{"markdown":"# Write / Indexing"}},{"type":"metric","width":12,"height":4,"x":0,"y":18,"properties":{"view":"timeSeries","title":"HTTP requests by error response code 4xx, 5xx","region":"${AWS::Region}","metrics":[["AWS/AOSS","4xx","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60,"stat":"Sum"}],["AWS/AOSS","5xx","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":18,"properties":{"view":"timeSeries","title":"IngestionDataRate: The indexing rate per second to a collection (Bytes/s)","region":"${AWS::Region}","metrics":[["AWS/AOSS","IngestionDataRate","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"text","width":12,"height":4,"x":0,"y":22,"properties":{"markdown":""}},{"type":"metric","width":12,"height":4,"x":12,"y":22,"properties":{"view":"timeSeries","title":"IngestionDocumentRate: The rate per second at which documents are being ingested to a collection (Counts)","region":"${AWS::Region}","metrics":[["AWS/AOSS","IngestionDocumentRate","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"text","width":12,"height":1,"x":0,"y":26,"properties":{"markdown":"# Read / Search"}},{"type":"metric","width":12,"height":4,"x":12,"y":26,"properties":{"view":"timeSeries","title":"IngestionDocumentErrors: The total number of document errors during ingestion (counts)","region":"${AWS::Region}","metrics":[["AWS/AOSS","IngestionDocumentErrors","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"color":"#d13212","period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":0,"y":30,"properties":{"view":"timeSeries","title":"SearchOCU: The number of OCUs used to search collection data","region":"${AWS::Region}","metrics":[["AWS/AOSS","SearchOCU","ClientId","${AWS::AccountId}",{"period":3600,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":30,"properties":{"view":"timeSeries","title":"IndexingOCU: The number of OCUs used to ingest collection data","region":"${AWS::Region}","metrics":[["AWS/AOSS","IndexingOCU","ClientId","${AWS::AccountId}",{"period":3600,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":0,"y":34,"properties":{"view":"timeSeries","title":"SearchRequestRate: The total number of search requests (Counts/min)","region":"${AWS::Region}","metrics":[["AWS/AOSS","SearchRequestRate","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":34,"properties":{"view":"timeSeries","title":"IngestionRequestRate: The total number of bulk write operations (Counts/min)","region":"${AWS::Region}","metrics":[["AWS/AOSS","IngestionRequestRate","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":0,"y":38,"properties":{"view":"timeSeries","title":"SearchRequestLatency: The time to complete a search operation (milliseconds)","region":"${AWS::Region}","metrics":[["AWS/AOSS","SearchRequestLatency","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":38,"properties":{"view":"timeSeries","title":"IngestionRequestLatency: The time to complete bulk write operations (milliseconds)","region":"${AWS::Region}","metrics":[["AWS/AOSS","IngestionRequestLatency","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"period":60}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":0,"y":42,"properties":{"view":"timeSeries","title":"SearchRequestErrors: The total number of query errors (Counts/min)","region":"${AWS::Region}","metrics":[["AWS/AOSS","SearchRequestErrors","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"color":"#d13212","period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"metric","width":12,"height":4,"x":12,"y":42,"properties":{"view":"timeSeries","title":"IngestionRequestErrors: The total number of bulk indexing request errors (counts)","region":"${AWS::Region}","metrics":[["AWS/AOSS","IngestionRequestErrors","ClientId","${AWS::AccountId}","CollectionId","${Param1}","CollectionName","${DomainOrCollectionName}",{"color":"#d13212","period":60,"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}}}},{"type":"text","width":24,"height":1,"x":0,"y":46,"properties":{"markdown":"# SQS"}},{"type":"metric","width":12,"height":4,"x":0,"y":47,"properties":{"view":"timeSeries","title":"${AesSiemSqsSplitLogs0191F431.QueueName}: NumberOfMessagesReceived (Count)","region":"${AWS::Region}","metrics":[["AWS/SQS","NumberOfMessagesReceived","QueueName","${AesSiemSqsSplitLogs0191F431.QueueName}",{"stat":"Sum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"metric","width":12,"height":4,"x":12,"y":47,"properties":{"view":"timeSeries","title":"${AesSiemDlq1CD8439D.QueueName}: ApproximateNumberOfMessagesVisible (Count)","region":"${AWS::Region}","metrics":[["AWS/SQS","ApproximateNumberOfMessagesVisible","QueueName","${AesSiemDlq1CD8439D.QueueName}",{"stat":"Maximum"}]],"yAxis":{"left":{"showUnits":false}},"legend":{"position":"hidden"}}},{"type":"text","width":24,"height":1,"x":0,"y":51,"properties":{"markdown":"# Lambda Function Logs: ${LambdaEsLoader4B1E2DD9}"}},{"type":"log","width":24,"height":6,"x":0,"y":52,"properties":{"view":"table","title":"CRITICAL Logs","region":"${AWS::Region}","query":"SOURCE ''/aws/lambda/${LambdaEsLoader4B1E2DD9}'' | fields @timestamp, message, s3_key\n | filter level == \"CRITICAL\"\n | sort @timestamp desc\n | limit 100"}},{"type":"log","width":24,"height":6,"x":0,"y":58,"properties":{"view":"table","title":"ERROR Logs","region":"${AWS::Region}","query":"SOURCE ''/aws/lambda/${LambdaEsLoader4B1E2DD9}'' | fields @timestamp, message, s3_key\n | filter level == \"ERROR\"\n | sort @timestamp desc\n | limit 100"}},{"type":"text","width":12,"height":3,"x":0,"y":64,"properties":{"markdown":"## Sample query\nTo investigate critical/error log with CloudWatch Logs Insights\n\n```\nfields @timestamp, @message\n| filter s3_key == \"copy s3_key and paste here\"\nOR @requestId == \"copy function_request_id and paste here\"```"}},{"type":"log","width":24,"height":6,"x":0,"y":67,"properties":{"view":"table","title":"Exception Logs","region":"${AWS::Region}","query":"SOURCE ''/aws/lambda/${LambdaEsLoader4B1E2DD9}'' | fields @timestamp, @message\n | filter @message =~ /^\\[ERROR]/\n | filter @message not like /No active exception to reraise/\n # exclude raise without Exception\n | sort @timestamp desc\n | limit 100"}}]}' - Param1: !Select - 0 - !Split - . - !GetAtt 'AesSiemDomainDeployedR2.endpoint' DashboardName: SIEM-Serverless Condition: IsServerless Outputs: RoleDeploy: Value: !GetAtt 'AesSiemDeployRoleForLambda654D64F2.Arn' Export: Name: role-deploy DashboardsUrl: Value: !Sub 'https://${AesSiemDomainDeployedR2.endpoint}/_dashboards/' Export: Name: dashboards-url DashboardsPassword: Description: Please change the password in OpenSearch Dashboards ASAP Value: !GetAtt 'AesSiemDomainDeployedR2.kibanapass' Export: Name: dashboards-pass DashboardsAdminID: Value: !GetAtt 'AesSiemDomainDeployedR2.kibanaadmin' Export: Name: dashboards-admin Rules: {}