AWSTemplateFormatVersion: 2010-09-09 Description: (SO8019) - This template is used for setting up a ClickHouse cluster. (qs-1s4ldl6u7) Metadata: LICENSE: Apache License Version 2.0 AWS::CloudFormation::Interface: ParameterGroups: - Label: default: Network configuration Parameters: - PrivateSubnetID1 - PrivateSubnetID2 - Label: default: EC2 configuration Parameters: - KeyPairName - ClickHouseServerSecurityGroup - AdminServerSecurityGroup - Label: default: ClickHouse configuration Parameters: - ClickHouseNodeName1 - ClickHouseNodeName2 - ClickHouseNodeName3 - ClickHouseNodeName4 - ClickHouseNodeName5 - ClickHouseNodeName6 - ClickHouseNodeName7 - ClickHouseNodeName8 - ClickHouseInstanceType - ClickHouseVersion - ClickHouseTimezone - ClickHouseNodeCount - SecretId - ClickHouseBucketName - MoveFactor - RootStackName - DeviceName - VolumeSize - VolumeType - Iops - MaxThreads - MaxMemoryUsage - MaxInsertThreads - DistributedProductMode - LoadBalancing - MaxDataPartSize - DemoDataSize - ZookeeperPrivateIp1 - ZookeeperPrivateIp2 - ZookeeperPrivateIp3 - InstanceRoleArn - LatestAmiId - Label: default: ClickHouse operation configuration Parameters: - GrafanaVersion - CloudWatchAgentURL - Label: default: AWS Quick Start configuration Parameters: - QSS3BucketName - QSS3KeyPrefix ParameterLabels: PrivateSubnetID1: default: Private subnet ID 1 PrivateSubnetID2: default: Private subnet ID 2 KeyPairName: default: Key pair name ClickHouseServerSecurityGroup: default: ClickHouse server security group AdminServerSecurityGroup: default: ClickHouse admin server security group ClickHouseNodeName1: default: ClickHouse node name 1 ClickHouseNodeName2: default: ClickHouse node name 2 ClickHouseNodeName3: default: ClickHouse node name 3 ClickHouseNodeName4: default: ClickHouse node name 4 ClickHouseNodeName5: default: ClickHouse node name 5 ClickHouseNodeName6: default: ClickHouse node name 6 ClickHouseNodeName7: default: ClickHouse node name 7 ClickHouseNodeName8: default: ClickHouse node name 8 ClickHouseInstanceType: default: ClickHouse instance type ClickHouseVersion: default: ClickHouse version ClickHouseTimezone: default: ClickHouse time zone ClickHouseNodeCount: default: Number of ClickHouse nodes ClickHouseBucketName: default: ClickHouse bucket name MoveFactor: default: Move factor RootStackName: default: Prefix name of ClickHouse cluster DeviceName: default: Device name of ClickHouse nodes VolumeType: default: Volume type of ClickHouse nodes VolumeSize: default: Volume size of ClickHouse nodes Iops: default: IOPS of ClickHouse nodes DistributedProductMode: default: Distributed product mode LoadBalancing: default: Load balancing MaxDataPartSize: default: Max data part size MaxInsertThreads: default: Max insert threads MaxMemoryUsage: default: Max memory usage MaxThreads: default: Max threads DemoDataSize: default: Demo data size GrafanaVersion: default: Grafana version ZookeeperPrivateIp1: default: ZooKeeper private IP 1 ZookeeperPrivateIp2: default: ZooKeeper private IP 2 ZookeeperPrivateIp3: default: ZooKeeper private IP 3 CloudWatchAgentURL: default: CloudWatch agent URL InstanceRoleArn: default: Instance role arn for aws or aws-cn QSS3BucketName: default: AWS Quick Start S3 bucket name QSS3KeyPrefix: default: AWS Quick Start S3 key prefix SecretId: default: Secret id in SecretsManager LatestAmiId: default: Reference the latest Amazon Linux AMI in a CloudFormation template Parameters: PrivateSubnetID1: Description: Private Subnet 1 ID. Type: String PrivateSubnetID2: Description: Private Subnet 2 ID. Type: String ClickHouseServerSecurityGroup: Description: ID of the ClickHouse server access security group (e.g., sg-xxxxxxxx). Type: AWS::EC2::SecurityGroup::Id AdminServerSecurityGroup: Description: ID of the ClickHouse admin server access security group (e.g., sg-xxxxxxxx). Type: AWS::EC2::SecurityGroup::Id KeyPairName: Type: AWS::EC2::KeyPair::KeyName Description: Public/private key pairs allow you to securely connect to your instance after it launches. ClickHouseNodeName1: Default: ClickHouseNode1 Description: ClickHouse node name 1. Type: String ClickHouseNodeName2: Default: ClickHouseNode2 Description: ClickHouse node name 2. Type: String ClickHouseNodeName3: Default: ClickHouseNode3 Description: ClickHouse node name 3. Type: String ClickHouseNodeName4: Default: ClickHouseNode4 Description: ClickHouse node name 4. Type: String ClickHouseNodeName5: Default: ClickHouseNode5 Description: ClickHouse node name 5. Type: String ClickHouseNodeName6: Default: ClickHouseNode6 Description: ClickHouse node name 6. Type: String ClickHouseNodeName7: Default: ClickHouseNode7 Description: ClickHouse node name 7. Type: String ClickHouseNodeName8: Default: ClickHouseNode8 Description: ClickHouse node name 8. Type: String ClickHouseInstanceType: Description: Amazon EC2 instance type for the ClickHouse nodes. Type: String Default: i3.2xlarge AllowedValues: - t2.micro - t2.small - t2.medium - t2.large - m4.large - m4.xlarge - m4.2xlarge - m4.4xlarge - m4.10xlarge - m5.large - m5.xlarge - m5.2xlarge - m5.4xlarge - c4.large - c4.xlarge - c4.2xlarge - c4.4xlarge - c4.8xlarge - r5.large - r5.xlarge - r5.2xlarge - r5.4xlarge - r5.8xlarge - i3.large - i3.xlarge - i3.2xlarge - i3.4xlarge - i3.8xlarge - i3.16xlarge ClickHouseVersion: AllowedValues: # - '21.4.5.46-2' # - '21.5.5.12-2' # Default: '21.4.5.46-2' - '21.4.7.3-2' - '21.5.9.4-2' Default: '21.5.9.4-2' Description: ClickHouse version (21.4.5 or 21.5.5). Type: String ClickHouseTimezone: Default: Asia/Shanghai Description: ClickHouse time zone. Type: String ClickHouseNodeCount: Default: 2 Description: ClickHouse node count. Type: Number ClickHouseBucketName: Default: clickhouse-data Description: ClickHouse bucket name. Type: String MoveFactor: Default: 0.3 Description: Tiered storage parameter. Type: String RootStackName: Default: quickstart-clickhouse-cluster Description: Prefix name of the ClickHouse cluster. Type: String DeviceName: Description: The device name (for example, /dev/sdh or xvdh). Type: String Default: /dev/xvdh VolumeSize: Type: String Description: EBS volume size (data) to be attached to node in GBs. Default: 500 VolumeType: Type: String Description: EBS volume type (data) to be attached to node in GBs [gp2,gp3,st1], one volume for data storage is mounted automatically by CloudFormation stack. Default: gp2 AllowedValues: - gp2 - gp3 - st1 - io1 Iops: Type: String Description: IOPS of EBS volume when io1 type is chosen. Otherwise ignored. Default: 1000 MaxThreads: AllowedValues: - '2' - '4' - '8' - '16' - '32' Default: '8' Description: The max_threads for ClickHouse default users.xml //Recommend The number of cores using the instance. Type: String MaxMemoryUsage: AllowedValues: - '10737418240' - '21474836480' - '42949672960' - '107374182400' Default: '10737418240' Description: The max_memory_usage for ClickHouse default users.xml // Unit is byte, for example 21474836480=20g //In a single ClickHouse service process, the maximum amount of memory used by running a query is limited, and the default value is 10g. Type: String MaxInsertThreads: AllowedValues: - '1' - '4' - '8' - '16' Default: '4' Description: The max_insert_threads for ClickHouse default users.xml // The maximum number of threads to run the INSERT SELECT query. Type: String DistributedProductMode: AllowedValues: - 'deny' - 'local' - 'global' - 'allow' Default: 'deny' Description: The distributed_product_mode for ClickHouse default users.xml // ClickHouse applies this setting when the query contains the product of distributed tables, i.e. when the query for a distributed table contains a non-GLOBAL subquery for the distributed table. Type: String LoadBalancing: AllowedValues: - 'random' - 'nearest_hostname' - 'in_order' - 'first_or_random' Default: 'random' Description: The load_balancing for ClickHouse default users.xml // Specifies the algorithm of replicas selection that is used for distributed query processing. Type: String MaxDataPartSize: AllowedValues: - '1073741824' - '5368709120' - '10737418240' Default: '1073741824' Description: The max_data_part_size_bytes for ClickHouse default storage.xml // Unit is byte, the maximum size of a part that can be stored on any of the volume’s disks. Type: String GrafanaVersion: Default: 8.0.1-1 Description: Grafana version. Type: String DemoDataSize: AllowedValues: - 'none' - 'small' - 'medium' Default: 'small' Description: AWS CloudFormation creates a demo table(OnTime dataset) for you and load data. You can choose the size of demo data --'small'100m / / -'medium'3g / / -'large'15g. Type: String ZookeeperPrivateIp1: Description: ZooKeeper private IP 1. Type: String ZookeeperPrivateIp2: Description: ZooKeeper private IP 2. Type: String ZookeeperPrivateIp3: Description: ZooKeeper private IP 3. Type: String CloudWatchAgentURL: Description: For each download link, you can download and install the CloudWatch agent using the command line. Type: String InstanceRoleArn: Description: For aws or aws-cn. Type: String Default: aws QSS3BucketName: AllowedPattern: ^[0-9a-zA-Z]+([0-9a-zA-Z-]*[0-9a-zA-Z])*$ ConstraintDescription: Quick Start bucket name can include numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen (-). Default: aws-quickstart Description: S3 bucket name for the Quick Start assets. This string can include numbers, lowercase letters, uppercase letters, and hyphens (-). It cannot start or end with a hyphen (-). Type: String QSS3KeyPrefix: AllowedPattern: ^[0-9a-zA-Z-/.]*$ ConstraintDescription: Quick Start key prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slashes (/). Default: quickstart-clickhouse-cluster/ Description: S3 key prefix for the Quick Start assets. Quick Start key prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slashes (/). Type: String SecretId: Type: String Description: Dynamically generate a random text string stored in SecretsManager to use as a password. LatestAmiId: Description: Reference the latest Amazon Linux AMI in a CloudFormation template. Type: AWS::SSM::Parameter::Value Default: /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2 Conditions: UseIops: Fn::Equals: [!Ref VolumeType,"io1"] 2NodesCondition: !Or - !Equals [!Ref 'ClickHouseNodeCount', '2'] - !Equals [!Ref 'ClickHouseNodeCount', '4'] - !Equals [!Ref 'ClickHouseNodeCount', '6'] - !Equals [!Ref 'ClickHouseNodeCount', '8'] 4NodesCondition: !Or - !Equals [!Ref 'ClickHouseNodeCount', '4'] - !Equals [!Ref 'ClickHouseNodeCount', '6'] - !Equals [!Ref 'ClickHouseNodeCount', '8'] 6NodesCondition: !Or - !Equals [!Ref 'ClickHouseNodeCount', '6'] - !Equals [!Ref 'ClickHouseNodeCount', '8'] 8NodesCondition: !Equals [!Ref 'ClickHouseNodeCount', '8'] Resources: ClickHouseMainLogGroup: Type: AWS::Logs::LogGroup Properties: LogGroupName: "ClickHouseLogGroup" RetentionInDays: 180 ClickHouseInstanceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - !Sub ec2.${AWS::URLSuffix} Action: - sts:AssumeRole Path: / Policies: - PolicyDocument: Statement: - Effect: Allow Action: - logs:CreateLogGroup - logs:CreateLogStream - logs:PutLogEvents - logs:DescribeLogStreams Resource: !Sub arn:${InstanceRoleArn}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${ClickHouseMainLogGroup}:* - Effect: Allow Action: - s3:PutObjectAcl - s3:GetObjectAcl - s3:PutObject - s3:GetObject - s3:DeleteObject - s3:RestoreObject - s3:GetObjectVersion - s3:DeleteObjectVersion - s3:GetObjectRetention - s3:PutObjectRetention - s3:GetObjectVersionAcl - s3:PutObjectVersionAcl - s3:CreateBucket - s3:ListAllMyBuckets - s3:ListBucket - s3:PutBucketVersioning - s3:ListBucketVersions - s3:PutBucketLogging - s3:PutBucketAcl - s3:GetBucketAcl - s3:PutBucketPolicy - s3:GetBucketPolicy - s3:GetBucketLocation - s3:PutEncryptionConfiguration - s3:GetEncryptionConfiguration - s3:GetBucketVersioning - s3:GetLifecycleConfiguration - s3:GetIntelligentTieringConfiguration - s3:PutIntelligentTieringConfiguration - s3:PutLifecycleConfiguration Resource: !Sub arn:${InstanceRoleArn}:s3:::*/* - Effect: Allow Action: - ec2:CreateTags - ec2:DeleteTags Resource: !Sub arn:${InstanceRoleArn}:ec2:*:${AWS::AccountId}:instance/* - Effect: Allow Action: - ec2:DescribeTags Resource: '*' - Effect: Allow Action: - cloudwatch:PutDashboard - cloudwatch:DeleteAlarms - cloudwatch:DeleteInsightRules - cloudwatch:StartMetricStreams - cloudwatch:UntagResource - cloudwatch:ListTagsForResource - cloudwatch:GetDashboard - cloudwatch:GetInsightRuleReport - cloudwatch:DisableInsightRules - cloudwatch:DescribeAlarms - cloudwatch:GetMetricStream - cloudwatch:DeleteDashboards - cloudwatch:DescribeAlarmHistory - cloudwatch:StopMetricStreams - cloudwatch:DisableAlarmActions - cloudwatch:DeleteMetricStream - cloudwatch:SetAlarmState - cloudwatch:EnableInsightRules - cloudwatch:PutCompositeAlarm - cloudwatch:PutMetricStream - cloudwatch:PutInsightRule - cloudwatch:TagResource - cloudwatch:PutMetricAlarm - cloudwatch:EnableAlarmActions Resource: - !Sub arn:${InstanceRoleArn}:cloudwatch::${AWS::AccountId}:dashboard/* - !Sub arn:${InstanceRoleArn}:cloudwatch:*:${AWS::AccountId}:alarm:* - !Sub arn:${InstanceRoleArn}:cloudwatch:*:${AWS::AccountId}:insight-rule/* - !Sub arn:${InstanceRoleArn}:cloudwatch:*:${AWS::AccountId}:metric-stream/* - Effect: Allow Action: - cloudwatch:DescribeInsightRules - cloudwatch:PutMetricData - cloudwatch:GetMetricData - cloudwatch:ListMetricStreams - cloudwatch:DescribeAlarmsForMetric - cloudwatch:ListDashboards - cloudwatch:PutAnomalyDetector - cloudwatch:GetMetricStatistics - cloudwatch:GetMetricWidgetImage - cloudwatch:DeleteAnomalyDetector - cloudwatch:ListMetrics - cloudwatch:DescribeAnomalyDetectors Resource: '*' - Effect: Allow Action: - secretsmanager:CreateSecret - secretsmanager:GetSecretValue Resource: !Sub arn:${InstanceRoleArn}:secretsmanager:*:${AWS::AccountId}:secret:* PolicyName: ClickHouse-Instance-Role ClickHouseInstanceProfile: Type: AWS::IAM::InstanceProfile Properties: Path: / Roles: - !Ref ClickHouseInstanceRole AdminAutoScalingGroup: Condition: 2NodesCondition Type: 'AWS::AutoScaling::AutoScalingGroup' Properties: LaunchConfigurationName: !Ref AdminLaunchConfiguration VPCZoneIdentifier: - !Ref PrivateSubnetID1 - !Ref PrivateSubnetID2 MinSize: '1' MaxSize: '1' Cooldown: '900' DesiredCapacity: '1' Tags: - Key: Name Value: 'ClickHouseAdminClient' PropagateAtLaunch: true CreationPolicy: ResourceSignal: Count: 1 Timeout: PT120M AutoScalingCreationPolicy: MinSuccessfulInstancesPercent: 100 UpdatePolicy: AutoScalingReplacingUpdate: WillReplace: true AdminLaunchConfiguration: Type: 'AWS::AutoScaling::LaunchConfiguration' Condition: 2NodesCondition DependsOn: ClickHouseInstance1 Metadata: Comment: Install ClickHouse admin AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "ClickHouseClient", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.AdminLaunchConfiguration.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource AdminLaunchConfiguration --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile # ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SecurityGroups: - !Ref ClickHouseServerSecurityGroup - !Ref AdminServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack '${AWS::StackName}' --resource AdminLaunchConfiguration --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/downloaddata.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-client-install.sh ./ --region ${AWS::Region} function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> clickhouse-client-install.log rm -rf secretfile rm -rf find-secret.py return 0 else echo "Fail to create demo table" >> clickhouse-client-install.log return 1 fi } chmod +x clickhouse-client-install.sh ./clickhouse-client-install.sh ${ClickHouseVersion} `python ./find-secret.py secretfile ` ${RootStackName} ${AWS::Region} ${DemoDataSize} ${ClickHouseNodeCount} ${GrafanaVersion} > clickhouse-client-install.log cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource AdminAutoScalingGroup --region ${AWS::Region} CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT120M" ClickHouseInstance1: Condition: 2NodesCondition Type: AWS::EC2::Instance Metadata: Comment: Install ClickHouse replica 1 AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "${ClickHouseNodeName1}", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.ClickHouseInstance1.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource ClickHouseInstance1 --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile # ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SubnetId: !Ref PrivateSubnetID1 SecurityGroupIds: - !Ref ClickHouseServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ClickHouseInstance1 --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-shard01-replica01,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-cluster,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-install.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/create-table-demo.sql ./ --region ${AWS::Region} chmod +x clickhouse-install.sh ./clickhouse-install.sh ${ClickHouseVersion} ${ClickHouseNodeCount} ${RootStackName} ${AWS::Region} http://s3.${AWS::Region}.${AWS::URLSuffix}/${ClickHouseBucketName}/quickstart-clickhouse-data/ ${MoveFactor} ${ClickHouseTimezone} ${ZookeeperPrivateIp1} ${ZookeeperPrivateIp2} ${ZookeeperPrivateIp3} `python ./find-secret.py secretfile ` ${MaxThreads} ${MaxInsertThreads} ${DistributedProductMode} ${MaxMemoryUsage} ${LoadBalancing} ${MaxDataPartSize} "01" "01" > ch-install.log chmod +r /home/ec2-user/create-table-demo.sql flag=600 while((flag > 0)) do clickhouse-client --password `python ./find-secret.py secretfile ` --query "SELECT * FROM system.clusters;" > /dev/null if [ $? -eq 0 ] then echo "ClickHouse is running" >> ch-install.log break fi let flag-- sleep 1 done # Clean rm -rf /home/ec2-user/find-clickhouse-node.py rm -rf /home/ec2-user/clickhouse-install.sh rm -rf /home/ec2-user/instancelist-1 rm -rf /home/ec2-user/result-1 rm -rf /home/ec2-user/instancelist-2 rm -rf /home/ec2-user/result-2 rm -rf /home/ec2-user/instancelist-3 rm -rf /home/ec2-user/result-3 rm -rf /home/ec2-user/instancelist-4 rm -rf /home/ec2-user/result-4 rm -rf /home/ec2-user/instancelist-5 rm -rf /home/ec2-user/result-5 rm -rf /home/ec2-user/instancelist-6 rm -rf /home/ec2-user/result-6 rm -rf /home/ec2-user/instancelist-7 rm -rf /home/ec2-user/result-7 rm -rf /home/ec2-user/instancelist-8 rm -rf /home/ec2-user/result-8 function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> ch-install.log rm -rf secretfile rm -rf find-secret.py rm -rf create-table-demo.sql return 0 else echo "Fail to create demo table" >> ch-install.log return 1 fi } echo "Begin to create demo table" >> ch-install.log clickhouse-client --user default --password `python ./find-secret.py secretfile ` -d default --multiquery < create-table-demo.sql cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource ClickHouseInstance1 --region ${AWS::Region} Tags: - Key: Name Value: !Ref ClickHouseNodeName1 CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT60M" ClickHouseInstance2: Condition: 2NodesCondition Type: AWS::EC2::Instance Metadata: Comment: Install ClickHouse instance replica 2 AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "${ClickHouseNodeName2}", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.ClickHouseInstance2.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource ClickHouseInstance2 --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile #ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SubnetId: !Ref PrivateSubnetID2 SecurityGroupIds: - !Ref ClickHouseServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ClickHouseInstance2 --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-shard01-replica02,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-cluster,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-install.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/create-table-demo.sql ./ --region ${AWS::Region} chmod +x clickhouse-install.sh ./clickhouse-install.sh ${ClickHouseVersion} ${ClickHouseNodeCount} ${RootStackName} ${AWS::Region} http://s3.${AWS::Region}.${AWS::URLSuffix}/${ClickHouseBucketName}/quickstart-clickhouse-data/ ${MoveFactor} ${ClickHouseTimezone} ${ZookeeperPrivateIp1} ${ZookeeperPrivateIp2} ${ZookeeperPrivateIp3} `python ./find-secret.py secretfile ` ${MaxThreads} ${MaxInsertThreads} ${DistributedProductMode} ${MaxMemoryUsage} ${LoadBalancing} ${MaxDataPartSize} "01" "02" > ch-install.log chmod +r /home/ec2-user/create-table-demo.sql flag=600 while((flag > 0)) do clickhouse-client --password `python ./find-secret.py secretfile ` --query "SELECT * FROM system.clusters;" > /dev/null if [ $? -eq 0 ] then echo "ClickHouse is running" >> ch-install.log break fi let flag-- sleep 1 done # Clean rm -rf /home/ec2-user/find-clickhouse-node.py rm -rf /home/ec2-user/clickhouse-install.sh rm -rf /home/ec2-user/instancelist-1 rm -rf /home/ec2-user/result-1 rm -rf /home/ec2-user/instancelist-2 rm -rf /home/ec2-user/result-2 rm -rf /home/ec2-user/instancelist-3 rm -rf /home/ec2-user/result-3 rm -rf /home/ec2-user/instancelist-4 rm -rf /home/ec2-user/result-4 rm -rf /home/ec2-user/instancelist-5 rm -rf /home/ec2-user/result-5 rm -rf /home/ec2-user/instancelist-6 rm -rf /home/ec2-user/result-6 rm -rf /home/ec2-user/instancelist-7 rm -rf /home/ec2-user/result-7 rm -rf /home/ec2-user/instancelist-8 rm -rf /home/ec2-user/result-8 function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> ch-install.log rm -rf secretfile rm -rf find-secret.py rm -rf create-table-demo.sql return 0 else echo "Fail to create demo table" >> ch-install.log return 1 fi } echo "Begin to create demo table" >> ch-install.log clickhouse-client --user default --password `python ./find-secret.py secretfile ` -d default --multiquery < create-table-demo.sql cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource ClickHouseInstance2 --region ${AWS::Region} Tags: - Key: Name Value: !Ref ClickHouseNodeName2 CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT60M" ClickHouseInstance3: Condition: 4NodesCondition Type: AWS::EC2::Instance Metadata: Comment: Install ClickHouse replica 1 AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "${ClickHouseNodeName3}", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.ClickHouseInstance3.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource ClickHouseInstance3 --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile #ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SubnetId: !Ref PrivateSubnetID1 SecurityGroupIds: - !Ref ClickHouseServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ClickHouseInstance3 --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-shard02-replica01,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-cluster,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-install.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/create-table-demo.sql ./ --region ${AWS::Region} chmod +x clickhouse-install.sh ./clickhouse-install.sh ${ClickHouseVersion} ${ClickHouseNodeCount} ${RootStackName} ${AWS::Region} http://s3.${AWS::Region}.${AWS::URLSuffix}/${ClickHouseBucketName}/quickstart-clickhouse-data/ ${MoveFactor} ${ClickHouseTimezone} ${ZookeeperPrivateIp1} ${ZookeeperPrivateIp2} ${ZookeeperPrivateIp3} `python ./find-secret.py secretfile ` ${MaxThreads} ${MaxInsertThreads} ${DistributedProductMode} ${MaxMemoryUsage} ${LoadBalancing} ${MaxDataPartSize} "02" "01" > ch-install.log chmod +r /home/ec2-user/create-table-demo.sql flag=600 while((flag > 0)) do clickhouse-client --password `python ./find-secret.py secretfile ` --query "SELECT * FROM system.clusters;" > /dev/null if [ $? -eq 0 ] then echo "ClickHouse is running" >> ch-install.log break fi let flag-- sleep 1 done # Clean rm -rf /home/ec2-user/find-clickhouse-node.py rm -rf /home/ec2-user/clickhouse-install.sh rm -rf /home/ec2-user/instancelist-1 rm -rf /home/ec2-user/result-1 rm -rf /home/ec2-user/instancelist-2 rm -rf /home/ec2-user/result-2 rm -rf /home/ec2-user/instancelist-3 rm -rf /home/ec2-user/result-3 rm -rf /home/ec2-user/instancelist-4 rm -rf /home/ec2-user/result-4 rm -rf /home/ec2-user/instancelist-5 rm -rf /home/ec2-user/result-5 rm -rf /home/ec2-user/instancelist-6 rm -rf /home/ec2-user/result-6 rm -rf /home/ec2-user/instancelist-7 rm -rf /home/ec2-user/result-7 rm -rf /home/ec2-user/instancelist-8 rm -rf /home/ec2-user/result-8 function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> ch-install.log rm -rf secretfile rm -rf find-secret.py rm -rf create-table-demo.sql return 0 else echo "Fail to create demo table" >> ch-install.log return 1 fi } echo "Begin to create demo table" >> ch-install.log clickhouse-client --user default --password `python ./find-secret.py secretfile ` -d default --multiquery < create-table-demo.sql cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource ClickHouseInstance3 --region ${AWS::Region} Tags: - Key: Name Value: !Ref ClickHouseNodeName3 CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT60M" ClickHouseInstance4: Condition: 4NodesCondition Type: AWS::EC2::Instance Metadata: Comment: Install ClickHouse instance replica 2 AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "${ClickHouseNodeName4}", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.ClickHouseInstance4.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource ClickHouseInstance4 --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile #ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SubnetId: !Ref PrivateSubnetID2 SecurityGroupIds: - !Ref ClickHouseServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ClickHouseInstance4 --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-shard02-replica02,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-cluster,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-install.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/create-table-demo.sql ./ --region ${AWS::Region} chmod +x clickhouse-install.sh ./clickhouse-install.sh ${ClickHouseVersion} ${ClickHouseNodeCount} ${RootStackName} ${AWS::Region} http://s3.${AWS::Region}.${AWS::URLSuffix}/${ClickHouseBucketName}/quickstart-clickhouse-data/ ${MoveFactor} ${ClickHouseTimezone} ${ZookeeperPrivateIp1} ${ZookeeperPrivateIp2} ${ZookeeperPrivateIp3} `python ./find-secret.py secretfile ` ${MaxThreads} ${MaxInsertThreads} ${DistributedProductMode} ${MaxMemoryUsage} ${LoadBalancing} ${MaxDataPartSize} "02" "02" > ch-install.log chmod +r /home/ec2-user/create-table-demo.sql flag=600 while((flag > 0)) do clickhouse-client --password `python ./find-secret.py secretfile ` --query "SELECT * FROM system.clusters;" > /dev/null if [ $? -eq 0 ] then echo "ClickHouse is running" >> ch-install.log break fi let flag-- sleep 1 done # Clean rm -rf /home/ec2-user/find-clickhouse-node.py rm -rf /home/ec2-user/clickhouse-install.sh rm -rf /home/ec2-user/instancelist-1 rm -rf /home/ec2-user/result-1 rm -rf /home/ec2-user/instancelist-2 rm -rf /home/ec2-user/result-2 rm -rf /home/ec2-user/instancelist-3 rm -rf /home/ec2-user/result-3 rm -rf /home/ec2-user/instancelist-4 rm -rf /home/ec2-user/result-4 rm -rf /home/ec2-user/instancelist-5 rm -rf /home/ec2-user/result-5 rm -rf /home/ec2-user/instancelist-6 rm -rf /home/ec2-user/result-6 rm -rf /home/ec2-user/instancelist-7 rm -rf /home/ec2-user/result-7 rm -rf /home/ec2-user/instancelist-8 rm -rf /home/ec2-user/result-8 function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> ch-install.log rm -rf secretfile rm -rf find-secret.py rm -rf create-table-demo.sql return 0 else echo "Fail to create demo table" >> ch-install.log return 1 fi } echo "Begin to create demo table" >> ch-install.log clickhouse-client --user default --password `python ./find-secret.py secretfile ` -d default --multiquery < create-table-demo.sql cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource ClickHouseInstance4 --region ${AWS::Region} Tags: - Key: Name Value: !Ref ClickHouseNodeName4 CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT60M" ClickHouseInstance5: Condition: 6NodesCondition Type: AWS::EC2::Instance Metadata: Comment: Install ClickHouse replica 1 AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "${ClickHouseNodeName5}", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.ClickHouseInstance5.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource ClickHouseInstance5 --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile #ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SubnetId: !Ref PrivateSubnetID1 SecurityGroupIds: - !Ref ClickHouseServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ClickHouseInstance5 --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-shard03-replica01,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-cluster,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-install.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/create-table-demo.sql ./ --region ${AWS::Region} chmod +x clickhouse-install.sh ./clickhouse-install.sh ${ClickHouseVersion} ${ClickHouseNodeCount} ${RootStackName} ${AWS::Region} http://s3.${AWS::Region}.${AWS::URLSuffix}/${ClickHouseBucketName}/quickstart-clickhouse-data/ ${MoveFactor} ${ClickHouseTimezone} ${ZookeeperPrivateIp1} ${ZookeeperPrivateIp2} ${ZookeeperPrivateIp3} `python ./find-secret.py secretfile ` ${MaxThreads} ${MaxInsertThreads} ${DistributedProductMode} ${MaxMemoryUsage} ${LoadBalancing} ${MaxDataPartSize} "03" "01" > ch-install.log chmod +r /home/ec2-user/create-table-demo.sql flag=600 while((flag > 0)) do clickhouse-client --password `python ./find-secret.py secretfile ` --query "SELECT * FROM system.clusters;" > /dev/null if [ $? -eq 0 ] then echo "ClickHouse is running" >> ch-install.log break fi let flag-- sleep 1 done # Clean rm -rf /home/ec2-user/find-clickhouse-node.py rm -rf /home/ec2-user/clickhouse-install.sh rm -rf /home/ec2-user/instancelist-1 rm -rf /home/ec2-user/result-1 rm -rf /home/ec2-user/instancelist-2 rm -rf /home/ec2-user/result-2 rm -rf /home/ec2-user/instancelist-3 rm -rf /home/ec2-user/result-3 rm -rf /home/ec2-user/instancelist-4 rm -rf /home/ec2-user/result-4 rm -rf /home/ec2-user/instancelist-5 rm -rf /home/ec2-user/result-5 rm -rf /home/ec2-user/instancelist-6 rm -rf /home/ec2-user/result-6 rm -rf /home/ec2-user/instancelist-7 rm -rf /home/ec2-user/result-7 rm -rf /home/ec2-user/instancelist-8 rm -rf /home/ec2-user/result-8 function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> ch-install.log rm -rf secretfile rm -rf find-secret.py rm -rf create-table-demo.sql return 0 else echo "Fail to create demo table" >> ch-install.log return 1 fi } echo "Begin to create demo table" >> ch-install.log clickhouse-client --user default --password `python ./find-secret.py secretfile ` -d default --multiquery < create-table-demo.sql cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource ClickHouseInstance5 --region ${AWS::Region} Tags: - Key: Name Value: !Ref ClickHouseNodeName5 CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT60M" ClickHouseInstance6: Condition: 6NodesCondition Type: AWS::EC2::Instance Metadata: Comment: Install ClickHouse instance replica 2 AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "${ClickHouseNodeName6}", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.ClickHouseInstance6.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource ClickHouseInstance6 --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile #ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SubnetId: !Ref PrivateSubnetID2 SecurityGroupIds: - !Ref ClickHouseServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ClickHouseInstance6 --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-shard03-replica02,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-cluster,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-install.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/create-table-demo.sql ./ --region ${AWS::Region} chmod +x clickhouse-install.sh ./clickhouse-install.sh ${ClickHouseVersion} ${ClickHouseNodeCount} ${RootStackName} ${AWS::Region} http://s3.${AWS::Region}.${AWS::URLSuffix}/${ClickHouseBucketName}/quickstart-clickhouse-data/ ${MoveFactor} ${ClickHouseTimezone} ${ZookeeperPrivateIp1} ${ZookeeperPrivateIp2} ${ZookeeperPrivateIp3} `python ./find-secret.py secretfile ` ${MaxThreads} ${MaxInsertThreads} ${DistributedProductMode} ${MaxMemoryUsage} ${LoadBalancing} ${MaxDataPartSize} "03" "02" > ch-install.log chmod +r /home/ec2-user/create-table-demo.sql flag=600 while((flag > 0)) do clickhouse-client --password `python ./find-secret.py secretfile ` --query "SELECT * FROM system.clusters;" > /dev/null if [ $? -eq 0 ] then echo "ClickHouse is running" >> ch-install.log break fi let flag-- sleep 1 done # Clean rm -rf /home/ec2-user/find-clickhouse-node.py rm -rf /home/ec2-user/clickhouse-install.sh rm -rf /home/ec2-user/instancelist-1 rm -rf /home/ec2-user/result-1 rm -rf /home/ec2-user/instancelist-2 rm -rf /home/ec2-user/result-2 rm -rf /home/ec2-user/instancelist-3 rm -rf /home/ec2-user/result-3 rm -rf /home/ec2-user/instancelist-4 rm -rf /home/ec2-user/result-4 rm -rf /home/ec2-user/instancelist-5 rm -rf /home/ec2-user/result-5 rm -rf /home/ec2-user/instancelist-6 rm -rf /home/ec2-user/result-6 rm -rf /home/ec2-user/instancelist-7 rm -rf /home/ec2-user/result-7 rm -rf /home/ec2-user/instancelist-8 rm -rf /home/ec2-user/result-8 function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> ch-install.log rm -rf secretfile rm -rf find-secret.py rm -rf create-table-demo.sql return 0 else echo "Fail to create demo table" >> ch-install.log return 1 fi } echo "Begin to create demo table" >> ch-install.log clickhouse-client --user default --password `python ./find-secret.py secretfile ` -d default --multiquery < create-table-demo.sql cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource ClickHouseInstance6 --region ${AWS::Region} Tags: - Key: Name Value: !Ref ClickHouseNodeName6 CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT60M" ClickHouseInstance7: Condition: 8NodesCondition Type: AWS::EC2::Instance Metadata: Comment: Install ClickHouse instance replica 1 AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "${ClickHouseNodeName7}", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.ClickHouseInstance7.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource ClickHouseInstance7 --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile #ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SubnetId: !Ref PrivateSubnetID1 SecurityGroupIds: - !Ref ClickHouseServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ClickHouseInstance7 --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-shard04-replica01,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-cluster,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-install.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/create-table-demo.sql ./ --region ${AWS::Region} chmod +x clickhouse-install.sh ./clickhouse-install.sh ${ClickHouseVersion} ${ClickHouseNodeCount} ${RootStackName} ${AWS::Region} http://s3.${AWS::Region}.${AWS::URLSuffix}/${ClickHouseBucketName}/quickstart-clickhouse-data/ ${MoveFactor} ${ClickHouseTimezone} ${ZookeeperPrivateIp1} ${ZookeeperPrivateIp2} ${ZookeeperPrivateIp3} `python ./find-secret.py secretfile ` ${MaxThreads} ${MaxInsertThreads} ${DistributedProductMode} ${MaxMemoryUsage} ${LoadBalancing} ${MaxDataPartSize} "04" "01" > ch-install.log chmod +r /home/ec2-user/create-table-demo.sql flag=600 while((flag > 0)) do clickhouse-client --password `python ./find-secret.py secretfile ` --query "SELECT * FROM system.clusters;" > /dev/null if [ $? -eq 0 ] then echo "ClickHouse is running" >> ch-install.log break fi let flag-- sleep 1 done # Clean rm -rf /home/ec2-user/find-clickhouse-node.py rm -rf /home/ec2-user/clickhouse-install.sh rm -rf /home/ec2-user/instancelist-1 rm -rf /home/ec2-user/result-1 rm -rf /home/ec2-user/instancelist-2 rm -rf /home/ec2-user/result-2 rm -rf /home/ec2-user/instancelist-3 rm -rf /home/ec2-user/result-3 rm -rf /home/ec2-user/instancelist-4 rm -rf /home/ec2-user/result-4 rm -rf /home/ec2-user/instancelist-5 rm -rf /home/ec2-user/result-5 rm -rf /home/ec2-user/instancelist-6 rm -rf /home/ec2-user/result-6 rm -rf /home/ec2-user/instancelist-7 rm -rf /home/ec2-user/result-7 rm -rf /home/ec2-user/instancelist-8 rm -rf /home/ec2-user/result-8 function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> ch-install.log rm -rf secretfile rm -rf find-secret.py rm -rf create-table-demo.sql return 0 else echo "Fail to create demo table" >> ch-install.log return 1 fi } echo "Begin to create demo table" >> ch-install.log clickhouse-client --user default --password `python ./find-secret.py secretfile ` -d default --multiquery < create-table-demo.sql cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource ClickHouseInstance7 --region ${AWS::Region} Tags: - Key: Name Value: !Ref ClickHouseNodeName7 CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT60M" ClickHouseInstance8: Condition: 8NodesCondition Type: AWS::EC2::Instance Metadata: Comment: Install ClickHouse instance replica 2 AWS::CloudFormation::Authentication: S3AccessCreds: type: S3 roleName: !Ref ClickHouseInstanceRole AWS::CloudFormation::Init: configSets: default: - 01_setupCfnHup - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent UpdateEnvironment: - 02_config-amazon-cloudwatch-agent - 03_restart_amazon-cloudwatch-agent # Definition of json configuration of AmazonCloudWatchAgent, you can change the configuration below. 02_config-amazon-cloudwatch-agent: files: '/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json': content: !Sub | { "agent": { "metrics_collection_interval": 1, "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log" }, "metrics": { "append_dimensions": { "AutoScalingGroupName": "${!aws:AutoScalingGroupName}", "ImageId": "${!aws:ImageId}", "InstanceId": "${!aws:InstanceId}", "InstanceType": "${!aws:InstanceType}" }, "namespace": "ClickHouseNamespace", "metrics_collected": { "mem": { "measurement": [ "mem_used_percent" ] }, "cpu": { "resources": [ "*" ], "measurement": [ {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"}, {"name": "cpu_usage_nice", "unit": "Percent"}, {"name": "cpu_usage_user", "unit": "Percent"}, {"name": "cpu_usage_system", "unit": "Percent"} ], "totalcpu": true, "metrics_collection_interval": 10 }, "disk": { "resources": [ "/" ], "measurement": [ "used_percent", "total", "used" ], "ignore_file_system_types": [ "sysfs", "devtmpfs" ], "metrics_collection_interval": 60 }, "diskio": { "resources": [ "*" ], "measurement": [ "reads", "writes", "read_time", "write_time", "io_time" ], "metrics_collection_interval": 60 }, "swap": { "measurement": [ "swap_used_percent" ] }, "net": { "resources": [ "eth0" ], "measurement": [ "bytes_sent", "bytes_recv", "drop_in", "drop_out" ] } } }, "logs": { "force_flush_interval": 5, "logs_collected": { "files": { "collect_list": [ { "file_path": "/var/log/clickhouse-server/clickhouse-server.log", "log_group_name": "${ClickHouseMainLogGroup}", "log_stream_name": "${ClickHouseNodeName8}", "timestamp_format": "%Y-%m-%d %H:%M:%S", "timezone": "UTC" } ] } } } } # Invoke amazon-cloudwatch-agent-ctl to restart the AmazonCloudWatchAgent. 03_restart_amazon-cloudwatch-agent: commands: 01_stop_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a stop 02_start_service: command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json -s # Cfn-hup setting, it is to monitor the change of metadata. # When there is change in the contents of json file in the metadata section, cfn-hup will call cfn-init to restart the AmazonCloudWatchAgent. 01_setupCfnHup: files: '/etc/cfn/cfn-hup.conf': content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} interval=1 mode: '000400' owner: root group: root '/etc/cfn/hooks.d/amazon-cloudwatch-agent-auto-reloader.conf': content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.ClickHouseInstance8.Metadata.AWS::CloudFormation::Init.02_config-amazon-cloudwatch-agent action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackId} --resource ClickHouseInstance8 --region ${AWS::Region} --configsets UpdateEnvironment runas=root mode: '000400' owner: root group: root "/lib/systemd/system/cfn-hup.service": content: | [Unit] Description=cfn-hup daemon [Service] Type=simple ExecStart=/opt/aws/bin/cfn-hup Restart=always [Install] WantedBy=multi-user.target commands: 01enable_cfn_hup: command: | systemctl enable cfn-hup.service 02start_cfn_hup: command: | systemctl start cfn-hup.service Properties: KeyName: !Ref KeyPairName IamInstanceProfile: !Ref ClickHouseInstanceProfile #ImageId: !FindInMap [AWSRegion2AMI, !Ref 'AWS::Region', HVM64] ImageId: !Ref LatestAmiId InstanceType: !Ref ClickHouseInstanceType SubnetId: !Ref PrivateSubnetID2 SecurityGroupIds: - !Ref ClickHouseServerSecurityGroup BlockDeviceMappings: - DeviceName: !Ref DeviceName Ebs: VolumeSize: !Ref VolumeSize VolumeType: !Ref VolumeType Iops: !If [UseIops,!Ref Iops,!Ref AWS::NoValue] DeleteOnTermination: true UserData: !Base64 Fn::Sub: | #!/bin/bash yum update -y # This script below is to install AmazonCloudWatchAgent, restart AmazonCloudWatchAgent and tell the result to cloudformation. rpm -Uvh ${CloudWatchAgentURL} /opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ClickHouseInstance8 --region ${AWS::Region} sudo mkfs.xfs ${DeviceName} sudo mkdir -p /home/clickhouse/data sudo mount ${DeviceName} /home/clickhouse/data echo "${DeviceName} /home/clickhouse/data xfs defaults 0 0" >> /etc/fstab aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-shard04-replica02,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` aws ec2 create-tags --region ${AWS::Region} --resources `curl http://169.254.169.254/latest/meta-data/instance-id` --tags Key=${RootStackName}-clickhouse-cluster,Value=`curl http://169.254.169.254/latest/meta-data/local-ipv4` cd /home/ec2-user/ aws secretsmanager get-secret-value --secret-id ${SecretId} --region ${AWS::Region} > secretfile aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-secret.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/find-clickhouse-node.py ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/clickhouse-install.sh ./ --region ${AWS::Region} aws s3 cp s3://${QSS3BucketName}/${QSS3KeyPrefix}scripts/create-table-demo.sql ./ --region ${AWS::Region} chmod +x clickhouse-install.sh ./clickhouse-install.sh ${ClickHouseVersion} ${ClickHouseNodeCount} ${RootStackName} ${AWS::Region} http://s3.${AWS::Region}.${AWS::URLSuffix}/${ClickHouseBucketName}/quickstart-clickhouse-data/ ${MoveFactor} ${ClickHouseTimezone} ${ZookeeperPrivateIp1} ${ZookeeperPrivateIp2} ${ZookeeperPrivateIp3} `python ./find-secret.py secretfile ` ${MaxThreads} ${MaxInsertThreads} ${DistributedProductMode} ${MaxMemoryUsage} ${LoadBalancing} ${MaxDataPartSize} "04" "02" > ch-install.log chmod +r /home/ec2-user/create-table-demo.sql flag=600 while((flag > 0)) do clickhouse-client --password `python ./find-secret.py secretfile ` --query "SELECT * FROM system.clusters;" > /dev/null if [ $? -eq 0 ] then echo "ClickHouse is running" >> ch-install.log break fi let flag-- sleep 1 done # Clean rm -rf /home/ec2-user/find-clickhouse-node.py rm -rf /home/ec2-user/clickhouse-install.sh rm -rf /home/ec2-user/instancelist-1 rm -rf /home/ec2-user/result-1 rm -rf /home/ec2-user/instancelist-2 rm -rf /home/ec2-user/result-2 rm -rf /home/ec2-user/instancelist-3 rm -rf /home/ec2-user/result-3 rm -rf /home/ec2-user/instancelist-4 rm -rf /home/ec2-user/result-4 rm -rf /home/ec2-user/instancelist-5 rm -rf /home/ec2-user/result-5 rm -rf /home/ec2-user/instancelist-6 rm -rf /home/ec2-user/result-6 rm -rf /home/ec2-user/instancelist-7 rm -rf /home/ec2-user/result-7 rm -rf /home/ec2-user/instancelist-8 rm -rf /home/ec2-user/result-8 function cfn_check_result { if [ $? = 0 ]; then echo "Create demo table success" >> ch-install.log rm -rf secretfile rm -rf find-secret.py rm -rf create-table-demo.sql return 0 else echo "Fail to create demo table" >> ch-install.log return 1 fi } echo "Begin to create demo table" >> ch-install.log clickhouse-client --user default --password `python ./find-secret.py secretfile ` -d default --multiquery < create-table-demo.sql cfn_check_result /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource ClickHouseInstance8 --region ${AWS::Region} Tags: - Key: Name Value: !Ref ClickHouseNodeName8 CreationPolicy: ResourceSignal: Count: 1 Timeout: "PT60M" Outputs: ClickHouseInstanceID1: Condition: 2NodesCondition Description: ClickHouse instance Value: !Ref ClickHouseInstance1 ClickHousePrivateIp1: Condition: 2NodesCondition Value: !GetAtt ClickHouseInstance1.PrivateIp ClickHouseInstanceID2: Condition: 2NodesCondition Description: ClickHouse instance Value: !Ref ClickHouseInstance2 ClickHousePrivateIp2: Condition: 2NodesCondition Value: !GetAtt ClickHouseInstance2.PrivateIp ClickHouseInstanceID3: Condition: 4NodesCondition Description: ClickHouse instance Value: !Ref ClickHouseInstance3 ClickHousePrivateIp3: Condition: 4NodesCondition Value: !GetAtt ClickHouseInstance3.PrivateIp ClickHouseInstanceID4: Condition: 4NodesCondition Description: ClickHouse instance Value: !Ref ClickHouseInstance4 ClickHousePrivateIp4: Condition: 4NodesCondition Value: !GetAtt ClickHouseInstance4.PrivateIp ClickHouseInstanceID5: Condition: 6NodesCondition Description: ClickHouse instance Value: !Ref ClickHouseInstance5 ClickHousePrivateIp5: Condition: 6NodesCondition Value: !GetAtt ClickHouseInstance5.PrivateIp ClickHouseInstanceID6: Condition: 6NodesCondition Description: ClickHouse instance Value: !Ref ClickHouseInstance6 ClickHousePrivateIp6: Condition: 6NodesCondition Value: !GetAtt ClickHouseInstance6.PrivateIp ClickHouseInstanceID7: Condition: 8NodesCondition Description: ClickHouse instance Value: !Ref ClickHouseInstance7 ClickHousePrivateIp7: Condition: 8NodesCondition Value: !GetAtt ClickHouseInstance7.PrivateIp ClickHouseInstanceID8: Condition: 8NodesCondition Description: ClickHouse instance Value: !Ref ClickHouseInstance8 ClickHousePrivateIp8: Condition: 8NodesCondition Value: !GetAtt ClickHouseInstance8.PrivateIp ImageId: Value: !Ref LatestAmiId