AWSTemplateFormatVersion: 2010-09-09 Description: >- AWS Deep Learning Ubuntu Desktop with DCV Server. Connect using a DCV Client. Parameters: AWSUbuntuAMIType: Description: Ubuntu Pro 22.04 LTS, Ubuntu Pro 20.04 LTS. Type: String Default: "UbuntuPro2204LTS" AllowedValues: - "UbuntuPro2004LTS" - "UbuntuPro2204LTS" S3Bucket: Description: S3 bucket name for read write access from desktop Type: String S3Import: Description: >- (Optional) Advanced option to specify S3 import prefix for FSx file-system. See FSxForLustre parameter to enable FSx for Lustre file-system. Type: String Default: "" UbuntuAMIOverride: Description: >- (Optional) Advanced option to override the Ubuntu 20.04 or 22.04 AMI. Leave blank, if unsure. Type: String AllowedPattern: '(ami-[0-9a-z]{17})?' DesktopVpcId: Description: Desktop VPC ID Type: 'AWS::EC2::VPC::Id' DesktopVpcSubnetId: Description: Desktop VPC Subnet ID. Subnet must be public for access over Internet. Type: 'AWS::EC2::Subnet::Id' DesktopHasPublicIpAddress: Description: Should a Public Ip Address be associated with the Desktop? Type: String Default: "true" AllowedValues: - "true" - "false" EbsVolumeSize: Default: 200 Description: Ebs volume size (GB) Type: Number MinValue: 200 EbsVolumeType: Default: 'gp3' Description: Ebs volume type Type: String AllowedValues: - 'gp2' - 'gp3' EFSFileSystemId: Description: >- (Optional) Advanced option to specify an existing EFS File System Id with an existing mount target in your subnet. Leave blank to create a new EFS file system. Type: String AllowedPattern: '(^fs-[0-9a-f]+)$|()$' Default: '' ConstraintDescription: Should be a Valid EFS File System Id, or blank EFSMountPath: Description: EFS file-system mount directory path. Type: String Default: '/home/ubuntu/efs' ConstraintDescription: Should be a valid file-system directory path DesktopSecurityGroupId: Description: >- (Optional) Advanced option to specify existing Desktop Security Group Id. Leave blank to create new Security Group. Type: String AllowedPattern: '(^sg-[0-9a-z]+)$|()$' Default: '' ConstraintDescription: Should be a Valid SecurityGroup Id in selected VPC, or blank FSxCapacity: Description: Must be a multiple of 1200 GBs Type: Number Default: 1200 MinValue: 1200 FSxForLustre: Description: >- Advanced option to enable, disable FSx for Lustre file-system. If enabled, a FSx for Lustre file-system is created and mounted on the desktop. The FSx for Lustre file-system automatically imports data from s3://S3bucket/S3Import. See S3Bucket and S3Import parameters. Type: String Default: 'disabled' AllowedValues: - 'enabled' - 'disabled' FSxMountPath: Description: FSx file-system mount directory path. Type: String Default: '/home/ubuntu/fsx' ConstraintDescription: Should be a valid file-system directory path KeyName: Description: >- Name of an existing Amazon EC2 KeyPair to enable SSH and DCV access to the desktop Type: 'AWS::EC2::KeyPair::KeyName' DesktopInstanceType: Description: EC2 instance type for desktop Type: String Default: g4dn.xlarge AllowedValues: - m5d.2xlarge - m5d.4xlarge - m5d.8xlarge - m5d.12xlarge - m5d.16xlarge - m5d.24xlarge - r5d.2xlarge - r5d.4xlarge - r5d.8xlarge - r5d.12xlarge - r5d.16xlarge - r5d.24xlarge - c5d.2xlarge - c5d.4xlarge - c5d.9xlarge - c5d.12xlarge - c5d.18xlarge - c5d.24xlarge - g3s.xlarge - g3.4xlarge - g3.8xlarge - g3.16xlarge - g4dn.xlarge - g4dn.2xlarge - g4dn.4xlarge - g4dn.8xlarge - g4dn.12xlarge - g4dn.16xlarge - g5.xlarge - g5.2xlarge - g5.4xlarge - g5.8xlarge - g5.12xlarge - g5.16xlarge - g5.24xlarge - g5.48xlarge - p3.2xlarge - p3.8xlarge - p3.16xlarge - p3dn.24xlarge - p4d.24xlarge - trn1.2xlarge - trn1.32xlarge - trn1n.32xlarge ConstraintDescription: Must be a valid CPU, or GPU instance type. DesktopAccessCIDR: Description: >- Restrict desktop access for SSH and DCV client from a valid CIDR range Type: String MinLength: '9' MaxLength: '18' AllowedPattern: '(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})' ConstraintDescription: Must be a valid CIDR range of the form x.x.x.x/x EBSOptimized: Description: >- Is the instance EBS optimized? Type: String Default: 'true' AllowedValues: - 'false' - 'true' Mappings: UbuntuPro2004LTS: us-east-1: AMI: ami-007ccac74cc85cb43 us-east-2: AMI: ami-099e81f8d52ddb352 us-west-2: AMI: ami-0f48fa756eb8ae717 eu-west-1: AMI: ami-06e7d4ed8752ea1a1 eu-central-1: AMI: ami-0ebdbdc3009d263a1 ap-southeast-1: AMI: ami-09b43c6a9d2db239d ap-southeast-2: AMI: ami-093ee8ca5cccbd9dd ap-south-1: AMI: ami-0c281c09ef103f5c3 ap-northeast-1: AMI: ami-0734a013c43b8d8a2 ap-northeast-2: AMI: ami-08accefb19e265cff UbuntuPro2204LTS: us-east-1: AMI: ami-0026ffa8c08991c66 us-east-2: AMI: ami-05e2702b01f78d2c5 us-west-2: AMI: ami-02731d1741a1e602c eu-west-1: AMI: ami-0a44faf0e41d4bbf0 eu-central-1: AMI: ami-0233eddd69ff36e9a ap-southeast-1: AMI: ami-0129527ba7a47484c ap-southeast-2: AMI: ami-0a868a395ec98483f ap-south-1: AMI: ami-09fd0f1beba83f58e ap-northeast-1: AMI: ami-05785a950811cb02d ap-northeast-2: AMI: ami-0d574b5641a75986b Conditions: OverrideAMI: !Not - !Equals - !Ref UbuntuAMIOverride - '' CreateNewEFSFileSystem: !Equals - !Ref EFSFileSystemId - '' FSxForLustreEnabled: !Equals - !Ref FSxForLustre - 'enabled' CreateNewSecurityGroup: !Equals - !Ref DesktopSecurityGroupId - '' EfaEnabled: !Or - !Equals - !Ref DesktopInstanceType - 'trn1.32xlarge' - !Equals - !Ref DesktopInstanceType - 'p4d.24xlarge' - !Equals - !Ref DesktopInstanceType - 'trn1n.32xlarge' - !Equals - !Ref DesktopInstanceType - 'p3dn.24xlarge' Resources: DesktopSecurityGroup: Condition: CreateNewSecurityGroup Type: 'AWS::EC2::SecurityGroup' Properties: GroupDescription: Desktop security group VpcId: !Ref DesktopVpcId Tags: - Key: Name Value: !Ref 'AWS::StackName' DesktopSecurityGroupCIDRIngressDCV: Condition: CreateNewSecurityGroup Type: 'AWS::EC2::SecurityGroupIngress' Properties: Description: Ingress from CIDR for NICE-DCV access to graphics desktop GroupId: !GetAtt DesktopSecurityGroup.GroupId CidrIp: !Ref DesktopAccessCIDR IpProtocol: tcp FromPort: 8443 ToPort: 8443 DesktopSecurityGroupCIDRIngressSSH: Condition: CreateNewSecurityGroup Type: 'AWS::EC2::SecurityGroupIngress' Properties: Description: Ingress from CIDR for SSH access to graphics desktop GroupId: !GetAtt DesktopSecurityGroup.GroupId CidrIp: !Ref DesktopAccessCIDR IpProtocol: tcp FromPort: 22 ToPort: 22 DesktopSecurityGroupCIDRIngressSelf: Condition: CreateNewSecurityGroup Type: 'AWS::EC2::SecurityGroupIngress' Properties: Description: Ingress from self security group GroupId: !GetAtt DesktopSecurityGroup.GroupId IpProtocol: "tcp" FromPort: 0 ToPort: 65535 SourceSecurityGroupId: !GetAtt DesktopSecurityGroup.GroupId DesktopSecurityGroupCIDREgress: Condition: CreateNewSecurityGroup Type: 'AWS::EC2::SecurityGroupEgress' Properties: Description: Egress rule for out bound traffic GroupId: !GetAtt DesktopSecurityGroup.GroupId IpProtocol: tcp FromPort: 0 ToPort: 65535 CidrIp: '0.0.0.0/0' EFSFileSystem: Type: 'AWS::EFS::FileSystem' Condition: CreateNewEFSFileSystem DeletionPolicy: Retain UpdateReplacePolicy: Retain Properties: Encrypted : true PerformanceMode: generalPurpose FileSystemTags: - Key: Name Value: !Ref 'AWS::StackName' EFSMountTarget: Condition: CreateNewEFSFileSystem Type: 'AWS::EFS::MountTarget' Properties: FileSystemId: !Ref EFSFileSystem SubnetId: !Ref DesktopVpcSubnetId SecurityGroups: - !GetAtt EFSSecurityGroup.GroupId EFSSecurityGroup: Condition: CreateNewEFSFileSystem Type: 'AWS::EC2::SecurityGroup' Properties: GroupDescription: Security group for EFS mount target VpcId: !Ref DesktopVpcId SecurityGroupIngress: - IpProtocol: tcp FromPort: 2049 ToPort: 2049 SourceSecurityGroupId: !If - CreateNewSecurityGroup - !GetAtt DesktopSecurityGroup.GroupId - !Ref DesktopSecurityGroupId Description: Ingress from desktop SecurityGroupEgress: - IpProtocol: tcp FromPort: 2049 ToPort: 2049 DestinationSecurityGroupId: !If - CreateNewSecurityGroup - !GetAtt DesktopSecurityGroup.GroupId - !Ref DesktopSecurityGroupId Description: Egress to desktop EFSSecurityGroupIngress: Condition: CreateNewEFSFileSystem Type: 'AWS::EC2::SecurityGroupIngress' Properties: Description: Ingress from within the EFSSecurityGroup GroupId: !GetAtt EFSSecurityGroup.GroupId IpProtocol: tcp FromPort: 2049 ToPort: 2049 SourceSecurityGroupId: !GetAtt EFSSecurityGroup.GroupId EFSSecurityGroupEgress: Condition: CreateNewEFSFileSystem Type: 'AWS::EC2::SecurityGroupEgress' Properties: Description: Ingress from within the EFSSecurityGroup GroupId: !GetAtt EFSSecurityGroup.GroupId IpProtocol: tcp FromPort: 2049 ToPort: 2049 DestinationSecurityGroupId: !GetAtt EFSSecurityGroup.GroupId FSxSecurityGroup: Condition: FSxForLustreEnabled Type: 'AWS::EC2::SecurityGroup' Properties: GroupDescription: Security group for FSx mount target VpcId: !Ref DesktopVpcId SecurityGroupIngress: - IpProtocol: tcp FromPort: 988 ToPort: 988 SourceSecurityGroupId: !If - CreateNewSecurityGroup - !GetAtt DesktopSecurityGroup.GroupId - !Ref DesktopSecurityGroupId Description: Ingress from desktop SecurityGroupEgress: - IpProtocol: tcp FromPort: 988 ToPort: 988 DestinationSecurityGroupId: !If - CreateNewSecurityGroup - !GetAtt DesktopSecurityGroup.GroupId - !Ref DesktopSecurityGroupId Description: Egress to desktop FSxSecurityGroupIngress: Condition: FSxForLustreEnabled Type: 'AWS::EC2::SecurityGroupIngress' Properties: Description: Ingress from within the FSxSecurityGroup GroupId: !GetAtt FSxSecurityGroup.GroupId IpProtocol: tcp FromPort: 988 ToPort: 988 SourceSecurityGroupId: !GetAtt FSxSecurityGroup.GroupId FSxSecurityGroupEgress: Condition: FSxForLustreEnabled Type: 'AWS::EC2::SecurityGroupEgress' Properties: Description: Ingress from within the FSxSecurityGroup GroupId: !GetAtt FSxSecurityGroup.GroupId IpProtocol: tcp FromPort: 988 ToPort: 988 DestinationSecurityGroupId: !GetAtt FSxSecurityGroup.GroupId FSxFileSystem: Condition: FSxForLustreEnabled Type: AWS::FSx::FileSystem Properties: FileSystemType: 'LUSTRE' LustreConfiguration: DeploymentType: SCRATCH_2 AutoImportPolicy: NEW_CHANGED ImportPath: !Sub s3://${S3Bucket}/${S3Import} SecurityGroupIds: - !GetAtt FSxSecurityGroup.GroupId StorageCapacity: !Ref FSxCapacity StorageType: 'SSD' SubnetIds: - !Ref DesktopVpcSubnetId Tags: - Key: Name Value: !Ref 'AWS::StackName' InstanceRole: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: - ec2.amazonaws.com - sagemaker.amazonaws.com Action: - 'sts:AssumeRole' Path: / ManagedPolicyArns: - !Sub 'arn:${AWS::Partition}:iam::aws:policy/AmazonSageMakerFullAccess' Policies: - PolicyName: s3-read-write PolicyDocument: Statement: - Effect: Allow Action: - "s3:Get*" - "s3:List*" - "s3:PutObject*" - "s3:DeleteObject*" Resource: - !Sub 'arn:aws:s3:::${S3Bucket}' - !Sub 'arn:aws:s3:::${S3Bucket}/*' - PolicyName: dcv-license-s3 PolicyDocument: Statement: - Effect: Allow Action: - 's3:GetObject' Resource: - !Sub 'arn:aws:s3:::dcv-license.${AWS::Region}/*' InstanceProfile: Type: 'AWS::IAM::InstanceProfile' Properties: Path: / Roles: - !Ref InstanceRole DesktopLaunchTemplate: Type: AWS::EC2::LaunchTemplate Properties: LaunchTemplateData: MetadataOptions: HttpTokens: "required" HttpEndpoint: "enabled" NetworkInterfaces: - AssociatePublicIpAddress: !Ref DesktopHasPublicIpAddress DeviceIndex: "0" Groups: - !If - CreateNewSecurityGroup - !GetAtt DesktopSecurityGroup.GroupId - !Ref DesktopSecurityGroupId SubnetId: !Ref DesktopVpcSubnetId BlockDeviceMappings: - DeviceName: "/dev/sda1" Ebs: VolumeSize: !Ref EbsVolumeSize VolumeType: !Ref EbsVolumeType Encrypted: true DeleteOnTermination: true ImageId: !If - OverrideAMI - !Ref UbuntuAMIOverride - !FindInMap - !Ref 'AWSUbuntuAMIType' - !Ref 'AWS::Region' - AMI EbsOptimized: !Ref EBSOptimized EfaSecurityGroup: Condition: EfaEnabled Type: 'AWS::EC2::SecurityGroup' Properties: GroupDescription: Efa security group VpcId: !Ref DesktopVpcId Tags: - Key: Name Value: !Ref 'AWS::StackName' EfaSecurityGroupIngressSelf: Condition: EfaEnabled Type: 'AWS::EC2::SecurityGroupIngress' Properties: Description: Ingress from self security group GroupId: !Ref EfaSecurityGroup IpProtocol: "-1" FromPort: 0 ToPort: 65535 SourceSecurityGroupId: !Ref EfaSecurityGroup EfaSecurityGroupEgressSelf: Condition: EfaEnabled Type: 'AWS::EC2::SecurityGroupEgress' Properties: Description: Egress rule for out bound traffic GroupId: !Ref EfaSecurityGroup IpProtocol: "-1" FromPort: 0 ToPort: 65535 DestinationSecurityGroupId: !Ref EfaSecurityGroup EfaDesktopLaunchTemplate: Condition: EfaEnabled Type: AWS::EC2::LaunchTemplate Properties: LaunchTemplateData: MetadataOptions: HttpTokens: "required" HttpEndpoint: "enabled" NetworkInterfaces: - AssociatePublicIpAddress: !Ref DesktopHasPublicIpAddress Description: NetworkInterfaces Configuration For EFA NetworkCardIndex: 0 DeviceIndex: 0 DeleteOnTermination: true Groups: - !Ref EfaSecurityGroup - !If - CreateNewSecurityGroup - !GetAtt DesktopSecurityGroup.GroupId - !Ref DesktopSecurityGroupId InterfaceType: efa BlockDeviceMappings: - DeviceName: "/dev/sda1" Ebs: VolumeSize: !Ref EbsVolumeSize VolumeType: !Ref EbsVolumeType Encrypted: true DeleteOnTermination: true ImageId: !If - OverrideAMI - !Ref UbuntuAMIOverride - !FindInMap - !Ref 'AWSUbuntuAMIType' - !Ref 'AWS::Region' - AMI EbsOptimized: !Ref EBSOptimized DesktopInstance: Type: 'AWS::EC2::Instance' Properties: LaunchTemplate: LaunchTemplateId: !If - EfaEnabled - !Ref EfaDesktopLaunchTemplate - !Ref DesktopLaunchTemplate Version: !GetAtt DesktopLaunchTemplate.LatestVersionNumber InstanceType: !Ref DesktopInstanceType IamInstanceProfile: !Ref InstanceProfile KeyName: !Ref KeyName Tags: - Key: "Name" Value: !Sub '${AWS::StackName}-deep-learning-ubuntu-desktop' UserData: !Base64 'Fn::Join': - '' - - | Content-Type: multipart/mixed; boundary="//" MIME-Version: 1.0 --// Content-Type: text/cloud-config; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="cloud-config.txt" #cloud-config cloud_final_modules: - [scripts-user, always] --// Content-Type: text/x-shellscript; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="userdata.txt" #!/bin/bash -xe echo "Cloud init in progress!" > /etc/motd . /etc/os-release distro=ubuntu${VERSION_ID//[.]/""} arch="x86_64" echo "Ubuntu $distro/$arch" # setup graphics desktop export DEBIAN_FRONTEND=noninteractive export DEBCONF_NONINTERACTIVE_SEEN=true CUDA=11.8 CUDA_DASH=11-8 CUDNN=8.9.2.26 # check if we have a GPU and if Nvidia drivers and CUDA need to be installed [[ ! -z $(lspci -v | grep NVIDIA) ]] && \ [[ ! -x "$(command -v nvidia-smi)" ]] && \ apt-get -y install linux-headers-$(uname -r) && \ wget https://developer.download.nvidia.com/compute/cuda/repos/$distro/$arch/cuda-keyring_1.0-1_all.deb && \ dpkg -i cuda-keyring_1.0-1_all.deb && \ apt-get update && apt-get -y purge cuda && apt-get -y purge nvidia-* && apt-get -y purge libnvidia-* && apt-get -y autoremove && \ apt-get -y install cuda-${CUDA_DASH} && \ apt-get -y install libcudnn8=${CUDNN}-1+cuda${CUDA} && \ apt-get -y install libcudnn8-dev=${CUDNN}-1+cuda${CUDA} && \ apt-get -y install cuda-compat-12-1 && \ apt-get -y install cuda-compat-12-2 && \ echo "export PATH=/usr/local/cuda-${CUDA}/bin:$PATH" >> /home/ubuntu/.bashrc && \ echo "export LD_LIBRARY_PATH=/usr/local/cuda-12.2/compat:/usr/local/cuda-12.1/compat:/usr/local/cuda-${CUDA}/lib64:$LD_LIBRARY_PATH" >> /home/ubuntu/.bashrc && \ reboot # check if we have a Trainium instance [[ -z $(lspci -v | grep NVIDIA) ]] && update-pciids if [[ ! -z $(lspci -v | grep Trainium) ]] && [[ ! -x "$(command -v /opt/aws/neuron/neuron-ls)" ]] then echo "deb https://apt.repos.neuron.amazonaws.com ${VERSION_CODENAME} main" > /etc/apt/sources.list.d/neuron.list wget -qO - https://apt.repos.neuron.amazonaws.com/GPG-PUB-KEY-AMAZON-AWS-NEURON.PUB | apt-key add - # Update OS packages apt-get -y update # Install OS headers apt-get -y install linux-headers-$(uname -r) # install git apt-get -y install git # install Neuron Driver apt-get -y install aws-neuronx-dkms # Install Neuron Tools apt-get -y install aws-neuronx-tools # Install Neuron Runtime apt-get -y install aws-neuronx-collectives apt-get -y install aws-neuronx-runtime-lib echo "export PATH=/opt/aws/neuron/bin:$PATH" >> /home/ubuntu/.bashrc fi # setup software repo for docker curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - apt-key fingerprint 0EBFCD88 add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" # setup software repo for fsx-lustre client wget -O - https://fsx-lustre-client-repo-public-keys.s3.amazonaws.com/fsx-ubuntu-public-key.asc | apt-key add - # add key for NICE-DCV wget https://d1uj6qtbmh3dt5.cloudfront.net/NICE-GPG-KEY gpg --import NICE-GPG-KEY # add visual code repository wget -q https://packages.microsoft.com/keys/microsoft.asc -O- | apt-key add - add-apt-repository "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main" # update and install required packages apt-get update apt-get -y install git tar apt-get -y install software-properties-common # install docker if it is not installed if [ ! -x "$(command -v docker)" ]; then apt-get -y install docker-ce docker-ce-cli containerd.io usermod -aG docker ubuntu # install nvidia container toolkit if we have a nvidia GPU if [[ -x "$(command -v nvidia-smi)" ]] then curl -s -L https://nvidia.github.io/nvidia-container-runtime/gpgkey | apt-key add - distribution=$ID$VERSION_ID curl -s -L https://nvidia.github.io/nvidia-container-runtime/$distribution/nvidia-container-runtime.list | \ tee /etc/apt/sources.list.d/nvidia-container-runtime.list apt-get update apt-get -y install nvidia-container-toolkit fi fi apt-get -y install tzdata apt-get -y install keyboard-configuration apt-get -y install gnupg2 apt-get -y install lsb-core apt-get -y install libopenmpi-dev apt-get -y install protobuf-compiler # install DCV server echo "install DCV server..." apt-get -y install ubuntu-desktop if [[ "$VERSION_ID" == "20.04" ]] then bash -c 'echo "deb https://fsx-lustre-client-repo.s3.amazonaws.com/ubuntu focal main" > /etc/apt/sources.list.d/fsxlustreclientrepo.list && apt-get update' if [[ ! -x "$(command -v dcv)" ]] then apt-get -y install gdm3 echo "/usr/sbin/gdm3" > /etc/X11/default-display-manager dpkg-reconfigure gdm3 sed -i -e "s/#WaylandEnable=false/WaylandEnable=false/g" /etc/gdm3/custom.conf systemctl restart gdm3 apt-get -y install mesa-utils if [[ -x "$(command -v nvidia-xconfig)" ]] && [[ -x "$(command -v nvidia-smi)" ]] then nvidia-xconfig --preserve-busid --enable-all-gpus fi #restart X server echo "restart X-server" systemctl set-default graphical.target systemctl isolate graphical.target wget https://d1uj6qtbmh3dt5.cloudfront.net/2023.0/Servers/nice-dcv-2023.0-15022-ubuntu2004-x86_64.tgz tar -xvzf nice-dcv-2023.0-15022-ubuntu2004-x86_64.tgz cd nice-dcv-2023.0-15022-ubuntu2004-x86_64 apt-get -y install ./nice-dcv-server_2023.0.15022-1_amd64.ubuntu2004.deb reboot fi elif [[ "$VERSION_ID" == "22.04" ]] then bash -c 'echo "deb https://fsx-lustre-client-repo.s3.amazonaws.com/ubuntu jammy main" > /etc/apt/sources.list.d/fsxlustreclientrepo.list && apt-get update' if [[ ! -x "$(command -v dcv)" ]] then apt-get -y install gdm3 echo "/usr/sbin/gdm3" > /etc/X11/default-display-manager dpkg-reconfigure gdm3 sed -i -e "s/#WaylandEnable=false/WaylandEnable=false/g" /etc/gdm3/custom.conf systemctl restart gdm3 apt-get -y install mesa-utils if [[ -x "$(command -v nvidia-xconfig)" ]] && [[ -x "$(command -v nvidia-smi)" ]] then nvidia-xconfig --preserve-busid --enable-all-gpus fi #restart X server echo "restart X-server" systemctl set-default graphical.target systemctl isolate graphical.target wget https://d1uj6qtbmh3dt5.cloudfront.net/2023.0/Servers/nice-dcv-2023.0-15022-ubuntu2204-x86_64.tgz tar -xvzf nice-dcv-2023.0-15022-ubuntu2204-x86_64.tgz cd nice-dcv-2023.0-15022-ubuntu2204-x86_64 apt-get -y install ./nice-dcv-server_2023.0.15022-1_amd64.ubuntu2204.deb reboot fi else echo "Ubuntu $VERSION_ID is not supported; must be one of 20.04, or 22.04" exit 1 fi #restart X server systemctl set-default graphical.target systemctl isolate graphical.target # Create DCV server configuration file [[ -d /opt/dcv-session-store ]] || mkdir /opt/dcv-session-store echo "[license]" >> dcv.conf echo "[log]" >> dcv.conf echo "[session-management]" >> dcv.conf echo "create-session = true" >> dcv.conf echo "[session-management/defaults]" >> dcv.conf echo "[session-management/automatic-console-session]" >> dcv.conf echo "owner=ubuntu" >> dcv.conf echo "storage-root=\"/opt/dcv-session-store/\"" >> dcv.conf echo "[display]" >> dcv.conf echo "[connectivity]" >> dcv.conf echo "[security]" >> dcv.conf echo "authentication=\"system\"" >> dcv.conf echo "[clipboard]" >> dcv.conf echo "primary-selection-copy=true" >> dcv.conf echo "primary-selection-paste=true" >> dcv.conf mv dcv.conf /etc/dcv/dcv.conf # Create DCV session permissions files rm -f /home/ubuntu/dcv.perms echo "[permissions]" >> /home/ubuntu/dcv.perms echo "%owner% allow builtin" >> /home/ubuntu/dcv.perms # Enable DCV server systemctl enable dcvserver systemctl restart dcvserver echo "install DCV server complete" # install nfs-common apt-get -y install nfs-common - AWS_REGION= - !Ref AWS::Region - |+ - EFS_FS_ID= - !If - CreateNewEFSFileSystem - !Ref EFSFileSystem - !Ref EFSFileSystemId - |+ - EFS_MOUNT_PATH= - !Ref EFSMountPath - |+ - FSX_MOUNT_PATH= - !Ref FSxMountPath - |+ - DESKTOP_ROLE_ARN= - !GetAtt InstanceRole.Arn - |+ - DESKTOP_SUBNET_ID= - !Ref DesktopVpcSubnetId - |+ - DESKTOP_SG_ID= - !If - CreateNewSecurityGroup - !GetAtt DesktopSecurityGroup.GroupId - !Ref DesktopSecurityGroupId - |+ - FSX_ENABLED= - !If - FSxForLustreEnabled - "true" - "false" - |+ - FSX_FS_ID= - !If - FSxForLustreEnabled - !Ref FSxFileSystem - '' - |+ - FSX_MOUNT_NAME= - !If - FSxForLustreEnabled - !GetAtt FSxFileSystem.LustreMountName - '' - |+ - EFA_ENABLED= - !If - EfaEnabled - "true" - "false" - |+ - | # Install EFA software, if Efa is enabled if [[ "$EFA_ENABLED" == "true" ]] then # disable ptrace sysctl -w kernel.yama.ptrace_scope=0 if [[ ! -d "/opt/amazon/efa" ]] then curl -O https://efa-installer.amazonaws.com/aws-efa-installer-latest.tar.gz wget https://efa-installer.amazonaws.com/aws-efa-installer.key && gpg --import aws-efa-installer.key cat aws-efa-installer.key | gpg --fingerprint wget https://efa-installer.amazonaws.com/aws-efa-installer-latest.tar.gz.sig && gpg --verify ./aws-efa-installer-latest.tar.gz.sig tar -xvf aws-efa-installer-latest.tar.gz cd aws-efa-installer && bash efa_installer.sh --yes cd / && rm -rf aws-efa-installer-latest.tar.gz aws-efa-installer fi fi # Create EFS mount script cat >/usr/local/bin/mount-efs.sh </usr/local/bin/mount-fsx.sh </home/ubuntu/.aws/config <> /home/ubuntu/.bashrc echo "export desktop_sg_id=${DESKTOP_SG_ID}" >> /home/ubuntu/.bashrc echo "export desktop_subnet_id=${DESKTOP_SUBNET_ID}" >> /home/ubuntu/.bashrc echo "export efs_fs_id=${EFS_FS_ID}" >> /home/ubuntu/.bashrc if [[ "$FSX_ENABLED" == "true" ]] then echo "export fsx_fs_id=${FSX_FS_ID}" >> /home/ubuntu/.bashrc echo "export fsx_mount_name=${FSX_MOUNT_NAME}" >> /home/ubuntu/.bashrc fi # install miniconda3 if anaconda3, or miniconda3 are not installed if [[ ! -d "/home/ubuntu/anaconda3" ]] && [[ ! -d "/home/ubuntu/miniconda3" ]] then wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O /home/ubuntu/miniconda3.sh HOME=/home/ubuntu bash /home/ubuntu/miniconda3.sh -b -p /home/ubuntu/miniconda3 echo "source /home/ubuntu/miniconda3/etc/profile.d/conda.sh" >> /home/ubuntu/.bashrc rm /home/ubuntu/miniconda3.sh source /home/ubuntu/miniconda3/etc/profile.d/conda.sh conda update -y --name base -c defaults conda # install jupyterlab and boto3 in base env conda activate && \ conda install -y -c conda-forge jupyterlab && \ conda install -y ipykernel && \ conda install -y boto3 && \ conda install -y nb_conda_kernels && \ conda deactivate # create conda environments conda create -y --prefix /home/ubuntu/miniconda3/envs/tensorflow python=3.10 && \ conda activate tensorflow && \ pip3 install --upgrade pip && \ pip3 install tensorflow==2.12.1 && \ pip3 install datasets && \ pip3 install transformers && \ conda install -y ipykernel && \ conda install -y boto3 && \ conda deactivate conda create -y --prefix /home/ubuntu/miniconda3/envs/pytorch python=3.10 && \ conda activate pytorch && \ pip3 install --upgrade pip && \ pip3 install torch==2.0.1 torchvision && \ pip3 install datasets && \ pip3 install transformers && \ conda install -y ipykernel && \ conda install -y boto3 && \ conda deactivate conda create -y --prefix /home/ubuntu/miniconda3/envs/mxnet python=3.10 && \ conda activate mxnet && \ pip3 install --upgrade pip && \ pip3 install mxnet && \ conda install -y ipykernel && \ conda install -y boto3 && \ conda deactivate if [[ ! -z $(/opt/aws/neuron/bin/neuron-ls | grep instance-type | grep trn1) ]] || \ [[ ! -z $(/opt/aws/neuron/bin/neuron-ls | grep instance-type | grep trn1n) ]] || \ [[ ! -z $(/opt/aws/neuron/bin/neuron-ls | grep instance-type | grep inf2) ]] then conda activate pytorch && \ pip3 config set global.extra-index-url https://pip.repos.neuron.amazonaws.com && \ pip3 install neuronx-cc torch-neuronx torchvision && \ pip3 install optimum[neuronx] && \ conda deactivate conda activate tensorflow && \ pip3 config set global.extra-index-url https://pip.repos.neuron.amazonaws.com && \ pip3 install neuronx-cc tensorflow-neuronx && \ pip3 install optimum[neuronx] && \ conda deactivate fi chown -R ubuntu:ubuntu /home/ubuntu/miniconda3 chown -R ubuntu:ubuntu /home/ubuntu/.conda fi echo "conda activate" >> /home/ubuntu/.bashrc # install nodejs 20.x curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && \ apt-get install -y nodejs # install aws cli apt-get -y install awscli # install visual code apt-get -y install code echo "NICE DCV server is enabled!" > /etc/motd Outputs: DesktopInstanceId: Description: Desktop instance Id Value: !Ref DesktopInstance DesktopRole: Description: IAM role attached to desktop instance profile Value: !GetAtt InstanceRole.Arn DesktopSecurityGroup: Description: Desktop security group Value: !If - CreateNewSecurityGroup - !GetAtt DesktopSecurityGroup.GroupId - !Ref DesktopSecurityGroupId EFSFileSystemId: Description: EFS file system attached to the desktop Value: !If - CreateNewEFSFileSystem - !Ref EFSFileSystem - !Ref EFSFileSystemId