:toc: macro
# Supplemental information to the Implementation Guide
toc::[]
## Optimizing Semiconductor Design Tools and Applications on AWS
#### Disable Hyper-Threading
You can run this command to get detailed information about each core (physical core and Hyper-Thread):
[source,bash]
$ cat /proc/cpuinfo
If you run the ```lscpu --extended``` command before and after disabling Hyper-Threading, you can see which threads are online and offline:
[source,bash]
----
$ lscpu --extended
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
0 0 0 0 0:0:0:0 yes
1 0 0 1 1:1:1:0 yes
2 0 0 2 2:2:2:0 yes
3 0 0 3 3:3:3:0 yes
4 0 0 0 0:0:0:0 yes
5 0 0 1 1:1:1:0 yes
6 0 0 2 2:2:2:0 yes
7 0 0 3 3:3:3:0 yes
$ ./disable_ht.sh
$ lscpu --extended
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
0 0 0 0 0:0:0:0 yes
1 0 0 1 1:1:1:0 yes
2 0 0 2 2:2:2:0 yes
3 0 0 3 3:3:3:0 yes
4 - - - ::: no
5 - - - ::: no
6 - - - ::: no
7 - - - ::: no
----
To view all thread siblings, you can use the following command, or substitute “*†with a CPU number:
[source,bash]
----
$ cat/sys/devices/system/cpu/cpu*/topology/thread_siblings_list | sort -un
0,4
1,5
2,6
3,7
----
**Disable HT Using the AWS feature - CPU Options**
To disable Hyper-Threading using CPU Options, use the AWS CLI with run-instances and the --cpu-options flag. The following is an example with the Z1d.12xlarge instance:
[source, bash]
----
$ aws ec2 run-instances --image-id ami- asdfasdfasdfasdf \
--instance-type z1d.12xlarge --cpu-options \
"CoreCount=24,ThreadsPerCore=1" --key-name My_Key_Name
----
To verify the CpuOptions were set, use describe-instances:
[source, bash]
----
$ aws ec2 describe-instances --instance-ids i-1234qwer1234qwer
...
"CpuOptions": {
"CoreCount": 24,
"ThreadsPerCore": 1
},
...
----
**Disable HT on a Running System**
You can run the following script on a Linux instance to disable HT Technology while the system is running. This can be set up to run from an init script so that it applies to any instance when you launch the instance.
For example:
[source,bash]
----
for cpunum in $(cat/sys/devices/system/cpu/cpu*/topology/thread_siblings_list | sort -un | cut -s -d, -f2-)
do
echo 0 | sudo tee /sys/devices/system/cpu/cpu${cpunum}/online
done
----
**Disable HT Using the Boot File**
For example, the maxcpus value for a Z1d.12xlarge instance is 24 to disable Hyper-Threading:
[source,bash]
----
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 nvme_core.io_timeout=4294967295 maxcpus=24"
----
For instructions on how to update the kernel command line, see <<update-kernel, Update the Linux Kernel Command Line.>>
#### Change Clocksource to TSC (only for Xen based instances, not for Nitro based instances)
To change the clocksource on a Xen based instance , run the following command:
[source,bash]
----
$ sudo su -c "echo tsc > /sys/devices/system/cl*/cl*/current_clocksource"
----
To verify that the clocksource is set to tsc, run the following command:
[source,bash]
----
$ cat /sys/devices/system/cl*/cl*/current_clocksource
tsc
----
Verify that the clocksource changed with the dmesg command, as shown below:
[source,bash]
----
$ dmesg | grep clocksource
...
clocksource: Switched to clocksource tsc
----
#### Limiting Deeper C-states (Sleep State)
[source,bash]
----
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8 net.ifnames=0
biosdevname=0 nvme_core.io_timeout=4294967295 intel_idle.max_cstate=1"
----
For instructions on how to update the kernel command line, see <<update-kernel, Update the Linux Kernel Command Line>>.
**Enable Turbo Mode (Processor State) on Xen-Based Instances**
Amazon Linux and Amazon Linux 2 have turbo mode enabled by default, but other distributions may not. To ensure that turbo mode is enabled, run the following command:
[source,bash]
----
sudo su -c "echo 0 > /sys/devices/system/cpu/intel_pstate/no_turbo"
----
For more information about Amazon EC2 instance processor states, see the https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/processor_state_control.html[Processor State Control for Your EC2 Instance] page in the Amazon Elastic Compute Cloud User Guide for Linux Instances.
#### Change to Optimal Spinlock Setting on Xen-Based Instances
To disable the paravirtualized mode of spinlock on a Xen based instance, add xen_nopvspin=1 to the kernel command line in /boot/grub/grub.conf and restart. The following is an example kernel command:
[source,bash]
----
kernel /boot/vmlinuz-4.4.41-36.55.amzn1.x86_64 root=LABEL=/ console=tty1 console=ttyS0 selinux=0 xen_nopvspin=1
----
[[update-kernel]]
#### Update the Linux Kernel Command Line
You can update the Linux kernel command line with either the /etc/default/grub file or the /boot/grub/grub.conf file. To update the Linux kernel with the /etc/default/grub file:
1. Open the /etc/default/grub file: ```$ sudo vim /etc/default/grub```
2. Edit the ```GRUB_CMDLINE_LINUX_DEFAULT``` line, and make any necessary changes. For example: +
+
[source,bash]
----
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 nvme_core.io_timeout=4294967295 intel_idle.max_cstate=1"
----
+
3. Save the file and exit your editor.
4. Run the following command to rebuild the boot configuration.
$ grub2-mkconfig -o /boot/grub2/grub.cfg
+
[source,bash]
----
$ grub2-mkconfig -o /boot/grub2/grub.cfg
----
+
5. Reboot your instance to enable the new kernel option.
To update the Linux kernel with the /boot/grub/grub.conf file:
1. Open the /boot/grub/grub.conf file.
+
[source,bash]
----
$ sudo vim /boot/grub/grub.conf
----
+
2. Edit the kernel line. For example:
+
[source,bash]
----
# created by imagebuilder
default=0
timeout=1
hiddenmenu
title Amazon Linux 2014.09 (3.14.26-24.46.amzn1.x86_64)
root (hd0,0)
kernel /boot/vmlinuz-ver.amzn1.x86_64 <other_info> intel_idle.max_cstate=1
initrd /boot/initramfs-3.14.26-24.46.amzn1.x86_64.img
----
+
3. Save the file and exit your editor.
4. Reboot your instance to enable the new kernel option.
**Verify the Kernel Line**
After you update the kernel, you can verify the setting by running dmesg or /proc/cmdline at the kernel command line:
[source,bash]
----
$ dmesg | grep "Kernel command line"
[ 0.000000] Kernel command line: root=LABEL=/ console=tty1 console=ttyS0 maxcpus=18 xen_nopvspin=1
$ cat /proc/cmdline
root=LABEL=/ console=tty1 console=ttyS0 maxcpus=18 xen_nopvspin=1
----
**NFS Configuration and Optimization**
Prior to setting up an NFS server on AWS, you need to enable Amazon EC2 enhanced networking. We recommend using Amazon Linux 2 for your NFS server AMI.
A crucial part of high performing NFS are the mount parameters on the client. For example:
[source,bash]
----
rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2
----
A typical EFS mount command is shown in following example:
[source,bash]
----
$ sudo mount -t nfs4 –o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2
file-system-id.efs.aws-region.amazonaws.com:/ /efs-mount-point
----
### Kernel Virtual Memory
A good starting point to optimize memory utilization of the instances:
[source,bash]
----
vm.min_free_kbytes=1048576
vm.dirty_background_bytes=107374182
----