= Test Read Performance :toc: :icons: :linkattrs: :imagesdir: ../resources/images == Summary This section will test the read performance of the local FSx File Gateway in Region 2, the same region as this Windows instance, and compare it to the read performance of the remote FSx for Windows File Server in Region 1. There are many disk performance tools out there. link:https://aka.ms/diskspd[DiskSpd] have been pre-installed on the *WindowsInstance*. == Duration NOTE: It will take approximately 15 minutes to complete this section. == Step-by-step Guide IMPORTANT: Read through all steps below and watch the quick video before *_clicking_* the *Deploy to AWS* button. image::read-csvs.gif[title="Quick Step-by-step Video", align="left", width=600] === Compare read durations of large files from the local FSx File Gateway and the remote FSx for Windows File Server There are three CSV files preloaded on the file system. This is sample sales data downloaded from link:http://eforexcel.com/wp/downloads-18-sample-csv-files-data-sets-for-testing-sales/[eforexcel.com]. There is a 10,000 records file (10k Sales Records.csv @ 1.18 MB), a 1,000,000 records file (1m Sales Records.csv @ 119 MB), and a 5,000,000 records file (5m Sales Records.csv @ 595 MB). . On your local computer, Amazon EC2 Windows Instance, or your Smart Phone, open any stopwatch application (e.g. link:https://stopwatchtimers.com/stopwatch/[Stopwatch]). In the next few steps you will time how long it takes for the Windows Instance to open these three CSV files and compare the durations between opening them from the FSx file Gateway in the local region and the FSx for Windows File Server in the remote region. . From the *File Explorer* window mapped to the *Z:* drive pointing to the remote FSx for Windows File Server, *_context-click_* (*_right-click_*) the 10k Sales Records.csv file and *_select_* *Open With > Notepad*. *_Start_* the stopwatch. *_Stop_* the stopwatch once data first appears in Notepad. *_Record_* the time. . From the *File Explorer* window mapped to the *Y:* drive pointing to the local FSx File Gateway, *_context-click_* (*_right-click_*) the 10k Sales Records.csv file and *_select_* *Open With > Notepad*. *_Start_* the stopwatch. *_Stop_* the stopwatch once data first appears in Notepad. *_Record_* the time. + Due to the small size of the 10k Sales Records.csv file, the time to read the entire file is so short that it will be difficult to measure this using the stopwatch. + . Repeat the two steps above with the 1m Sales Records.csv and 5m Sales Records.csv files. . What times did you record? . Repeat the three read tests above . + NOTE: The initial read from the FSx File Gateway will experience higher latencies as the FSx File Gateway reads the file into its cache from the remote FSx for Windows File Server. Subsequent reads deliver the file directly from the local FSx File Gateway. + Below are some sample times recorded when accessing these files from a Windows Instance and FSx File Gateway in the us-west-1 (N. California) region and an FSx for Windows File Server in the us-east-1 (N. Virginia) region. + [cols="3,10,10"] |=== | File | FSx for Windows File Server (us-east-1) duration | FSx File Gateway (us-west-1) duration | 10k Sales Records.csv a| ~2.0 seconds a| ~0.5 seconds | 1m Sales Records.csv a| ~15.5 seconds a| ~1.5 seconds | 5m Sales Records.csv a| ~81.6 seconds a| ~7.9 seconds |=== + . Repeat the tests to see if you continue to see differences between accessing files from the remote FSx for Windows File Server and the local FSx File Gateway. === DiskSpd Read tests IMPORTANT: Read through all steps below and watch the quick video before *_clicking_* the *Deploy to AWS* button. image::read-test.gif[title="Quick Step-by-step Video", align="left", width=600] ==== Test read performance accessing the remote FSx for Windows File Server. . From the remote desktop session to the *Windows Instance*, *_click_* *Start* >> *Windows PowerShell*. + IMPORTANT: This section assumes the remote FSx for Windows File Server is mapped as the *Z:/* drive. + . Run the DiskSpeed script below to test read performance of the remote FSx for Windows File Server mapped as the **Z:** drive. + ```sh C:\Tools\DiskSpd\amd64\DiskSpd.exe -d120 -w0 -r -t1 -o32 -b1M -Sr -L Z:\${env:computername}-fsxw.dat ``` + While the script is running, open *Task Manager* and monitor network performance (e.g. Start >> Task Manager >> More details >> Performance (tab) >> Ethernet). The DiskSpd script will complete in 120 seconds. After the script has completed, the output will be displayed in the PowerShell window. + * What was the P99 (99th %-tile) latency in ms of your test? - This is found in the DiskSpd output. It is in the *total* table at the bottom. * What was the P99.99 (99.99th %-tile) latency in ms of your test? - This is found in the DiskSpd output. It is in the *total* table at the bottom. * What was the Total Read IO MiB/s? - This is found in the DiskSpd output. It is under *Read IO* under the *MiB/s* column. * What was the I/O per second? - This is found in the DiskSpd output. It is under *Read IO* under the *I/O per s* column. * What was the AvgLat? - This is found in the DiskSpd output. It is under *Read IO* under the *AvgLat* column. + . Copy the following table to your local computer and record the results in the *FSx for Windows File Server* Test 1 column. . Repeat the test and record the results in the *FSx for Windows File Server* Test 2 column. + [width="50%",cols="d,d,d,d,d",frame="topbot",options="header"] |=== h| EC2 Instance 2+h| FSx for Windows File Server 2+h| FSx File Gateway s| DiskSpd Read (metric) h| Test 1 h| Test 2 h| Test 1 h| Test 2 | Read IO throughput (MiB/s) | | | | | Read IO I/O per s | | | | | Read IO AvgLat (ms) | | | | | Min %-tile (ms) | | | | | 50th %-tile (ms) | | | | | 90th %-tile (ms) | | | | | 99th %-tile (ms) | | | | | 99.99th %-tile (ms) | | | | |=== + . Experiment with different DiskSpd parameter settings. Use the table below as a guide. Test with different block sizes (-b), number of outstanding I/O requests (-o), number of threads per file (-t), and disable local caching (-Sr). + [cols="3,10"] |=== | Parameter | Description | `-b[K\|M\|G]` a| Block size in bytes or KiB, MiB, or GiB (default = 64K). | `-o` a| Number of outstanding I/O requests per-target per-thread. (1 = synchronous I/O, unless more than one thread is specified with by using `-F`.) (default = 2) | `-r[K\|M\|G]` a| Random I/O aligned to the specified number of bytes or KiB, MiB, GiB, or blocks. Overrides -s (default stride = block size). | `-s[K\|M\|G]` a| Sequential stride size, offset between subsequent I/O operations in bytes or KiB, MiB, GiB, or blocks. Ignored if -r is specified (default access = sequential, default stride = block size). | `-t` a| Number of threads per target. Conflicts with `-F`, which specifies the total number of threads. | `-Sr` a| Disable local caching. |=== * What different parameters did you test? * How did the different parameter options alter the results? ==== Test read performance accessing the local FSx File Gateway. . From the remote desktop session to the *Windows Instance*, open another *Windows PowerShell* window by *_clicking_* *Start* >> *Windows PowerShell*. + IMPORTANT: This section assumes the local FSx File Gateway is mapped as the *Y:/* drive. + . Run the DiskSpeed script below to test read performance of the remote FSx File Gateway mapped as the **Y:** drive. + ```sh C:\Tools\DiskSpd\amd64\DiskSpd.exe -d120 -w0 -r -t1 -o32 -b1M -Sr -L Y:\${env:computername}-fsxw.dat ``` + While the script is running, open *Task Manager* and monitor network performance (e.g. Start >> Task Manager >> More details >> Performance (tab) >> Ethernet). The DiskSpd script will complete in 120 seconds. After the script has completed, the output will be displayed in the PowerShell window. + * What was the P99 (99th %-tile) latency in ms of your test? - This is found in the DiskSpd output. It is in the *total* table at the bottom. * What was the P99.99 (99.99th %-tile) latency in ms of your test? - This is found in the DiskSpd output. It is in the *total* table at the bottom. * What was the Total Read IO MiB/s? - This is found in the DiskSpd output. It is under *Read IO* under the *MiB/s* column. * What was the I/O per second? - This is found in the DiskSpd output. It is under *Read IO* under the *I/O per s* column. * What was the AvgLat? - This is found in the DiskSpd output. It is under *Read IO* under the *AvgLat* column. + . Record the results in the *FSx File Gateway* Test 1 column of the table you copied to your local computer earlier. . Repeat the test and record the results in the *FSx File Gateway* Test 2 column. + NOTE: The initial read from the FSx File Gateway will experience higher latencies as the FSx File Gateway reads the file into its cache from the remote FSx for Windows File Server. Subsequent reads deliver the file directly from the local FSx File Gateway. + . Experiment with different DiskSpd parameter settings. Use the table below as a guide. Test with different block sizes (-b), number of outstanding I/O requests (-o), number of threads per file (-t), and disable local caching (-Sr). + [cols="3,10"] |=== | Parameter | Description | `-b[K\|M\|G]` a| Block size in bytes or KiB, MiB, or GiB (default = 64K). | `-o` a| Number of outstanding I/O requests per-target per-thread. (1 = synchronous I/O, unless more than one thread is specified with by using `-F`.) (default = 2) | `-r[K\|M\|G]` a| Random I/O aligned to the specified number of bytes or KiB, MiB, GiB, or blocks. Overrides -s (default stride = block size). | `-s[K\|M\|G]` a| Sequential stride size, offset between subsequent I/O operations in bytes or KiB, MiB, GiB, or blocks. Ignored if -r is specified (default access = sequential, default stride = block size). | `-t` a| Number of threads per target. Conflicts with `-F`, which specifies the total number of threads. | `-Sr` a| Disable local caching. |=== + * What different parameters did you test? * How did the different parameter options alter the results? === DiskSpd Read Results The following table and charts show the results from a previous test. These results show a significant improvement when an Amazon EC2 Windows instance (us-west-1) reads a large file from the local FSx File Gateway (us-west-1) compared to a remote FSx for Windows File Server (us-east-1). Remember, the initial read from the FSx File Gateway will experience higher latencies as the FSx File Gateway reads the file into its cache from the remote FSx for Windows File Server. Subsequent reads deliver the file directly from the local FSx File Gateway. * Compare your test results with those in following. Do they differ substantially? Why? image::read-results-table.png[title="DiskSpd Read Results Table - sample", align="left", width=1000] --- image::read-results-throughput-chart.png[title="DiskSpd Read Throughput Chart - sample", align="left", width=800] --- image::read-results-latencies-chart.png[title="DiskSpd Read Latencies Chart - sample", align="left", width=800] --- == Next section Click the button below to go to the next section. image::tear-down-workshop.png[link=../09-tear-down-workshop/, align="left",width=420]