{ "cells": [ { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "# TensorFlow Keras MNIST Classifier - Local Example\n", "\n", "_**Train and export a TF.Keras CNN classifier for (a subset of) the [MNIST DIGITS](https://en.wikipedia.org/wiki/MNIST_database) dataset: Performing all storage and computation locally on the notebook.**_\n", "\n", "This notebook works well with the `Python 3 (TensorFlow 2.3 Python 3.7 CPU Optimized)` kernel on SageMaker Studio, or `conda_tensorflow2_p37` on classic SageMaker Notebook Instances.\n", "\n", "---\n", "\n", "The [dataset](https://s3.amazonaws.com/fast-ai-imageclas/mnist_png.tgz) is hosted in the [Registry of Open Data on AWS](https://registry.opendata.aws/fast-ai-imageclas/) and contains PNG images organized in folders by which digit they represent.\n", "\n", ">❓*Can you figure out how to re-create this notebook's workflow using SageMaker more effectively?*\n", "\n", "## Contents\n", "\n", "1. **[Notebook Setup](#Notebook-Setup)**\n", "1. **[Prepare the Data](#Prepare-the-Data)**\n", "1. **[Load the Data From File](#Load-the-Data-From-File)**\n", "1. **[Pre-Process the Data for our CNN](#Pre-Process-the-Data-for-our-CNN)**\n", "1. **[Build a Model](#Build-a-Model)**\n", "1. **[Fit the Model](#Fit-the-Model)**\n", "1. **[Save the Trained Model](#Save-the-Trained-Model)**\n", "1. **[Explore Results](#Explore-Results)**\n", "\n", "See the accompanying **Instructions** notebook for more guidance!" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## Notebook Setup\n", "\n", "As usual, we'll start by installing any extra required libraries and importing dependencies.\n", "\n", "> ℹ️ **Troubleshooting install issues** if you run in to `ModuleNotFoundError` below, or the interactive widgets don't render for you:\n", ">\n", "> - Check you ran the below `!pip install` commands, and restarted the notebook kernel before running other code cells.\n", "> - If a module looks like it was installed successfully but is missing from the notebook kernel environment, you may need to `%pip install` instead to install them in the right place.\n", "> - You may need to restart your notebook kernel after installing pip libraries - Especially if you already `import`ed affected modules or are running on a SageMaker Notebook Instance instead of Studio.\n", "> - Check you have the **`@jupyter-widgets/jupyterlab-manager`** and **`ipycanvas`** JupyterLab widgets installed (Use the puzzle piece icon \"Extension Manager\" sidebar tab, or click *Settings > Enable Extension Manager* if you don't see it). \"Rebuild\" JupyterLab if prompted, then save your work and refresh the page once the build is complete. In Studio, you may also need to open a system terminal and run `restart-jupyter-server`.\n", ">\n", "> In practice (and in this workshop's CloudFormation templates), JupyterLab extensions are typically installed via **lifecycle configuration scripts** for [Studio](https://aws.amazon.com/blogs/machine-learning/customize-amazon-sagemaker-studio-using-lifecycle-configurations/) or [Notebook Instances](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html) - rather than manual installs by users." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "# First install some libraries which might not be available across all kernels:\n", "!pip install \"ipycanvas<0.13\" \"ipywidgets<8\" matplotlib" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> ⚠️ ***RESTART your notebook kernel*** before continuing! (Circular arrow button in the toolbar)\n", "\n", "If you don't restart the kernel after the install, the interactive drawing widget might not work later." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2\n", "\n", "# Python Built-Ins:\n", "import glob\n", "import os\n", "\n", "# External Dependencies:\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import tensorflow as tf\n", "from tensorflow.keras import backend as K\n", "from tensorflow.keras.layers import Conv2D, Dense, Dropout, Flatten, MaxPooling2D\n", "from tensorflow.keras.models import Sequential\n", "\n", "# Local Notebook Utils:\n", "import util\n", "\n", "%matplotlib inline\n", "\n", "print(f\"Using TensorFlow version {tf.__version__}\")\n", "print(f\"Keras version {tf.keras.__version__}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prepare the Data\n", "\n", "Now let's download the image data.\n", "\n", "The original MNIST data has 70,000 small 28x28 pixel PNG files (60,000 in the training dataset, and 10,000 in the test dataset). This format is nice and familiar - but a large number of tiny files is inefficient for storage and transfer - so **to keep things performant** we will:\n", "\n", "- Download the data to a local temporary folder under `/tmp` (meaning you won't see the files in the left sidebar in SageMaker)\n", "- Sample just a subset of the data to work with." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "local_dir = \"/tmp/mnist\"\n", "training_dir = f\"{local_dir}/training\"\n", "testing_dir = f\"{local_dir}/testing\"\n", "\n", "# Download the MNIST data from the Registry of Open Data on AWS\n", "!rm -rf {local_dir}\n", "!mkdir -p {local_dir}\n", "!aws s3 cp s3://fast-ai-imageclas/mnist_png.tgz {local_dir} --no-sign-request\n", "\n", "# Un-tar the MNIST data, stripping the leading path element; this will leave us with directories\n", "# {local_dir}/testing/ and {local_dir/training/\n", "!tar zxf {local_dir}/mnist_png.tgz -C {local_dir}/ --strip-components=1 --no-same-owner\n", "\n", "# Get the list of files in the training and testing directories recursively\n", "train_files = sorted(list(glob.iglob(os.path.join(training_dir, \"*/*.png\"), recursive=True)))\n", "test_files = sorted(list(glob.iglob(os.path.join(testing_dir, \"*/*.png\"), recursive=True)))\n", "\n", "print(f\"Training files: {len(train_files)}\")\n", "print(f\"Testing files: {len(test_files)}\")\n", "\n", "# Reduce the data by keeping every Nth file and dropping the rest of the files.\n", "reduction_factor = 2\n", "train_files_to_keep = train_files[::reduction_factor]\n", "test_files_to_keep = test_files[::reduction_factor]\n", "\n", "print(f\"Training files kept: {len(train_files_to_keep)}\")\n", "print(f\"Testing files kept: {len(test_files_to_keep)}\")\n", "\n", "# Delete all the files not to be kept\n", "for fname in set(train_files) ^ set(train_files_to_keep):\n", " os.remove(fname)\n", "\n", "for fname in set(test_files) ^ set(test_files_to_keep):\n", " os.remove(fname)\n", "\n", "print(\"Done!\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load the Data From File\n", "\n", "Now our images are stored in the `{local_dir}` folder, let's read our training and testing sets in from these files.\n", "\n", "```\n", " {local_dir}\n", " |----------------.\n", " `-- testing `-- training\n", " |-- 0 |-- 0\n", " | `-- 1.png\n", " |-- 1 |-- 1\n", " |-- 2 |-- 2\n", " |-- 3 |-- 3\n", " |-- 4 |-- 4\n", " |-- 5 |-- 5\n", " |-- 6 |-- 6\n", " |-- 7 |-- 7\n", " |-- 8 |-- 8\n", " `-- 9 `-- 9\n", "```\n", "\n", "(For both training and testing) We'll loop through each folder taking the target label (`0`-`9`) from the folder name and loading each PNG into an image matrix." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from PIL import Image\n", "\n", "labels = sorted(os.listdir(training_dir))\n", "n_labels = len(labels)\n", "\n", "x_train = []\n", "y_train = []\n", "x_test = []\n", "y_test = []\n", "print(\"Loading label \", end=\"\")\n", "for ix_label in range(n_labels):\n", " label_str = labels[ix_label]\n", " print(f\"{label_str}...\", end=\"\")\n", " trainfiles = filter(\n", " lambda s: s.endswith(\".png\"),\n", " os.listdir(os.path.join(training_dir, label_str)),\n", " )\n", "\n", " for filename in trainfiles:\n", " # Can't just use tf.keras.preprocessing.image.load_img(), because it doesn't close its file\n", " # handles! So get \"Too many open files\" error... Grr\n", " with open(os.path.join(training_dir, label_str, filename), \"rb\") as imgfile:\n", " x_train.append(\n", " # Squeeze (drop) that extra channel dimension, to be consistent with prev format:\n", " np.squeeze(tf.keras.preprocessing.image.img_to_array(Image.open(imgfile)))\n", " )\n", " y_train.append(ix_label)\n", "\n", " # Repeat for test data:\n", " testfiles = filter(\n", " lambda s: s.endswith(\".png\"),\n", " os.listdir(os.path.join(testing_dir, label_str)),\n", " )\n", "\n", " for filename in testfiles:\n", " with open(os.path.join(testing_dir, label_str, filename), \"rb\") as imgfile:\n", " x_test.append(\n", " np.squeeze(tf.keras.preprocessing.image.img_to_array(Image.open(imgfile)))\n", " )\n", " y_test.append(ix_label)\n", "print()\n", "\n", "\n", "print(\"Shuffling trainset...\")\n", "train_shuffled = [(x_train[ix], y_train[ix]) for ix in range(len(y_train))]\n", "np.random.shuffle(train_shuffled)\n", "\n", "x_train = np.array([datum[0] for datum in train_shuffled])\n", "y_train = np.array([datum[1] for datum in train_shuffled])\n", "train_shuffled = None\n", "\n", "print(\"Shuffling testset...\")\n", "test_shuffled = [(x_test[ix], y_test[ix]) for ix in range(len(y_test))]\n", "np.random.shuffle(test_shuffled)\n", "\n", "x_test = np.array([datum[0] for datum in test_shuffled])\n", "y_test = np.array([datum[1] for datum in test_shuffled])\n", "test_shuffled = None\n", "\n", "print(\"Done!\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Before we go ahead**, let's just quickly visualize the data distribution." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(f\"x_train.shape {x_train.shape}; dtype {x_train.dtype}\")\n", "print(f\"y_train.shape {y_train.shape}; dtype {y_train.dtype}\")\n", "print(f\"x_test.shape {x_test.shape}; dtype {x_test.dtype}\")\n", "print(f\"y_test.shape {y_test.shape}; dtype {y_test.dtype}\")\n", "\n", "fig = plt.figure(figsize=(14, 3))\n", "ax = plt.subplot(1, 2, 1)\n", "plt.hist(x_train.flatten())\n", "ax.set_title(\"Histogram of Training Image Data\")\n", "ax.set_ylabel(\"Frequency in Training Set\")\n", "ax.set_xlabel(\"Pixel Value\")\n", "\n", "ax = plt.subplot(1, 2, 2)\n", "plt.hist(y_train)\n", "ax.set_title(\"Histogram of Training Set Labels\")\n", "ax.set_ylabel(\"Frequency in Training Set\")\n", "ax.set_xlabel(\"Y Label Value\")\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It looks like the data is pretty evenly distributed between labels 0-9, and our images are encoded by fixed-size 28x28 matrices from 0 to 255. Here we will just plot a few examples to get a feel for them:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(\"Some example images:\")\n", "fig = plt.figure(figsize=(14, 2))\n", "for i in range(5):\n", " fig = plt.subplot(1, 5, i + 1)\n", " ax = plt.imshow(x_train[i], cmap=\"gray\")\n", " fig.set_title(f\"Number {y_train[i]}\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Pre-Process the Data for our CNN\n", "\n", "Next, we'll tweak this format for our neural network:\n", "\n", "- Normalizing pixel values to improve the numerical conditioning\n", "- One-hot encoding our labels to suit a softmax classifier output of probabilities for each digit\n", "- Adding both a batch dimension (for processing multiple samples in parallel) and a channel dimension (e.g. as if this were a 3-channel RGB image, except single-channel for grayscale) - as well as the X and Y axes." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Since we're actually feeding the images in to nets this time, we should actually pay attention\n", "# to which way around Keras wants the channel dimension:\n", "if K.image_data_format() == \"channels_first\":\n", " x_train = np.expand_dims(x_train, 1)\n", " x_test = np.expand_dims(x_train, 1)\n", "else:\n", " x_train = np.expand_dims(x_train, len(x_train.shape))\n", " x_test = np.expand_dims(x_test, len(x_test.shape))\n", "\n", "x_train = x_train.astype(\"float32\")\n", "x_test = x_test.astype(\"float32\")\n", "x_train /= 255\n", "x_test /= 255\n", "\n", "input_shape = x_train.shape[1:]\n", "\n", "print(\"x_train shape:\", x_train.shape)\n", "print(\"input_shape:\", input_shape)\n", "print(x_train.shape[0], \"train samples\")\n", "print(x_test.shape[0], \"test samples\")\n", "\n", "# convert class vectors to binary class matrices\n", "y_train = tf.keras.utils.to_categorical(y_train, n_labels)\n", "y_test = tf.keras.utils.to_categorical(y_test, n_labels)\n", "\n", "print(\"n_labels:\", n_labels)\n", "print(\"y_train shape:\", y_train.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Build a Model\n", "\n", "At its core, the model is a 2D convolutional network with a softmax output layer that'll yield a confidence score for every possible label (e.g. 10 options for digit = 0 to 9).\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "model = Sequential()\n", "model.add(Conv2D(32, kernel_size=(3, 3), activation=\"relu\", input_shape=input_shape))\n", "model.add(Conv2D(64, (3, 3), activation=\"relu\"))\n", "model.add(MaxPooling2D(pool_size=(2, 2)))\n", "model.add(Dropout(0.25))\n", "model.add(Flatten())\n", "model.add(Dense(128, activation=\"relu\"))\n", "model.add(Dropout(0.5))\n", "model.add(Dense(n_labels, activation=\"softmax\"))\n", "\n", "model.compile(\n", " loss=tf.keras.losses.categorical_crossentropy,\n", " optimizer=tf.keras.optimizers.Adadelta(),\n", " metrics=[\"accuracy\"],\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fit the Model\n", "\n", "Keras makes fitting and evaluating the model straightforward enough: We don't have any fancy hooks, and are happy with the default logging:\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%time\n", "batch_size = 128\n", "epochs = 12\n", "\n", "model.fit(\n", " x_train,\n", " y_train,\n", " batch_size=batch_size,\n", " epochs=epochs,\n", " shuffle=True,\n", " verbose=1, # Hint: You might prefer =2 for running in SageMaker!\n", " validation_data=(x_test, y_test),\n", ")\n", "\n", "score = model.evaluate(x_test, y_test, verbose=0)\n", "print(f\"Test loss={score[0]}\")\n", "print(f\"Test accuracy={score[1]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Save the Trained Model\n", "\n", "Keras has a built-in `model.save()` command, which in TensorFlow v2 can directly produce TensorFlow Serving-compatible outputs!\n", "\n", "...However, this notebook runs TensorFlow v1. To save you the frustration of figuring it out (there's a nice blog post on the subject [here](https://aws.amazon.com/blogs/machine-learning/deploy-trained-keras-or-tensorflow-models-using-amazon-sagemaker/)), we'll give you a hint by saving the model here in TensorFlow Serving-ready format.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# The export folder needs to be empty, or non-existent\n", "!rm -rf data/model/model/1\n", "\n", "# Please ignore Layer.updates(...) warning if any while running the notebook in < TFv2.4\n", "model.save(os.path.join(\"data/model\", \"model/1\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Explore Results\n", "\n", "To try out the model we can take a sample image from the test set, predict the label, and plot it:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Choose an image:\n", "label = \"2\"\n", "filename = os.listdir(f\"{testing_dir}/{label}\")[0]\n", "\n", "# Load the image (and normalize to 0-1):\n", "img = tf.keras.preprocessing.image.img_to_array(\n", " Image.open(f\"{testing_dir}/{label}/{filename}\")\n", ") / 255\n", "\n", "# Expand out the \"batch\" dimension, and send to the model:\n", "result = model.predict(np.expand_dims(img, 0))\n", "print(f\"Result confidences: {result}\")\n", "\n", "# Plot the result:\n", "plt.figure(figsize=(3, 3))\n", "fig = plt.subplot(1, 1, 1)\n", "ax = plt.imshow(np.squeeze(img), cmap=\"gray\")\n", "fig.set_title(f\"Predicted Number {np.argmax(result[0])}\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "...Or even better, draw your own digit using this interactive widget - then run the cell below to classify it!" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Create/reload the widget:\n", "widget = util.draw.PixelDrawCanvas(pen_size=4)\n", "widget.display()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Classify the image currently drawn in the widget:\n", "img = widget.data.mean(-1, keepdims=True) / 255 # (Convert full-RGB 0-255 to grayscale 0-1)\n", "result = model.predict(np.expand_dims(img, 0))\n", "print(f\"Result confidences: {result}\")\n", "\n", "# Plot the result:\n", "plt.figure(figsize=(3, 3))\n", "fig = plt.subplot(1, 1, 1)\n", "ax = plt.imshow(img, cmap=\"gray\")\n", "fig.set_title(f\"Predicted Number {np.argmax(result[0])}\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "All done!\n" ] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 } ], "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (TensorFlow 2.3 Python 3.7 CPU Optimized)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/tensorflow-2.3-cpu-py37-ubuntu18.04-v1" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 4 }