{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# PyTorch MNIST Classifier - Local Example\n",
"\n",
"_**Train and export a PyTorch CNN classifier for (a subset of) the [MNIST DIGITS](https://en.wikipedia.org/wiki/MNIST_database) dataset: Performing all storage and computation locally on the notebook.**_\n",
"\n",
"This notebook works well with the `Python 3 (PyTorch 1.13 Python 3.9 CPU Optimized)` kernel on SageMaker Studio, or `conda_pytorch_p38` on classic SageMaker Notebook Instances.\n",
"\n",
"---\n",
"\n",
"The [dataset](https://s3.amazonaws.com/fast-ai-imageclas/mnist_png.tgz) is hosted in the [Registry of Open Data on AWS](https://registry.opendata.aws/fast-ai-imageclas/) and contains PNG images organized in folders by which digit they represent.\n",
"\n",
">â“*Can you figure out how to re-create this notebook's workflow using SageMaker more effectively?*\n",
"\n",
"## Contents\n",
"\n",
"1. **[Notebook Setup](#Notebook-Setup)**\n",
"1. **[Prepare the Data](#Prepare-the-Data)**\n",
"1. **[Load the Data From File](#Load-the-Data-From-File)**\n",
"1. **[Pre-Process the Data for our CNN](#Pre-Process-the-Data-for-our-CNN)**\n",
"1. **[Build a Model](#Build-a-Model)**\n",
"1. **[Fit the Model](#Fit-the-Model)**\n",
"1. **[Save the Trained Model](#Save-the-Trained-Model)**\n",
"1. **[Explore Results](#Explore-Results)**\n",
"\n",
"See the accompanying **Instructions** notebook for more guidance!\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notebook Setup\n",
"\n",
"As usual, we'll start by installing any extra required libraries and importing dependencies.\n",
"\n",
"> â„¹ï¸ **Troubleshooting install issues** if you run in to `ModuleNotFoundError` below, or the interactive widgets don't render for you:\n",
">\n",
"> - Check you ran the below `!pip install` commands, and restarted the notebook kernel before running other code cells.\n",
"> - If a module looks like it was installed successfully but is missing from the notebook kernel environment, you may need to `%pip install` instead to install them in the right place.\n",
"> - You may need to restart your notebook kernel after installing pip libraries - Especially if you already `import`ed affected modules or are running on a SageMaker Notebook Instance instead of Studio.\n",
"> - Check you have the **`@jupyter-widgets/jupyterlab-manager`** and **`ipycanvas`** JupyterLab widgets installed (Use the puzzle piece icon \"Extension Manager\" sidebar tab, or click *Settings > Enable Extension Manager* if you don't see it). \"Rebuild\" JupyterLab if prompted, then save your work and refresh the page once the build is complete. In Studio, you may also need to open a system terminal and run `restart-jupyter-server`.\n",
">\n",
"> In practice (and in this workshop's CloudFormation templates), JupyterLab extensions are typically installed via **lifecycle configuration scripts** for [Studio](https://aws.amazon.com/blogs/machine-learning/customize-amazon-sagemaker-studio-using-lifecycle-configurations/) or [Notebook Instances](https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html) - rather than manual installs by users."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true,
"tags": []
},
"outputs": [],
"source": [
"# First install some libraries which might not be available across all kernels (e.g. in Studio):\n",
"!pip install \"ipycanvas<0.13\" \"ipywidgets<8\" matplotlib"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> âš ï¸ ***RESTART your notebook kernel*** before continuing! (Circular arrow button in the toolbar)\n",
"\n",
"If you don't restart the kernel after the install, the interactive drawing widget might not work later."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%load_ext autoreload\n",
"%autoreload 2\n",
"\n",
"# Python Built-Ins:\n",
"import glob\n",
"import os\n",
"\n",
"# External Dependencies:\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import torch\n",
"import torch.nn as nn\n",
"import torch.nn.functional as F\n",
"import torch.optim as optim\n",
"import torchvision\n",
"\n",
"# Local Notebook Utils:\n",
"import util\n",
"\n",
"%matplotlib inline\n",
"\n",
"print(f\"Using PyTorch version {torch.__version__}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prepare the Data\n",
"\n",
"Now let's download the image data.\n",
"\n",
"The original MNIST data has 70,000 small 28x28 pixel PNG files (60,000 in the training dataset, and 10,000 in the test dataset). This format is nice and familiar - but a large number of tiny files is inefficient for storage and transfer - so **to keep things performant** we will:\n",
"\n",
"- Download the data to a local temporary folder under `/tmp` (meaning you won't see the files in the left sidebar in SageMaker)\n",
"- Sample just a subset of the data to work with."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"local_dir = \"/tmp/mnist\"\n",
"training_dir = f\"{local_dir}/training\"\n",
"testing_dir = f\"{local_dir}/testing\"\n",
"\n",
"# Download the MNIST data from the Registry of Open Data on AWS\n",
"!rm -rf {local_dir}\n",
"!mkdir -p {local_dir}\n",
"!aws s3 cp s3://fast-ai-imageclas/mnist_png.tgz {local_dir} --no-sign-request\n",
"\n",
"# Un-tar the MNIST data, stripping the leading path element; this will leave us with directories\n",
"# {local_dir}/testing/ and {local_dir/training/\n",
"!tar zxf {local_dir}/mnist_png.tgz -C {local_dir}/ --strip-components=1 --no-same-owner\n",
"\n",
"# Get the list of files in the training and testing directories recursively\n",
"train_files = sorted(list(glob.iglob(os.path.join(training_dir, \"*/*.png\"), recursive=True)))\n",
"test_files = sorted(list(glob.iglob(os.path.join(testing_dir, \"*/*.png\"), recursive=True)))\n",
"\n",
"print(f\"Training files: {len(train_files)}\")\n",
"print(f\"Testing files: {len(test_files)}\")\n",
"\n",
"# Reduce the data by keeping every Nth file and dropping the rest of the files.\n",
"reduction_factor = 2\n",
"train_files_to_keep = train_files[::reduction_factor]\n",
"test_files_to_keep = test_files[::reduction_factor]\n",
"\n",
"print(f\"Training files kept: {len(train_files_to_keep)}\")\n",
"print(f\"Testing files kept: {len(test_files_to_keep)}\")\n",
"\n",
"# Delete all the files not to be kept\n",
"for fname in set(train_files) ^ set(train_files_to_keep):\n",
" os.remove(fname)\n",
"\n",
"for fname in set(test_files) ^ set(test_files_to_keep):\n",
" os.remove(fname)\n",
"\n",
"print(\"Done!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load the Data From File\n",
"\n",
"Now our images are stored in the `{local_dir}` folder, let's read our training and testing sets in from these files.\n",
"\n",
"```\n",
" {local_dir}\n",
" |----------------.\n",
" `-- testing `-- training\n",
" |-- 0 |-- 0\n",
" | `-- 1.png\n",
" |-- 1 |-- 1\n",
" |-- 2 |-- 2\n",
" |-- 3 |-- 3\n",
" |-- 4 |-- 4\n",
" |-- 5 |-- 5\n",
" |-- 6 |-- 6\n",
" |-- 7 |-- 7\n",
" |-- 8 |-- 8\n",
" `-- 9 `-- 9\n",
"```\n",
"\n",
"(For both training and testing) We'll loop through each folder taking the target label (`0`-`9`) from the folder name and loading each PNG into an image matrix."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from PIL import Image\n",
"\n",
"labels = sorted(os.listdir(training_dir))\n",
"n_labels = len(labels)\n",
"\n",
"x_train = []\n",
"y_train = []\n",
"x_test = []\n",
"y_test = []\n",
"print(\"Loading label \", end=\"\")\n",
"for ix_label in range(n_labels):\n",
" label_str = labels[ix_label]\n",
" print(f\"{label_str}...\", end=\"\")\n",
" trainfiles = filter(\n",
" lambda s: s.endswith(\".png\"),\n",
" os.listdir(os.path.join(training_dir, label_str)),\n",
" )\n",
" for filename in trainfiles:\n",
" with open(os.path.join(training_dir, label_str, filename), \"rb\") as imgfile:\n",
" x_train.append(np.squeeze(np.asarray(Image.open(imgfile))))\n",
" y_train.append(ix_label)\n",
" # Repeat for test data:\n",
" testfiles = filter(\n",
" lambda s: s.endswith(\".png\"),\n",
" os.listdir(os.path.join(testing_dir, label_str)),\n",
" )\n",
" for filename in testfiles:\n",
" with open(os.path.join(testing_dir, label_str, filename), \"rb\") as imgfile:\n",
" x_test.append(np.squeeze(np.asarray(Image.open(imgfile))))\n",
" y_test.append(ix_label)\n",
"print()\n",
"\n",
"print(\"Shuffling trainset...\")\n",
"train_shuffled = [(x_train[ix], y_train[ix]) for ix in range(len(y_train))]\n",
"np.random.shuffle(train_shuffled)\n",
"\n",
"x_train = np.array([datum[0] for datum in train_shuffled])\n",
"y_train = np.array([datum[1] for datum in train_shuffled])\n",
"train_shuffled = None\n",
"\n",
"print(\"Shuffling testset...\")\n",
"test_shuffled = [(x_test[ix], y_test[ix]) for ix in range(len(y_test))]\n",
"np.random.shuffle(test_shuffled)\n",
"\n",
"x_test = np.array([datum[0] for datum in test_shuffled])\n",
"y_test = np.array([datum[1] for datum in test_shuffled])\n",
"test_shuffled = None\n",
"\n",
"print(\"Done!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Before we go ahead**, let's just quickly visualize the data distribution."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"print(f\"x_train.shape {x_train.shape}; dtype {x_train.dtype}\")\n",
"print(f\"y_train.shape {y_train.shape}; dtype {y_train.dtype}\")\n",
"print(f\"x_test.shape {x_test.shape}; dtype {x_test.dtype}\")\n",
"print(f\"y_test.shape {y_test.shape}; dtype {y_test.dtype}\")\n",
"\n",
"fig = plt.figure(figsize=(14, 3))\n",
"ax = plt.subplot(1, 2, 1)\n",
"plt.hist(x_train.flatten())\n",
"ax.set_title(\"Histogram of Training Image Data\")\n",
"ax.set_ylabel(\"Frequency in Training Set\")\n",
"ax.set_xlabel(\"Pixel Value\")\n",
"\n",
"ax = plt.subplot(1, 2, 2)\n",
"plt.hist(y_train)\n",
"ax.set_title(\"Histogram of Training Set Labels\")\n",
"ax.set_ylabel(\"Frequency in Training Set\")\n",
"ax.set_xlabel(\"Y Label Value\")\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It looks like the data is pretty evenly distributed between labels 0-9, and our images are encoded by fixed-size 28x28 matrices from 0 to 255. Here we will just plot a few examples to get a feel for them:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"print(\"Some example images:\")\n",
"fig = plt.figure(figsize=(14, 2))\n",
"for i in range(5):\n",
" fig = plt.subplot(1, 5, i + 1)\n",
" ax = plt.imshow(x_train[i], cmap=\"gray\")\n",
" fig.set_title(f\"Number {y_train[i]}\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Pre-Process the Data for our CNN\n",
"\n",
"Next, we'll tweak this format for our neural network:\n",
"\n",
"- Normalizing pixel values to improve the numerical conditioning\n",
"- One-hot encoding our labels to suit a softmax classifier output of probabilities for each digit\n",
"- Adding both a batch dimension (for processing multiple samples in parallel) and a channel dimension (e.g. as if this were a 3-channel RGB image, except single-channel for grayscale) - as well as the X and Y axes.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"x_train = np.expand_dims(x_train, 1)\n",
"x_test = np.expand_dims(x_test, 1)\n",
"\n",
"x_train = x_train.astype(\"float32\")\n",
"x_test = x_test.astype(\"float32\")\n",
"x_train /= 255\n",
"x_test /= 255\n",
"\n",
"input_shape = x_train.shape[1:]\n",
"\n",
"print(\"x_train shape:\", x_train.shape)\n",
"print(\"input_shape:\", input_shape)\n",
"print(x_train.shape[0], \"train samples\")\n",
"print(x_test.shape[0], \"test samples\")\n",
"\n",
"\n",
"def to_categorical(y, num_classes):\n",
" \"\"\"1-hot encodes a tensor\"\"\"\n",
" return np.eye(num_classes, dtype=\"float32\")[y]\n",
"\n",
"\n",
"# convert class vectors to binary class matrices\n",
"y_train = to_categorical(y_train, n_labels)\n",
"y_test = to_categorical(y_test, n_labels)\n",
"\n",
"print(\"n_labels:\", n_labels)\n",
"print(\"y_train shape:\", y_train.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Build a Model\n",
"\n",
"At its core, the model is a 2D convolutional network with a softmax output layer that'll yield a confidence score for every possible label (e.g. 10 options for digit = 0 to 9).\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"class Net(nn.Module):\n",
" def __init__(self, num_classes):\n",
" super(Net, self).__init__()\n",
" self.conv1 = nn.Conv2d(in_channels=1, out_channels=32, kernel_size=(3, 3))\n",
" self.conv2 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=(3, 3))\n",
" self.max_pool2d = nn.MaxPool2d(kernel_size=(2, 2))\n",
" self.dropout1 = nn.Dropout2d(p=0.25)\n",
" self.flatten1 = nn.Flatten()\n",
" self.fc1 = nn.Linear(9216, 128)\n",
" self.dropout2 = nn.Dropout(p=0.5)\n",
" self.fc2 = nn.Linear(128, num_classes)\n",
"\n",
" def forward(self, x):\n",
" x = F.relu(self.conv1(x))\n",
" x = F.relu(self.conv2(x))\n",
" x = self.flatten1(self.dropout1(self.max_pool2d(x)))\n",
" x = F.relu(self.fc1(x))\n",
" x = self.fc2(self.dropout2(x))\n",
" return F.softmax(x, dim=-1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Define training and evaluation script here."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"def test(model, testloader, device):\n",
" loss_function = F.binary_cross_entropy\n",
" model.eval()\n",
" test_loss = 0.0\n",
" correct = 0\n",
" with torch.no_grad():\n",
" for data, target in testloader:\n",
" data, target = data.to(device), target.to(device)\n",
" output = model(data)\n",
" test_loss += loss_function(output, target, reduction=\"mean\").item() # sum up batch loss\n",
" pred = output.max(1, keepdim=True)[1] # get the index of the max log-probability\n",
" target_index = target.max(1, keepdim=True)[1]\n",
" correct += pred.eq(target_index).sum().item()\n",
"\n",
" test_loss /= len(testloader.dataset)\n",
" print(\"val_loss: {:.4f}\".format(test_loss))\n",
" print(\"val_acc: {:.4f}\".format(correct / len(testloader.dataset)))\n",
"\n",
"\n",
"def train(trainloader, testloader, epochs, num_classes):\n",
" model = Net(num_classes)\n",
" device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
" model.to(device)\n",
" optimizer = torch.optim.Adadelta(model.parameters())\n",
" loss_function = F.binary_cross_entropy\n",
"\n",
" for epoch in range(1, epochs + 1):\n",
" model.train()\n",
" running_loss = 0.0\n",
" for batch_idx, (x_train, y_train) in enumerate(trainloader):\n",
" data, target = x_train.to(device), y_train.to(device)\n",
" optimizer.zero_grad()\n",
" output = model(data)\n",
" loss = loss_function(output, target)\n",
" loss.backward()\n",
" optimizer.step()\n",
" running_loss += loss.item()\n",
" print(\"epoch: {}\".format(epoch))\n",
" print(\"train_loss: {:.6f}\".format(running_loss / (len(trainloader.dataset))))\n",
" print(\"Evaluating model\")\n",
" test(model, testloader, device)\n",
" return model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The custom Dataset class below is to allow data loading."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"class Dataset(torch.utils.data.Dataset):\n",
" def __init__(self, data, labels):\n",
" \"\"\"Initialization\"\"\"\n",
" self.labels = labels\n",
" self.data = data\n",
"\n",
" def __len__(self):\n",
" \"\"\"Denotes the total number of samples\"\"\"\n",
" return len(self.data)\n",
"\n",
" def __getitem__(self, index):\n",
" # Load data and get label\n",
" X = self.data[index]\n",
" y = self.labels[index]\n",
" return X, y"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Fit the Model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%%time\n",
"batch_size = 128\n",
"epochs = 6\n",
"\n",
"trainloader = torch.utils.data.DataLoader(\n",
" Dataset(x_train, y_train),\n",
" batch_size=batch_size,\n",
" shuffle=True,\n",
")\n",
"testloader = torch.utils.data.DataLoader(\n",
" Dataset(x_test, y_test),\n",
" batch_size=1,\n",
" shuffle=True,\n",
")\n",
"\n",
"model = train(trainloader, testloader, epochs=epochs, num_classes=10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Save the Trained Model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Torch JIT model allows us to store the inference script along with the model weights."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"path = \"./data/model/model.pth\"\n",
"# Ensure the subfolder exists:\n",
"os.makedirs(path.rpartition(\"/\")[0], exist_ok=True)\n",
"\n",
"x = torch.rand((1, 1, 28, 28), dtype=torch.float)\n",
"model = model.cpu()\n",
"model.eval()\n",
"m = torch.jit.trace(model, x)\n",
"torch.jit.save(m, path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Explore Results\n",
"\n",
"To try out the model we can take a sample image from the test set, predict the label, and plot it:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Choose an image:\n",
"label = \"2\"\n",
"filename = os.listdir(f\"{testing_dir}/{label}\")[0]\n",
"\n",
"# Load the image:\n",
"img = Image.open(f\"{testing_dir}/{label}/{filename}\")\n",
"input_data = np.squeeze(np.asarray(img)).astype(np.float32) / 255\n",
"input_data = torch.tensor(np.expand_dims(input_data, [0, 1])) # Add batch & leading channel dim\n",
"\n",
"# Send to the model:\n",
"with torch.no_grad():\n",
" result = model(input_data)\n",
"print(f\"Result confidences: {result}\")\n",
"\n",
"# Plot the result:\n",
"plt.figure(figsize=(3, 3))\n",
"fig = plt.subplot(1, 1, 1)\n",
"ax = plt.imshow(img, cmap=\"gray\")\n",
"fig.set_title(f\"Predicted Number {np.argmax(result[0])}\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"...Or even better, draw your own digit using this interactive widget - then run the cell below to classify it!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Create/reload the widget:\n",
"widget = util.draw.PixelDrawCanvas(pen_size=4)\n",
"widget.display()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Classify the image currently drawn in the widget:\n",
"img = widget.data.mean(-1) / 255 # (Convert full-RGB 0-255 to grayscale 0-1)\n",
"input_data = torch.FloatTensor(np.expand_dims(img, [0, 1])) # Add batch & leading channel dimension\n",
"\n",
"with torch.no_grad():\n",
" result = model(input_data)\n",
"print(f\"Result confidences: {result}\")\n",
"\n",
"# Plot the result:\n",
"plt.figure(figsize=(3, 3))\n",
"fig = plt.subplot(1, 1, 1)\n",
"ax = plt.imshow(img, cmap=\"gray\")\n",
"fig.set_title(f\"Predicted Number {np.argmax(result[0])}\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"All done!\n"
]
}
],
"metadata": {
"availableInstances": [
{
"_defaultOrder": 0,
"_isFastLaunch": true,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 4,
"name": "ml.t3.medium",
"vcpuNum": 2
},
{
"_defaultOrder": 1,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.t3.large",
"vcpuNum": 2
},
{
"_defaultOrder": 2,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.t3.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 3,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.t3.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 4,
"_isFastLaunch": true,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.m5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 5,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.m5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 6,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.m5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 7,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.m5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 8,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.m5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 9,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.m5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 10,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.m5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 11,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.m5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 12,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.m5d.large",
"vcpuNum": 2
},
{
"_defaultOrder": 13,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.m5d.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 14,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.m5d.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 15,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.m5d.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 16,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.m5d.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 17,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.m5d.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 18,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.m5d.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 19,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.m5d.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 20,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": true,
"memoryGiB": 0,
"name": "ml.geospatial.interactive",
"supportedImageNames": [
"sagemaker-geospatial-v1-0"
],
"vcpuNum": 0
},
{
"_defaultOrder": 21,
"_isFastLaunch": true,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 4,
"name": "ml.c5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 22,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.c5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 23,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.c5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 24,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.c5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 25,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 72,
"name": "ml.c5.9xlarge",
"vcpuNum": 36
},
{
"_defaultOrder": 26,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 96,
"name": "ml.c5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 27,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 144,
"name": "ml.c5.18xlarge",
"vcpuNum": 72
},
{
"_defaultOrder": 28,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.c5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 29,
"_isFastLaunch": true,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.g4dn.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 30,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.g4dn.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 31,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.g4dn.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 32,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.g4dn.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 33,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.g4dn.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 34,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.g4dn.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 35,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 61,
"name": "ml.p3.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 36,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 244,
"name": "ml.p3.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 37,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 488,
"name": "ml.p3.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 38,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.p3dn.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 39,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.r5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 40,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.r5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 41,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.r5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 42,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.r5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 43,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.r5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 44,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.r5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 45,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 512,
"name": "ml.r5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 46,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.r5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 47,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.g5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 48,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.g5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 49,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.g5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 50,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.g5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 51,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.g5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 52,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.g5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 53,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.g5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 54,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.g5.48xlarge",
"vcpuNum": 192
}
],
"instance_type": "ml.t3.medium",
"kernelspec": {
"display_name": "Python 3 (PyTorch 1.13 Python 3.9 CPU Optimized)",
"language": "python",
"name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/pytorch-1.13-cpu-py39"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
}
},
"nbformat": 4,
"nbformat_minor": 4
}