{ "cells": [ { "cell_type": "markdown", "id": "9c32787d-8982-4843-a3ec-7cb13096c924", "metadata": { "tags": [] }, "source": [ "
Note \n", "Set conda_pytorch_p39 kernel when prompted to set the kernel for this notebook\n", "
\n" ] }, { "cell_type": "markdown", "id": "e0815462-7243-4ca2-b0b6-caa1e2855f69", "metadata": {}, "source": [ "# Notebook 2 - Financial document summarization\n", "\n", "## Use case\n", "\n", "Financial news and reports can contain a vast amount of information, which can be challenging to analyze and understand. AI summarization can help investors make more informed decisions by providing them with a quick and concise summary of relevant news and reports. Summarization can help analysts quickly identify trends and patterns in the data and make more accurate predictions in market analysis. summarization can help financial institutions quickly identify potential risks and respond proactively. Common use cases for Financial documents(news, reports etc)\n", "\n", "1. Investment decisions\n", "2. Market analysis\n", "3. Competitive intelligence\n", "4. Risk management\n", "5. Regualtory compliance" ] }, { "cell_type": "markdown", "id": "3ce1c6e6", "metadata": {}, "source": [ "## What will you learn?\n", "\n", "In this notebook, you will learn how to serve an NLP model with your Python inference script, using NVIDIA Triton's Python backend. For demonstration purposes, we we will use a pre-trained T5 model to financial summarize text. T5 ia a multitask encoder-decoder Transformer model that is frequently used for text generation tasks. One of the details of running text generation is that the model's decoder - the component which predicts the next token in the sequence - is autoregressive, i.e. it needs to be called many times to create a single output sequence, on each run adding the last generated token to the input. \n", "1. [Install packages](#installs-and-set-up)\n", " \n", "2. [Creating a deployable Triton Python Model](#t5-transformer-model)
\n", " a. [Python model repository](#create-python-model-repo)
\n", " b. [Model artifacts, dependencies and script](#create-python-model)
\n", " c. [Python model configuration](#create-python-model-config)
\n", " \n", "3. [Export model artifacts to S3](#export-to-s3)\n", "\n", "4. [(Inference using deployed T5 model) Inference using deployed T5 model](#t5-inference)
\n" ] }, { "cell_type": "markdown", "id": "e15ce04d", "metadata": {}, "source": [ "## Install packages \n", "\n", "Install the dependencies required to package the model and run inference using Triton Server. Update SageMaker, boto3, awscli etc" ] }, { "cell_type": "code", "execution_count": null, "id": "b953b5d4", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "!pip install -qU pip awscli boto3 sagemaker transformers ipywidgets\n", "!pip install nvidia-pyindex\n", "!pip install tritonclient[http]" ] }, { "cell_type": "markdown", "id": "6abecdc2", "metadata": {}, "source": [ "#### Imports and variables" ] }, { "cell_type": "code", "execution_count": null, "id": "2e707f08", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "import boto3\n", "import sagemaker\n", "from sagemaker import get_execution_role\n", "import numpy as np\n", "import tritonclient.http as httpclient\n", "\n", "sess = boto3.Session()\n", "sm = sess.client('sagemaker')\n", "sagemaker_session = sagemaker.Session(boto_session=sess)\n", "role = get_execution_role()\n", "client = boto3.client('sagemaker-runtime')\n", "\n", "model_name = \"t5\"\n", "python_model_file_name = f\"{model_name}_py_v0.tar.gz\"\n", "\n", "prefix = \"financial-usecase-mme\"\n", "\n", "# sagemaker variables\n", "role = get_execution_role()\n", "sm_client = boto3.client(service_name=\"sagemaker\")\n", "runtime_sm_client = boto3.client(\"sagemaker-runtime\")\n", "sagemaker_session = sagemaker.Session(boto_session=boto3.Session())\n", "bucket = sagemaker_session.default_bucket()\n", "\n", "%store -r" ] }, { "cell_type": "markdown", "id": "6299b695-6bb2-4294-8bfd-470b4cb1c7a9", "metadata": {}, "source": [ "## Update below cell with the endpoint_name from lab3 notebook 1" ] }, { "cell_type": "code", "execution_count": null, "id": "2e3de81f-d3b5-4d97-b9e3-b2d0ff117965", "metadata": { "tags": [] }, "outputs": [], "source": [ "endpoint_name = ''" ] }, { "cell_type": "markdown", "id": "d836d8de", "metadata": {}, "source": [ "## Creating a deployable Triton Python Model \n", "\n", "This section presents overview of steps to prepare the T5 pre-trained model to be deployed on SageMaker MME using Triton Inference Server model configurations.\n", "\n", "\n", "
Note \n", "We are demonstrating deployment with a single Python backend model. However, you can deploy 100s of models using SageMager MME support for GPUs. The models may or may not share the same backend/framework.\n", "
" ] }, { "cell_type": "markdown", "id": "78727963", "metadata": { "tags": [] }, "source": [ "### Python model repository \n", "\n", "The model repository will contain the model and tokenizer artifacts, a packaged conda environment (with dependencies needed for inference), the Triton config file and the Python script used for inference. The latter is mandatory when you use the Python backend, and you should use the Python file `model.py`.\n", "\n", "```\n", "t5\n", "├── 1\n", "│ └── model.py\n", "├── model\n", "│ └── \n", "├── tokenizer\n", "│ └── \n", "├── config.pbtxt\n", "│\n", "└── mme_env.tar.gz\n", "```" ] }, { "cell_type": "markdown", "id": "a407280a", "metadata": {}, "source": [ "### Model artifacts, dependencies and script \n", "\n", "We will take the pre-trained T5-small model from [HuggingFace](https://huggingface.co/transformers/model_doc/t5.html) and save it to disk. This will exemplify how you can bring your own model parameters and load them in the Python inference script itself." ] }, { "cell_type": "code", "execution_count": null, "id": "32cfa4dd-030d-4ea1-a452-a0735e69c609", "metadata": { "tags": [] }, "outputs": [], "source": [ "model_id = \"t5-small\"\n", "from transformers import AutoTokenizer,T5ForConditionalGeneration\n", "\n", "tokenizer = AutoTokenizer.from_pretrained(model_id)\n", "model = T5ForConditionalGeneration.from_pretrained(model_id)\n", "tokenizer.save_pretrained('triton-serve-py/t5-summarization/tokenizer')\n", "model.save_pretrained('triton-serve-py/t5-summarization/model')" ] }, { "cell_type": "markdown", "id": "a32afa99-a549-4e30-8065-59a18ee8c4f9", "metadata": {}, "source": [ "The Python backend doesn't include other libraries by default; we need PyTorch and the Transformers library to run inference. In order to package the inference environment we use `conda pack`, which is the dependency management method recommended in the [Triton documentation](https://github.com/triton-inference-server/python_backend#creating-custom-execution-environments).\n", "\n", "
Note \n", "We have pre-packaged the conda env and made it available for this lab, installing and packaging dependencies using conda-pack will take 15 mins to run. In the interest of time, we have provided the tar file in triton-serve-py directory\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "29e67cff-a691-4672-a281-bfd6d733c96a", "metadata": { "tags": [] }, "outputs": [], "source": [ "!aws s3 cp s3://ee-assets-prod-us-east-1/modules/05fa7598d4d44836a42fde79b26568b2/v3/mme_env.tar.gz triton-serve-py/t5-summarization/" ] }, { "cell_type": "markdown", "id": "afabd7c3-ee6a-4a74-ac65-0c06e98ba5c5", "metadata": {}, "source": [ "Finally, we write the Python inference scripts (refer to the workshop README for more details on the required structure of the script). Notice that we load the model and tokenizer from the model repository directory. The script receives and returns text, and also supports variable batch dimensions." ] }, { "cell_type": "code", "execution_count": null, "id": "3325a94f", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "!pygmentize triton-serve-py/t5-summarization/1/model.py" ] }, { "cell_type": "markdown", "id": "e907c6f4", "metadata": {}, "source": [ "### Python Model configuration \n", "\n", "The model configuration file `config.pbtxt` must specify the name of the model (`t5-summarization`), the platform and backend properties (`python`), max_batch_size (16) and the input and output shapes along with the data type (TYPE_STRING) information. Additionally, you \n", "can specify `instance_group` and `dynamic_batching` properties to achieve high performance inference." ] }, { "cell_type": "code", "execution_count": null, "id": "b30b1950", "metadata": { "tags": [] }, "outputs": [], "source": [ "%%writefile triton-serve-py/t5-summarization/config.pbtxt\n", "name: \"t5-summarization\"\n", "backend: \"python\"\n", "max_batch_size: 16\n", "\n", "input [\n", " {\n", " name: \"INPUT0\"\n", " data_type: TYPE_STRING\n", " dims: [ 1 ]\n", " \n", " }\n", "]\n", "output [\n", " {\n", " name: \"SUMMARY\"\n", " data_type: TYPE_STRING\n", " dims: [ 1 ]\n", " }\n", "]\n", "\n", "instance_group [\n", " { \n", " count:1\n", " kind: KIND_GPU \n", " }\n", "]\n", "\n", "parameters: {\n", " key: \"EXECUTION_ENV_PATH\",\n", " value: {string_value: \"$$TRITON_MODEL_DIRECTORY/mme_env.tar.gz\"}\n", "}" ] }, { "cell_type": "markdown", "id": "9507e641", "metadata": {}, "source": [ "### Packaging model files and uploading to s3" ] }, { "cell_type": "code", "execution_count": null, "id": "082ccb88", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "!tar -C triton-serve-py/ -czf $python_model_file_name t5-summarization\n", "model_uri_py = sagemaker_session.upload_data(path=python_model_file_name, key_prefix=prefix)" ] }, { "cell_type": "code", "execution_count": null, "id": "e236b346", "metadata": { "tags": [] }, "outputs": [], "source": [ "print(f\"PyTorch Model S3 location: {model_uri_py}\")" ] }, { "cell_type": "markdown", "id": "f125770d-26ff-4f52-ab63-5000b20eb482", "metadata": {}, "source": [ "Note that, this T5 python model is uploaded to the same S3 location where MME is configured model_data_url. You can invoke the MME endpoint created in the previous notebook with TargetModel name(t5_py_v0.tar.gz) to invoke the summarization model " ] }, { "cell_type": "code", "execution_count": null, "id": "82525cd3", "metadata": { "tags": [] }, "outputs": [], "source": [ "text_1 =\"\"\"\n", "March 14 (Reuters) - U.S. prosecutors are investigating the collapse of Silicon Valley Bank, according to a source familiar with the matter, as scrutiny mounts over the firm's sudden collapse and regulators scramble to contain the fallout.\n", "The U.S. Justice Department is probing the sudden demise of the bank, which was shuttered on Friday following a bank run, the source said, declining to be named as the inquiry is not public. The Securities and Exchange Commission has launched a parallel investigation, according to the Wall Street Journal, which first reported the probes.\n", "The investigation is in early stages and may not result in allegations of wrongdoing or charges being filed, the source said. Officials are also examining stock sales by officers of SVB Financial Group (SIVB.O), which owned the bank, the WSJ reported, citing people familiar with the matter.\n", "SEC Chair Gary Gensler on Sunday said in a statement the agency is particularly focused on monitoring for market stability and identifying and prosecuting any form of misconduct that might threaten investors during periods of volatility.\n", "The rapid demise of Silicon Valley Bank and the fall of Signature Bank have left regulators racing to contain risks to the rest of the sector. On Tuesday, ratings agency Moody's cut its outlook on the U.S. banking system to \"negative\" from \"stable.\"\n", "\"\"\"\n", "\n", "text_2 =\"\"\"\n", "STOCKHOLM, March 15 (Reuters) - For years, Sweden has been warned that its dysfunctional housing market, plagued by under-supply and kept aloft by low rates and generous tax benefits, was a risk to the wider economy.\n", "Now those risks are becoming reality. Households with big mortgages are reining in spending as interest rates rise, and house-builders are pulling the plug on investment, tipping Sweden into recession.\n", "The country is set to be the only EU economy experiencing outright recession this year. The crown is trading at around its weakest level against the euro since the global financial crisis, partly due to housing market worries, making the central bank's job of curbing inflation more difficult.\n", "After years of ultra-low borrowing costs, the pandemic and the Ukraine war have served up a toxic cocktail of high inflation and rapidly rising interest rates to many countries.\n", "But in Sweden, the structural problems rooted in its housing market are magnifying the effects.\n", "\"\"\"\n", "\n", "preprocess_text_1 = text_1.strip().replace(\"\\n\",\"\")\n", "prompt_text_1 = \"summarize: \"+ preprocess_text_1\n", "\n", "preprocess_text_2 = text_2.strip().replace(\"\\n\",\"\")\n", "prompt_text_2 = \"summarize: \"+ preprocess_text_2\n", "\n", "text_inputs = [prompt_text_1, prompt_text_2] " ] }, { "cell_type": "code", "execution_count": null, "id": "fd9f8e14", "metadata": { "tags": [] }, "outputs": [], "source": [ "inputs = []\n", "inputs.append(httpclient.InferInput(\"INPUT0\", [len(text_inputs), 1], \"BYTES\"))\n", "\n", "batch_request = [[text_inputs[i]] for i in range(len(text_inputs))]\n", "\n", "input0_real = np.array(batch_request, dtype=np.object_)\n", "\n", "inputs[0].set_data_from_numpy(input0_real, binary_data=False)\n", "\n", "len(input0_real)" ] }, { "cell_type": "code", "execution_count": null, "id": "fbd840ff", "metadata": { "tags": [] }, "outputs": [], "source": [ "outputs = []\n", "\n", "outputs.append(httpclient.InferRequestedOutput(\"SUMMARY\"))" ] }, { "cell_type": "code", "execution_count": null, "id": "39af89fe", "metadata": { "tags": [] }, "outputs": [], "source": [ "request_body, header_length = httpclient.InferenceServerClient.generate_request_body(\n", " inputs, outputs=outputs\n", ")\n", "\n", "print(request_body)" ] }, { "cell_type": "code", "execution_count": null, "id": "2ad2f98e", "metadata": { "tags": [] }, "outputs": [], "source": [ "response = runtime_sm_client.invoke_endpoint(\n", " EndpointName=endpoint_name,\n", " ContentType='application/octet-stream',\n", " Body=request_body,\n", " TargetModel=python_model_file_name\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "4e0426b0", "metadata": { "tags": [] }, "outputs": [], "source": [ "header_length_prefix = \"application/vnd.sagemaker-triton.binary+json;json-header-size=\"\n", "header_length_str = response[\"ContentType\"][len(header_length_prefix) :]\n", "\n", "# Read response body\n", "result = httpclient.InferenceServerClient.parse_response_body(\n", " response[\"Body\"].read(), header_length=int(header_length_str)\n", ")\n", "\n", "outputs_data = result.as_numpy(\"SUMMARY\")\n", "\n", "for idx, output in enumerate(outputs_data):\n", " print(f'Original:\\n{text_inputs[idx]}\\n')\n", " print(f'Summary:\\n{output[0].decode()}\\n')" ] }, { "cell_type": "markdown", "id": "3d1aebf2-97fb-4a24-933e-fe6491a92d06", "metadata": {}, "source": [ "## Terminate endpoint and clean up artifacts" ] }, { "cell_type": "code", "execution_count": null, "id": "1601b665-7608-43b1-9f6d-5b0bebf3c09c", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false }, "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [], "source": [ "sm_client.delete_model(ModelName=sm_model_name)\n", "sm_client.delete_endpoint_config(EndpointConfigName=endpoint_config_name)\n", "sm_client.delete_endpoint(EndpointName=endpoint_name)" ] }, { "cell_type": "code", "execution_count": null, "id": "8c60aea5-74bc-4c04-87e7-ad4d77b8d3ab", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 } ], "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (PyTorch 1.10 Python 3.8 CPU Optimized)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/pytorch-1.10-cpu-py38" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.10" } }, "nbformat": 4, "nbformat_minor": 5 }