{
"cells": [
{
"cell_type": "markdown",
"id": "dab554c6",
"metadata": {},
"source": [
"# Deploy Stable Diffusion on a SageMaker GPU Multi-Model Endpoint with Triton"
]
},
{
"cell_type": "markdown",
"id": "5640df9b",
"metadata": {},
"source": [
"---\n",
"\n",
"This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.\n",
"\n",
"\n",
"\n",
"---\n",
"\n",
"In this notebook we will deploy multiple variations of Stable Diffusion on a SageMaker Multi-Model GPU Endpoint (MME GPU) powered by NVIDIA Triton Inference Server.\n",
"> ⚠ **Warning**: This notebook requires a minimum of an `ml.m5.large` instance to build the conda environment required for hosting the Stable Diffusion models. \n",
"\n",
"Skip to:\n",
"1. [Installs and imports](#installs)\n",
"2. [Download pretrained model](#modelartifact)\n",
"3. [Packaging a conda environment](#condaenv)\n",
"4. [Deploy to SageMaker Real-Time Endpoint](#deploy)\n",
"6. [Query Models](#query)\n",
"7. [Clean up](#cleanup)\n"
]
},
{
"cell_type": "markdown",
"id": "fdbf35ff",
"metadata": {},
"source": [
"### Part 1 - Installs and imports "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "69df6cd4",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%pip install -U sagemaker pillow huggingface-hub conda-pack"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "44c48876",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import boto3\n",
"import sagemaker\n",
"from sagemaker import get_execution_role\n",
"\n",
"import time\n",
"import json\n",
"from PIL import Image\n",
"import base64\n",
"from io import BytesIO\n",
"import numpy as np\n",
"\n",
"from utils import download_model\n",
"\n",
"from IPython.display import display\n",
"\n",
"# variables\n",
"s3_client = boto3.client(\"s3\")\n",
"ts = time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime())\n",
"\n",
"# sagemaker variables\n",
"role = get_execution_role()\n",
"sm_client = boto3.client(service_name=\"sagemaker\")\n",
"runtime_sm_client = boto3.client(\"sagemaker-runtime\")\n",
"sagemaker_session = sagemaker.Session(boto_session=boto3.Session())\n",
"bucket = sagemaker_session.default_bucket()\n",
"prefix = \"stable-diffusion-mme\""
]
},
{
"cell_type": "markdown",
"id": "7fda567c-afaa-4a7e-a947-98f1b15cb358",
"metadata": {},
"source": [
"### Part 2 - Save pretrained model "
]
},
{
"cell_type": "markdown",
"id": "d30d8b41-ffc3-49c6-831f-b70e08746284",
"metadata": {},
"source": [
"The `models` directory contains the inference code and the Triton configuration file for each of the Stable Diffusion models. In addition to these, we also need to download the pretrained model weights and save them to ther respective subdirectory within `models` directory. Once we have these downloaded, we can package the inference code and the model weights into a tarball and upload it to S3."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0cd50657-e4e9-4532-9763-473d1a0d9d38",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"models_local_path = {\n",
" \"stabilityai/stable-diffusion-2-1-base\": \"models/sd_base/1/checkpoint\",\n",
" \"stabilityai/stable-diffusion-2-depth\": \"models/sd_depth/1/checkpoint\",\n",
" \"stabilityai/stable-diffusion-2-inpainting\": \"models/sd_inpaint/1/checkpoint\",\n",
" \"stabilityai/stable-diffusion-x4-upscaler\": \"models/sd_upscale/1/checkpoint\",\n",
"}\n",
"\n",
"for model_name, model_local_path in models_local_path.items():\n",
" download_model(model_name, model_local_path)"
]
},
{
"cell_type": "markdown",
"id": "95d58e0e",
"metadata": {},
"source": [
"### Part 3 - Packaging a conda environment, extending Sagemaker Triton container "
]
},
{
"cell_type": "markdown",
"id": "8973a7d2",
"metadata": {},
"source": [
"When using the Triton Python backend (which our Stable Diffusion model will run on), you can include your own environment and dependencies. The recommended way to do this is to use [conda pack](https://conda.github.io/conda-pack/) to generate a conda environment archive in `tar.gz` format, and point to it in the `config.pbtxt` file of the models that should use it, adding the snippet: \n",
"\n",
"```\n",
"parameters: {\n",
" key: \"EXECUTION_ENV_PATH\",\n",
" value: {string_value: \"path_to_your_env.tar.gz\"}\n",
"}\n",
"\n",
"```\n",
"You can use a different environment per model, or the same for all models (read more on this [here](https://github.com/triton-inference-server/python_backend#creating-custom-execution-environments)). Since the all of the models that we'll be deploying have the same set of environment requirements, we will create a single conda environment and will use a Python backend to copy that environment into a location where it can be accessed by all models.\n",
"\n",
"> ⚠ **Warning**: The approach for a creating a shared conda environment highlighted here is limited to a single instance deployment only. In the event of auto-scaling, there is no guarantee that the new instance will have the conda environment configured. Since the conda environment for hosting Stable Diffusion models is quite large the recommended approach for production deployments is to create shared environment by extending the Triton Inference Image. \n",
"\n",
"Let's start by creating the conda environment with the necessary dependencies; running these cells will output a `sd_env.tar.gz` file."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5f523d03",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%%writefile environment.yml\n",
"name: mme_env\n",
"dependencies:\n",
" - python=3.8\n",
" - pip\n",
" - pip:\n",
" - numpy\n",
" - torch --extra-index-url https://download.pytorch.org/whl/cu118\n",
" - accelerate\n",
" - transformers\n",
" - diffusers\n",
" - xformers\n",
" - conda-pack"
]
},
{
"cell_type": "markdown",
"id": "b719d4ec-19a1-46cb-a887-9ccd72771e91",
"metadata": {},
"source": [
"Now we can create the environment using the above environment yaml spec\n",
"\n",
"🛈 It could take up to 5 min to create the conda environment. Make sure you are running this notebook in an `ml.m5.large` instance or above"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "147debc0",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!conda env create -f environment.yml"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3406a9fb",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!conda pack -n mme_env -o models/setup_conda/sd_env.tar.gz"
]
},
{
"cell_type": "markdown",
"id": "89830242-247e-458a-b2c2-6637c6845872",
"metadata": {
"tags": []
},
"source": [
"### Part 4 - Deploy endpoint "
]
},
{
"cell_type": "markdown",
"id": "551fa2a6-3562-49aa-88bc-e35c2cf75c79",
"metadata": {},
"source": [
"Now, we get the correct URI for the SageMaker Triton container image. Check out all the available Deep Learning Container images that AWS maintains [here](https://github.com/aws/deep-learning-containers/blob/master/available_images.md). "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b9b707c2-f78e-452a-9e99-4860232bd76b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# account mapping for SageMaker Triton Image\n",
"account_id_map = {\n",
" \"us-east-1\": \"785573368785\",\n",
" \"us-east-2\": \"007439368137\",\n",
" \"us-west-1\": \"710691900526\",\n",
" \"us-west-2\": \"301217895009\",\n",
" \"eu-west-1\": \"802834080501\",\n",
" \"eu-west-2\": \"205493899709\",\n",
" \"eu-west-3\": \"254080097072\",\n",
" \"eu-north-1\": \"601324751636\",\n",
" \"eu-south-1\": \"966458181534\",\n",
" \"eu-central-1\": \"746233611703\",\n",
" \"ap-east-1\": \"110948597952\",\n",
" \"ap-south-1\": \"763008648453\",\n",
" \"ap-northeast-1\": \"941853720454\",\n",
" \"ap-northeast-2\": \"151534178276\",\n",
" \"ap-southeast-1\": \"324986816169\",\n",
" \"ap-southeast-2\": \"355873309152\",\n",
" \"cn-northwest-1\": \"474822919863\",\n",
" \"cn-north-1\": \"472730292857\",\n",
" \"sa-east-1\": \"756306329178\",\n",
" \"ca-central-1\": \"464438896020\",\n",
" \"me-south-1\": \"836785723513\",\n",
" \"af-south-1\": \"774647643957\",\n",
"}\n",
"\n",
"\n",
"region = boto3.Session().region_name\n",
"if region not in account_id_map.keys():\n",
" raise (\"UNSUPPORTED REGION\")\n",
"\n",
"base = \"amazonaws.com.cn\" if region.startswith(\"cn-\") else \"amazonaws.com\"\n",
"mme_triton_image_uri = (\n",
" \"{account_id}.dkr.ecr.{region}.{base}/sagemaker-tritonserver:22.12-py3\".format(\n",
" account_id=account_id_map[region], region=region, base=base\n",
" )\n",
")"
]
},
{
"cell_type": "markdown",
"id": "50c7a738",
"metadata": {},
"source": [
"The next step is to package the model subdirectories and weights into individual tarballs and upload them to S3. This process can take a about 10 to 15 minutes."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ab3f34cb",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from pathlib import Path\n",
"\n",
"model_root_path = Path(\"./models\")\n",
"model_dirs = list(model_root_path.glob(\"*\"))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "db38cac8",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"model_upload_paths = {}\n",
"for model_path in model_dirs:\n",
" model_name = model_path.name\n",
" tar_name = model_path.name + \".tar.gz\"\n",
" !tar -C $model_root_path -czvf $tar_name $model_name\n",
" model_upload_paths[model_name] = sagemaker_session.upload_data(path=tar_name, bucket=bucket, key_prefix=prefix)\n",
" !rm $tar_name"
]
},
{
"cell_type": "markdown",
"id": "817c4844-d19c-4e4b-9634-63df5cd41464",
"metadata": {},
"source": [
"We are now ready to configure and deploy the multi-model endpoint"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5e328daa-3d5c-4c66-a6cc-7c80c4274dba",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"model_data_url = f\"s3://{bucket}/{prefix}/\" # s3 location where models are stored\n",
"ts = time.strftime(\"%Y-%m-%d-%H-%M-%S\", time.gmtime())\n",
"\n",
"container = {\n",
" \"Image\": mme_triton_image_uri,\n",
" \"ModelDataUrl\": model_data_url,\n",
" \"Mode\": \"MultiModel\",\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0cbcc545-a421-438c-b01f-e6420a504a0f",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"sm_model_name = f\"{prefix}-mdl-{ts}\"\n",
"\n",
"create_model_response = sm_client.create_model(\n",
" ModelName=sm_model_name, ExecutionRoleArn=role, PrimaryContainer=container\n",
")\n",
"\n",
"print(\"Model Arn: \" + create_model_response[\"ModelArn\"])"
]
},
{
"cell_type": "markdown",
"id": "9cb4c11a-4f8a-409e-832e-f27b8be07bb7",
"metadata": {},
"source": [
"Create a SageMaker endpoint configuration."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "75c9c1a6-92de-44d6-8b0b-7c6f388da957",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"endpoint_config_name = f\"{prefix}-epc-{ts}\"\n",
"instance_type = \"ml.g5.xlarge\"\n",
"\n",
"create_endpoint_config_response = sm_client.create_endpoint_config(\n",
" EndpointConfigName=endpoint_config_name,\n",
" ProductionVariants=[\n",
" {\n",
" \"InstanceType\": instance_type,\n",
" \"InitialVariantWeight\": 1,\n",
" \"InitialInstanceCount\": 1,\n",
" \"ModelName\": sm_model_name,\n",
" \"VariantName\": \"AllTraffic\",\n",
" }\n",
" ],\n",
")\n",
"\n",
"print(\"Endpoint Config Arn: \" + create_endpoint_config_response[\"EndpointConfigArn\"])"
]
},
{
"cell_type": "markdown",
"id": "2f09a802-706e-49f4-ae36-cb4e32960b6f",
"metadata": {},
"source": [
"Create the endpoint, and wait for it to transition to `InService` state."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "39066acd-e31e-433d-9200-65f7bb3b6975",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"endpoint_name = f\"{prefix}-ep-{ts}\"\n",
"\n",
"create_endpoint_response = sm_client.create_endpoint(\n",
" EndpointName=endpoint_name, EndpointConfigName=endpoint_config_name\n",
")\n",
"\n",
"print(\"Endpoint Arn: \" + create_endpoint_response[\"EndpointArn\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "abfee5c2-049d-4c96-89ea-725f99003bfe",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"resp = sm_client.describe_endpoint(EndpointName=endpoint_name)\n",
"status = resp[\"EndpointStatus\"]\n",
"print(\"Status: \" + status)\n",
"\n",
"while status == \"Creating\":\n",
" time.sleep(30)\n",
" resp = sm_client.describe_endpoint(EndpointName=endpoint_name)\n",
" status = resp[\"EndpointStatus\"]\n",
" print(\"Status: \" + status)\n",
"\n",
"print(\"Arn: \" + resp[\"EndpointArn\"])\n",
"print(\"Status: \" + status)"
]
},
{
"cell_type": "markdown",
"id": "aff32048-bf34-4b27-abe7-580929bdbe39",
"metadata": {},
"source": [
"### Query models \n",
"The endpoint is now deployed and we can query the individual models"
]
},
{
"cell_type": "markdown",
"id": "9be74afb-504c-48ed-b0fc-7177774c1e03",
"metadata": {},
"source": [
"Prior to invoking any of the Stable Diffusion Models, we first invoke the `setup_conda` which will copy the conda environment into a directory that can be shared with all the other models. Refer to the [model.py](./models/setup_conda/1/model.py) file in the `models/setup_conda/1` directory for more details on the implementation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "35d30696",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# invoke the setup_conda model to create the shared conda environment\n",
"\n",
"payload = {\n",
" \"inputs\": [\n",
" {\n",
" \"name\": \"TEXT\",\n",
" \"shape\": [1],\n",
" \"datatype\": \"BYTES\",\n",
" \"data\": [\"hello\"], # dummy data not used by the model\n",
" }\n",
" ]\n",
"}\n",
"\n",
"response = runtime_sm_client.invoke_endpoint(\n",
" EndpointName=endpoint_name,\n",
" ContentType=\"application/octet-stream\",\n",
" Body=json.dumps(payload),\n",
" TargetModel=\"setup_conda.tar.gz\",\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "093b1151",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# helper functions to encode and decode images\n",
"def encode_image(image):\n",
" buffer = BytesIO()\n",
" image.save(buffer, format=\"JPEG\")\n",
" img_str = base64.b64encode(buffer.getvalue())\n",
"\n",
" return img_str\n",
"\n",
"\n",
"def decode_image(img):\n",
" buff = BytesIO(base64.b64decode(img.encode(\"utf8\")))\n",
" image = Image.open(buff)\n",
" return image"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cd2ca25e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"inputs = dict(\n",
" prompt=\"Infinity pool on top of a high rise overlooking Central Park\",\n",
" negative_prompt=\"blur, signature, low detail, low quality\",\n",
" gen_args=json.dumps(dict(num_inference_steps=50, guidance_scale=8)),\n",
")\n",
"\n",
"payload = {\n",
" \"inputs\": [\n",
" {\"name\": name, \"shape\": [1, 1], \"datatype\": \"BYTES\", \"data\": [data]}\n",
" for name, data in inputs.items()\n",
" ]\n",
"}\n",
"\n",
"response = runtime_sm_client.invoke_endpoint(\n",
" EndpointName=endpoint_name,\n",
" ContentType=\"application/octet-stream\",\n",
" Body=json.dumps(payload),\n",
" TargetModel=\"sd_base.tar.gz\",\n",
")\n",
"output = json.loads(response[\"Body\"].read().decode(\"utf8\"))[\"outputs\"]\n",
"original_image = decode_image(output[0][\"data\"][0])\n",
"original_image"
]
},
{
"cell_type": "markdown",
"id": "f21b76a6-4914-4b5d-af27-fffa7198e4be",
"metadata": {},
"source": [
"Let's take the output from the Standard Model and modify it using the depth model.\n",
"\n",
"We can use the same model to change the style of the original image into an oil panting or change the setting from New York City Central Park to the Yellowstone National Park while preserving the orientation of the original image"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d71af00d",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"input_image = encode_image(original_image).decode(\"utf8\")\n",
"\n",
"inputs = dict(\n",
" prompt=\"highly detailed oil painting of an inifinity pool overlooking central park\",\n",
" image=input_image,\n",
" gen_args=json.dumps(dict(num_inference_steps=50, strength=0.8)),\n",
")\n",
"\n",
"\n",
"payload = {\n",
" \"inputs\": [\n",
" {\"name\": name, \"shape\": [1, 1], \"datatype\": \"BYTES\", \"data\": [data]}\n",
" for name, data in inputs.items()\n",
" ]\n",
"}\n",
"\n",
"response = runtime_sm_client.invoke_endpoint(\n",
" EndpointName=endpoint_name,\n",
" ContentType=\"application/octet-stream\",\n",
" Body=json.dumps(payload),\n",
" TargetModel=\"sd_depth.tar.gz\",\n",
")\n",
"output = json.loads(response[\"Body\"].read().decode(\"utf8\"))[\"outputs\"]\n",
"oil_painting = decode_image(output[0][\"data\"][0])\n",
"\n",
"\n",
"inputs = dict(\n",
" prompt=\"Infinity pool perched on a cliff overlooking Yellowstone National Park \",\n",
" image=input_image,\n",
" gen_args=json.dumps(dict(num_inference_steps=50, strength=0.8)),\n",
")\n",
"\n",
"\n",
"payload = {\n",
" \"inputs\": [\n",
" {\"name\": name, \"shape\": [1, 1], \"datatype\": \"BYTES\", \"data\": [data]}\n",
" for name, data in inputs.items()\n",
" ]\n",
"}\n",
"\n",
"response = runtime_sm_client.invoke_endpoint(\n",
" EndpointName=endpoint_name,\n",
" ContentType=\"application/octet-stream\",\n",
" Body=json.dumps(payload),\n",
" TargetModel=\"sd_depth.tar.gz\",\n",
")\n",
"output = json.loads(response[\"Body\"].read().decode(\"utf8\"))[\"outputs\"]\n",
"rocky_mountains = decode_image(output[0][\"data\"][0])\n",
"\n",
"\n",
"print(\"Original image\")\n",
"display(original_image)\n",
"\n",
"print(\"Oil painting\")\n",
"display(oil_painting)\n",
"\n",
"print(\"Yellowstone\")\n",
"display(rocky_mountains)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1d2aead9",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"source_image = Image.open(\"sample_images/bertrand-gabioud.png\")\n",
"\n",
"image = encode_image(source_image).decode(\"utf8\")\n",
"mask_image = encode_image(Image.open(\"sample_images/bertrand-gabioud-mask.png\")).decode(\"utf8\")\n",
"inputs = dict(\n",
" prompt=\"building, facade, paint, windows\",\n",
" image=image,\n",
" mask_image=mask_image,\n",
" negative_prompt=\"tree, obstruction, sky, clouds\",\n",
" gen_args=json.dumps(dict(num_inference_steps=50, guidance_scale=10)),\n",
")\n",
"\n",
"\n",
"payload = {\n",
" \"inputs\": [\n",
" {\"name\": name, \"shape\": [1, 1], \"datatype\": \"BYTES\", \"data\": [data]}\n",
" for name, data in inputs.items()\n",
" ]\n",
"}\n",
"\n",
"response = runtime_sm_client.invoke_endpoint(\n",
" EndpointName=endpoint_name,\n",
" ContentType=\"application/octet-stream\",\n",
" Body=json.dumps(payload),\n",
" TargetModel=\"sd_inpaint.tar.gz\",\n",
")\n",
"output = json.loads(response[\"Body\"].read().decode(\"utf8\"))[\"outputs\"]\n",
"print(\"source image\")\n",
"display(source_image)\n",
"\n",
"print(\"filled image\")\n",
"display(decode_image(output[0][\"data\"][0]))"
]
},
{
"cell_type": "markdown",
"id": "04832589-d909-460a-a038-9e6f8069f6a3",
"metadata": {},
"source": [
"For the final example we will downsize our original output image from 512x512 to 128x128. We will then use the upscaling model to upscale the image back to its original 512 resolution"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef41b693-5344-4ebc-a29b-1b931446a85a",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"low_res_image = original_image.resize((128, 128))\n",
"inputs = dict(\n",
" prompt=\"Infinity pool on top of a high rise overlooking Central Park\",\n",
" image=encode_image(low_res_image).decode(\"utf8\"),\n",
")\n",
"\n",
"payload = {\n",
" \"inputs\": [\n",
" {\"name\": name, \"shape\": [1, 1], \"datatype\": \"BYTES\", \"data\": [data]}\n",
" for name, data in inputs.items()\n",
" ]\n",
"}\n",
"\n",
"response = runtime_sm_client.invoke_endpoint(\n",
" EndpointName=endpoint_name,\n",
" ContentType=\"application/octet-stream\",\n",
" Body=json.dumps(payload),\n",
" TargetModel=\"sd_upscale.tar.gz\",\n",
")\n",
"output = json.loads(response[\"Body\"].read().decode(\"utf8\"))[\"outputs\"]\n",
"upscaled_image = decode_image(output[0][\"data\"][0])\n",
"\n",
"print(\"Low res image\")\n",
"display(low_res_image.resize((512, 512)))\n",
"\n",
"print(\"Upscaled image\")\n",
"display(upscaled_image)"
]
},
{
"cell_type": "markdown",
"id": "b6e7824e",
"metadata": {},
"source": [
"## Clean up "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2688904b-426a-4d3b-8e7a-30701f0f752a",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"sm_client.delete_endpoint(EndpointName=endpoint_name)\n",
"sm_client.delete_endpoint_config(EndpointConfigName=endpoint_config_name)\n",
"sm_client.delete_model(ModelName=sm_model_name)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "54a6ef61",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#delete models in respective paths\n",
"for model_name, model_local_path in models_local_path.items():\n",
" !rm -rf $model_local_path"
]
},
{
"cell_type": "markdown",
"id": "5d8f70db-8c31-4a94-a703-2b67c539c0ae",
"metadata": {},
"source": [
"## Notebook CI Test Results\n",
"\n",
"This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
}
],
"metadata": {
"availableInstances": [
{
"_defaultOrder": 0,
"_isFastLaunch": true,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 4,
"name": "ml.t3.medium",
"vcpuNum": 2
},
{
"_defaultOrder": 1,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.t3.large",
"vcpuNum": 2
},
{
"_defaultOrder": 2,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.t3.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 3,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.t3.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 4,
"_isFastLaunch": true,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.m5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 5,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.m5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 6,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.m5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 7,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.m5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 8,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.m5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 9,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.m5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 10,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.m5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 11,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.m5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 12,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.m5d.large",
"vcpuNum": 2
},
{
"_defaultOrder": 13,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.m5d.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 14,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.m5d.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 15,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.m5d.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 16,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.m5d.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 17,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.m5d.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 18,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.m5d.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 19,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.m5d.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 20,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": true,
"memoryGiB": 0,
"name": "ml.geospatial.interactive",
"supportedImageNames": [
"sagemaker-geospatial-v1-0"
],
"vcpuNum": 0
},
{
"_defaultOrder": 21,
"_isFastLaunch": true,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 4,
"name": "ml.c5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 22,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.c5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 23,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.c5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 24,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.c5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 25,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 72,
"name": "ml.c5.9xlarge",
"vcpuNum": 36
},
{
"_defaultOrder": 26,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 96,
"name": "ml.c5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 27,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 144,
"name": "ml.c5.18xlarge",
"vcpuNum": 72
},
{
"_defaultOrder": 28,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.c5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 29,
"_isFastLaunch": true,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.g4dn.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 30,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.g4dn.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 31,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.g4dn.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 32,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.g4dn.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 33,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.g4dn.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 34,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.g4dn.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 35,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 61,
"name": "ml.p3.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 36,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 244,
"name": "ml.p3.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 37,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 488,
"name": "ml.p3.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 38,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.p3dn.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 39,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.r5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 40,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.r5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 41,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.r5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 42,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.r5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 43,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.r5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 44,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.r5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 45,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 512,
"name": "ml.r5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 46,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.r5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 47,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.g5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 48,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.g5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 49,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.g5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 50,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.g5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 51,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.g5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 52,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.g5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 53,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.g5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 54,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.g5.48xlarge",
"vcpuNum": 192
},
{
"_defaultOrder": 55,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 1152,
"name": "ml.p4d.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 56,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 1152,
"name": "ml.p4de.24xlarge",
"vcpuNum": 96
}
],
"instance_type": "ml.m5.large",
"kernelspec": {
"display_name": "Python 3 (Base Python 3.0)",
"language": "python",
"name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-base-python-310-v1"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}