{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "8baba417-41ad-4c46-bdb6-2a8a9f3706db", "metadata": { "tags": [] }, "source": [ "# Deploy a pretrained HuggingFace InstructPix2Pix (Diffuser) Model into a SageMaker Endpoint\n", "\n", "---\n", "\n", "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.\n", "\n", "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-2/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "---\n", "\n", "HuggingFace Diffusers provide pretrained vision and audio diffusion models, and serves as a modular toolbox for inference and training. \n", "\n", "Amazon SageMaker is a fully managed service that provides developers and data scientists with the ability to build, train, and deploy machine learning (ML) models quickly. Amazon SageMaker removes the heavy lifting from each step of the machine learning process to make it easier to develop high-quality models. The SageMaker Python SDK provides open source APIs and containers that make it easy to train and deploy models in Amazon SageMaker with several machine learning and deep learning frameworks.\n", "\n", "HuggingFace models, which have a corresponding HuggingFace task, can be deployed to SageMaker as a `HuggingFaceModel` object, as described in [the documentation](https://huggingface.co/docs/sagemaker/inference). Yet, not all the models have a corresponding HuggingFace task.\n", "\n", "In this notebook, we will use an approach to deploy any model from HuggingFace hub by taking files from a Git repo. As an example, we will use a pretrained instruction-based image editing model (https://huggingface.co/timbrooks/instruct-pix2pix). To do this, we will implement a custom entry point `inference.py` that will initiate the model in a SageMaker container and handle inference requests, which contain multimodal input (text prompt and image data).\n", "\n", "We will execute the following steps:\n", "\n", "- Download a HuggingFace model to the local file system with Git LFS.\n", "- Tar-gzip the model and config files, and upload `model.tar.gz` to an S3 bucket.\n", "- Deploy the model to a SageMaker Endpoint and make an inference request.\n", "- Optionally, use inference recommender to check how different instance types perform as an endpoint.\n", "- Optionally, cleanup.\n", "\n", "Note that this notebook was adopted from another SageMaker example of [Pretrained PyTorch BERT model for sentiment analysis](https://github.com/aws/amazon-sagemaker-examples/blob/main/advanced_functionality/pytorch_deploy_pretrained_bert_model/pytorch_deploy_pretrained_bert_model.ipynb). Our example is different, because it doesn't require the model to be loaded with HuggingFace library into memory before saving it and sending for inference to SageMaker Endpoint.\n", "\n", "For a step by step, hands-on learning experience about deploying other large generative AI models, please check: https://catalog.us-east-1.prod.workshops.aws/workshops/bb62b5d7-313f-4733-88cd-9c1aa41c724d/en-US/" ] }, { "attachments": {}, "cell_type": "markdown", "id": "01895155-1623-4c17-83e8-a57f6d5bda1d", "metadata": {}, "source": [ "Let's start by creating a SageMaker session and specifying:\n", "\n", "- The S3 bucket and prefix that you want to use for the model data. This should be within the same region as the notebook instance, training, and hosting.\n", "- The IAM role arn used to give hosting access to your data. See the documentation for how to create these. Note, if you want to use another role, please replace the `sagemaker.get_execution_role()` with the appropriate full IAM role arn string." ] }, { "cell_type": "code", "execution_count": null, "id": "606589a3-02ae-4a96-945f-e18eeaeb167b", "metadata": { "tags": [] }, "outputs": [], "source": [ "import sagemaker\n", "\n", "sagemaker_session = sagemaker.Session()\n", "bucket = sagemaker_session.default_bucket()\n", "region = sagemaker_session.boto_region_name\n", "model_prefix = \"timbrooks/instruct-pix2pix\"\n", "role = sagemaker.get_execution_role()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "75be087c-c3d5-4a0a-878b-1dec2c03087e", "metadata": { "tags": [] }, "source": [ "## Download the `InstructPix2Pix` model artifacts from HuggingFace hub\n", "\n", "We install Git LFS to handle large files in the git repository. Then we clone the repository locally to save the pre-trained model on the file system. " ] }, { "cell_type": "code", "execution_count": null, "id": "35756c15-9d7a-4c60-ac3a-5afd23e64f52", "metadata": { "tags": [] }, "outputs": [], "source": [ "!apt-get -qq update\n", "!apt-get -qq install -y curl git-lfs" ] }, { "cell_type": "code", "execution_count": null, "id": "8229aafa-9abc-4baf-9190-220a1aa7debe", "metadata": { "tags": [] }, "outputs": [], "source": [ "!git lfs install" ] }, { "attachments": {}, "cell_type": "markdown", "id": "f6102229-a615-4a5c-a340-17b3a40fb41a", "metadata": {}, "source": [ "We skip downloading `safetensors` and `ckpt` files as they are heavyweight and not needed to deploy the model into SageMaker." ] }, { "cell_type": "code", "execution_count": null, "id": "52bd3eb0-8c1b-4e50-b4a3-7b572991054b", "metadata": { "tags": [] }, "outputs": [], "source": [ "!GIT_LFS_SKIP_SMUDGE=1 git clone --depth 1 https://huggingface.co/timbrooks/instruct-pix2pix\n", "!git -C instruct-pix2pix/ lfs pull --exclude='*.safetensors,*.ckpt'" ] }, { "attachments": {}, "cell_type": "markdown", "id": "40da28e1-3e29-4332-b0b3-ce8ca632e96b", "metadata": { "tags": [] }, "source": [ "*Note:* As of the date of writing of this example, `git clone` command with LFS had [memory constraints](https://github.com/git-lfs/git-lfs/issues/3524). To be able to run this example on `ml.t3.medium` instance that has only 4 GB of RAM, we split clone the command in two." ] }, { "attachments": {}, "cell_type": "markdown", "id": "977dea72-4736-47d9-bfe5-6a08abf39bf3", "metadata": {}, "source": [ "Cleanup the LFS pointers to the skipped files." ] }, { "cell_type": "code", "execution_count": null, "id": "5d46d981-5d29-42d9-8739-c5dabd863f2d", "metadata": { "tags": [] }, "outputs": [], "source": [ "!find ./instruct-pix2pix -name \"*.ckpt\" -type f -delete\n", "!find ./instruct-pix2pix -name \"*.safetensors\" -type f -delete" ] }, { "attachments": {}, "cell_type": "markdown", "id": "04f4e894-e467-4ee8-bed1-e682feaa6cf6", "metadata": {}, "source": [ "SageMaker does not need git information. Cleanup the git files to save disk space." ] }, { "cell_type": "code", "execution_count": null, "id": "4befc61a-8376-452b-8324-849c30739f7e", "metadata": { "tags": [] }, "outputs": [], "source": [ "!rm -r ./instruct-pix2pix/.git\n", "!rm ./instruct-pix2pix/.gitattributes" ] }, { "attachments": {}, "cell_type": "markdown", "id": "e8a65f13-cf85-4487-82de-50bfb2379cec", "metadata": {}, "source": [ "## Package the pre-trained model and upload it to S3\n", "\n", "Now you can see that there is a pretrained HuggingFace model under `instruct-pix2pix/` directory by listing the files in it, and you can upload it to S3." ] }, { "cell_type": "code", "execution_count": null, "id": "0a1bd163-0eb6-453f-a11a-cbbc7a8f627a", "metadata": { "tags": [] }, "outputs": [], "source": [ "!mkdir -p ./instruct-pix2pix/code/\n", "!cp -v ./code/* ./instruct-pix2pix/code/" ] }, { "cell_type": "code", "execution_count": null, "id": "f1671c44-f457-41a1-8ca4-f0f14bdebbb7", "metadata": { "tags": [] }, "outputs": [], "source": [ "!find ./instruct-pix2pix/" ] }, { "cell_type": "code", "execution_count": null, "id": "d462955f-0e27-40a9-9284-a680d2fa6426", "metadata": { "tags": [] }, "outputs": [], "source": [ "!tar --use-compress-program='gzip --fast' -cvf ./model.tar.gz -C ./instruct-pix2pix/ ." ] }, { "cell_type": "code", "execution_count": null, "id": "9f459dad-d60b-4cb7-9861-fba41779797d", "metadata": { "tags": [] }, "outputs": [], "source": [ "pretrained_model_data = sagemaker_session.upload_data(\n", " path=\"model.tar.gz\", bucket=bucket, key_prefix=model_prefix\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "fdb6678d-a8c5-442d-b344-71c650f9fdcd", "metadata": { "tags": [] }, "outputs": [], "source": [ "!pygmentize instruct-pix2pix/code/inference.py" ] }, { "cell_type": "code", "execution_count": null, "id": "9e81b5c0-46f7-4255-a53d-5e6c7fec21da", "metadata": { "tags": [] }, "outputs": [], "source": [ "!pygmentize instruct-pix2pix/code/requirements.txt" ] }, { "cell_type": "code", "execution_count": null, "id": "39af8805-de6d-43c4-9128-8cfc873753dd", "metadata": { "tags": [] }, "outputs": [], "source": [ "from sagemaker.pytorch.model import PyTorchModel\n", "\n", "pytorch_model = PyTorchModel(\n", " model_data=pretrained_model_data,\n", " role=role,\n", " framework_version=\"1.12\",\n", " py_version=\"py38\",\n", " source_dir=\"instruct-pix2pix/code\",\n", " entry_point=\"inference.py\",\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "fad2b0e3-9d0e-478c-a4b9-160972c02f69", "metadata": { "tags": [] }, "outputs": [], "source": [ "predictor = pytorch_model.deploy(initial_instance_count=1, instance_type=\"ml.g4dn.xlarge\")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "6e397d3c-6fac-42ff-b0ba-4a73e2e9e974", "metadata": {}, "source": [ "Since in the `input_fn` we declared that the incoming requests are JSON-encoded, we need to use a `JSONSerializer`.\n", "\n", "Also, we return a base64 string. So, we need to use a `StringDeserializer` to parse the response." ] }, { "cell_type": "code", "execution_count": null, "id": "93aa3226-ec99-4566-9342-26e6ce20e817", "metadata": { "tags": [] }, "outputs": [], "source": [ "from sagemaker.serializers import JSONSerializer\n", "from sagemaker.deserializers import StringDeserializer\n", "\n", "predictor.serializer = JSONSerializer()\n", "predictor.deserializer = StringDeserializer(accept=\"text/plain\")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "a0ed0f50-44b9-4323-8e8a-46773cf65ef5", "metadata": { "tags": [] }, "source": [ "## Test the model\n", "\n", "Using few samples, you can now invoke the SageMaker endpoint to get predictions. Note that we have a multimodal input as the model expects a prompt message as well as an image.\n", "So, we will pack the prompt and the image into a JSON object. For that we need to base64-encode the image." ] }, { "cell_type": "code", "execution_count": null, "id": "05cf6920-2d71-4ec1-a8e1-981ba69d4983", "metadata": { "tags": [] }, "outputs": [], "source": [ "from PIL import Image, ImageOps\n", "import requests\n", "\n", "url = \"https://raw.githubusercontent.com/timothybrooks/instruct-pix2pix/main/imgs/example.jpg\"\n", "\n", "\n", "def download_image(image_url):\n", " result = Image.open(requests.get(image_url, stream=True).raw)\n", " result = ImageOps.exif_transpose(result)\n", " result = result.convert(\"RGB\")\n", " return result\n", "\n", "\n", "image = download_image(url)\n", "image" ] }, { "cell_type": "code", "execution_count": null, "id": "c95703ad-ded6-4b8c-bce3-57e17c8d21ac", "metadata": { "tags": [] }, "outputs": [], "source": [ "import base64\n", "from io import BytesIO\n", "\n", "buffered = BytesIO()\n", "image.save(buffered, format=\"PNG\")\n", "img_str = base64.b64encode(buffered.getvalue())\n", "base64_string = img_str.decode(\"latin1\")" ] }, { "cell_type": "code", "execution_count": null, "id": "40ec904b-f572-4852-9e7c-f48ac1dc7633", "metadata": { "tags": [] }, "outputs": [], "source": [ "data = {\"inputs\": {\"prompt\": \"turn him into cyborg\", \"image\": base64_string}}" ] }, { "cell_type": "code", "execution_count": null, "id": "571e0293-7b9f-451b-b54c-a405365082c8", "metadata": { "tags": [] }, "outputs": [], "source": [ "prediction_result = predictor.predict(data)" ] }, { "cell_type": "code", "execution_count": null, "id": "5860dce3-f70f-435c-99b0-4d947483c702", "metadata": { "tags": [] }, "outputs": [], "source": [ "f = BytesIO(base64.b64decode(prediction_result))\n", "img = Image.open(f)\n", "img" ] }, { "attachments": {}, "cell_type": "markdown", "id": "f4124d74-8fda-48e1-b63e-737ef204e6da", "metadata": { "tags": [] }, "source": [ "## Optional: Use Inference Recommender to Select the Best Instance\n", "\n", "Inference Recommender uses information about your ML model to recommend the best instance types and endpoint configurations for deployment. \n", "\n", "As the first step, we fetch the images the inference-recommender should use in the multimodal input. " ] }, { "cell_type": "code", "execution_count": null, "id": "adfefecd-ac41-45da-8416-8a8f50f7b22b", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Save image(s) to use for payload generation\n", "from sagemaker.s3 import S3Downloader, S3Uploader\n", "\n", "!mkdir -p ./images/pets\n", "S3Downloader().download(s3_uri=f\"s3://sagemaker-example-files-{region}/datasets/image/pets\", local_path=\"./images/pets\")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "97ea307d-2fe8-4ccc-8152-723483d35cbb", "metadata": {}, "source": [ "Now, we can create the multimodal payloads using the downloaded images and a prompt. The inference recommender will do random sampling to use the payloads in load tests. Note that we resize the images to 512x512 to load test based on the same size. Additionally, we only use one prompt message in this example. Make sure that your sample-payloads used by the inference recommender have a similar distribution of data to the different payloads expected in production." ] }, { "cell_type": "code", "execution_count": null, "id": "6f9dc94a-d7e2-4295-856e-5c5efee46732", "metadata": { "tags": [] }, "outputs": [], "source": [ "from io import BytesIO\n", "import json\n", "\n", "\n", "def create_sample_payload(images_directory):\n", " for image_path in os.listdir(images_directory):\n", " input_path = os.path.join(images_directory, image_path)\n", " print(input_path)\n", " img = Image.open(input_path)\n", " img = img.resize((512, 512)) # resizing to load test based on the same size\n", "\n", " buffered = BytesIO()\n", " img.save(buffered, format=\"PNG\")\n", " img_str = base64.b64encode(buffered.getvalue())\n", " base64_string = img_str.decode(\"latin1\")\n", " payload = {\"inputs\": {\"prompt\": \"turn him into cyborg\", \"image\": base64_string}}\n", " output_path = os.path.join(\"sample-payload\", image_path.replace(\".\", \"_\") + \"_payload.txt\")\n", " with open(output_path, \"w\") as file:\n", " file.write(json.dumps(payload))" ] }, { "cell_type": "code", "execution_count": null, "id": "78f9e5e7-3674-497a-ae12-fc2da62fcd9a", "metadata": { "tags": [] }, "outputs": [], "source": [ "!mkdir ./sample-payload\n", "import os\n", "\n", "create_sample_payload(\"./images/pets\")" ] }, { "cell_type": "code", "execution_count": null, "id": "ccc1216f-b7ad-499f-87b7-0445df4c802c", "metadata": { "tags": [] }, "outputs": [], "source": [ "payload_archive_name = \"payload.tar.gz\"\n", "!cd ./sample-payload/ && tar czvf ../{payload_archive_name} *" ] }, { "cell_type": "code", "execution_count": null, "id": "1ad13cf1-d1f4-4931-9f32-b935067c88a6", "metadata": { "tags": [] }, "outputs": [], "source": [ "sample_payload_url = sagemaker.Session().upload_data(\n", " payload_archive_name, bucket=bucket, key_prefix=model_prefix + \"/inference-recommender\"\n", ")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "6e92c8cb-542c-4372-a153-48ebf7c1986d", "metadata": {}, "source": [ "Provide domain and model related information to Inference Recommender:\n", "\n", "Example ML Domains: COMPUTER_VISION, NATURAL_LANGUAGE_PROCESSING, MACHINE_LEARNING\n", "Example ML Tasks: CLASSIFICATION, REGRESSION, OBJECT_DETECTION, OTHER\n", "Note: Select the task that is the closest match to your model. Chose OTHER if none apply." ] }, { "cell_type": "code", "execution_count": null, "id": "fbd60228-58c3-40d9-b815-011ee9a24273", "metadata": { "tags": [] }, "outputs": [], "source": [ "ml_domain = \"MACHINE_LEARNING\"\n", "ml_task = \"OTHER\"\n", "ml_framework = \"PYTORCH\"\n", "framework_version = \"1.12\"" ] }, { "cell_type": "code", "execution_count": null, "id": "38b65a57-1a68-47af-9ef3-5aeae5d111e4", "metadata": { "tags": [] }, "outputs": [], "source": [ "from sagemaker import image_uris\n", "\n", "inference_image = image_uris.retrieve(\n", " framework=\"pytorch\",\n", " region=region,\n", " version=framework_version,\n", " py_version=\"py38\",\n", " instance_type=\"ml.p3.8xlarge\",\n", " image_scope=\"inference\",\n", ")\n", "\n", "print(inference_image)" ] }, { "cell_type": "code", "execution_count": null, "id": "fd47f280-90cb-45b8-b826-48133f803ea0", "metadata": { "tags": [] }, "outputs": [], "source": [ "import time\n", "import boto3\n", "\n", "client = boto3.client(\"sagemaker\", region)\n", "\n", "model_package_group_name = \"instruct-pix2pix-\" + str(round(time.time()))\n", "print(model_package_group_name)\n", "model_package_group_response = client.create_model_package_group(\n", " ModelPackageGroupName=str(model_package_group_name),\n", " ModelPackageGroupDescription=\"instruct-pix2pix model group\",\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "9c24438e-6d2a-459e-ac37-acb464f1e52e", "metadata": { "tags": [] }, "outputs": [], "source": [ "model_package_version_response = client.create_model_package(\n", " ModelPackageGroupName=str(model_package_group_name),\n", " ModelPackageDescription=\"instruct-pix2pix Inference Recommender Demo\",\n", " Domain=ml_domain,\n", " Task=ml_task,\n", " SamplePayloadUrl=sample_payload_url,\n", " InferenceSpecification={\n", " \"Containers\": [\n", " {\n", " \"ContainerHostname\": \"pytorch\",\n", " \"Image\": inference_image,\n", " \"ModelDataUrl\": pretrained_model_data,\n", " \"Framework\": ml_framework,\n", " \"FrameworkVersion\": framework_version,\n", " \"Environment\": {\n", " \"SAGEMAKER_CONTAINER_LOG_LEVEL\": \"20\",\n", " \"SAGEMAKER_PROGRAM\": \"inference.py\",\n", " \"SAGEMAKER_REGION\": region,\n", " \"SAGEMAKER_SUBMIT_DIRECTORY\": pretrained_model_data,\n", " },\n", " },\n", " ],\n", " \"SupportedRealtimeInferenceInstanceTypes\": [\n", " \"ml.g4dn.xlarge\",\n", " \"ml.g4dn.12xlarge\",\n", " \"ml.g5.xlarge\",\n", " \"ml.p3.2xlarge\",\n", " \"ml.p3.8xlarge\",\n", " ],\n", " \"SupportedContentTypes\": [\"application/json\"],\n", " \"SupportedResponseMIMETypes\": [\"text/plain\"],\n", " },\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "251d0a15-35c5-4bfe-8bd0-408d6bafe775", "metadata": { "tags": [] }, "outputs": [], "source": [ "import datetime\n", "\n", "default_job = \"instrpx2px-\" + datetime.datetime.now().strftime(\"%d-%H-%M-%S\")\n", "default_response = client.create_inference_recommendations_job(\n", " JobName=str(default_job),\n", " JobDescription=\"instruct-pix2pix Inference Basic Recommender Job\",\n", " JobType=\"Default\",\n", " RoleArn=role,\n", " InputConfig={\"ModelPackageVersionArn\": model_package_version_response[\"ModelPackageArn\"]},\n", ")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "acdb3620-ae43-4fdb-8016-6b1c94a8cec8", "metadata": {}, "source": [ "The inference recommender job provides multiple endpoint recommendations in its result. The recommendation includes `InstanceType`, `InitialInstanceCount`, `EnvironmentParameters` which includes tuned parameters for better performance. We also include the benchmarking results like `MaxInvocations`, `ModelLatency`, `CostPerHour` and `CostPerInference` for deeper analysis. The information provided will help you narrow down to a specific endpoint configuration that suits your use case." ] }, { "cell_type": "code", "execution_count": null, "id": "19660816-2418-4ae3-bb37-0ac2e6f0d92f", "metadata": { "tags": [] }, "outputs": [], "source": [ "%%time\n", "\n", "import boto3\n", "\n", "client = boto3.client(\"sagemaker\", region)\n", "\n", "ended = False\n", "while not ended:\n", " inference_recommender_job = client.describe_inference_recommendations_job(\n", " JobName=str(default_job)\n", " )\n", " if inference_recommender_job[\"Status\"] in [\"COMPLETED\", \"STOPPED\", \"FAILED\"]:\n", " ended = True\n", " else:\n", " print(\"Inference recommender job in progress\")\n", " time.sleep(60)\n", "\n", "if inference_recommender_job[\"Status\"] == \"FAILED\":\n", " print(\"Inference recommender job failed \")\n", " print(\"Failed Reason: {}\".inference_recommender_job[\"FailedReason\"])\n", "else:\n", " print(\"Inference recommender job completed\")" ] }, { "cell_type": "code", "execution_count": null, "id": "37711199-16a7-4dce-adce-66afbbe9cad1", "metadata": { "tags": [] }, "outputs": [], "source": [ "import pandas as pd\n", "\n", "data = [\n", " {**x[\"EndpointConfiguration\"], **x[\"ModelConfiguration\"], **x[\"Metrics\"]}\n", " for x in inference_recommender_job[\"InferenceRecommendations\"]\n", "]\n", "df = pd.DataFrame(data)\n", "dropFilter = df.filter([\"VariantName\"])\n", "df.drop(dropFilter, inplace=True, axis=1)\n", "pd.set_option(\"max_colwidth\", 400)" ] }, { "cell_type": "code", "execution_count": null, "id": "2135295e-1b03-4496-bb53-b1b3aff61963", "metadata": { "tags": [] }, "outputs": [], "source": [ "df[\n", " [\n", " \"EndpointName\",\n", " \"InstanceType\",\n", " \"CostPerHour\",\n", " \"CostPerInference\",\n", " \"MaxInvocations\",\n", " \"ModelLatency\",\n", " ]\n", "].sort_values(by=[\"MaxInvocations\"], ascending=False).head()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "4fb61550-0520-4e56-a157-df5c63892080", "metadata": {}, "source": [ "Note how the new generation instances such as G5 are outperforming the G4 instances of the previous generation!" ] }, { "attachments": {}, "cell_type": "markdown", "id": "f36f6d91-50c8-407f-a428-d105bf8884a5", "metadata": {}, "source": [ "Clean up Inference Recommender related artifacts." ] }, { "cell_type": "code", "execution_count": null, "id": "81f1e792-2dfd-4bcb-ae18-03d25b0b5f55", "metadata": { "tags": [] }, "outputs": [], "source": [ "!rm -rf ./sample-payload\n", "!rm -rf ./images\n", "!rm payload.tar.gz\n", "!aws s3 rm --quiet $sample_payload_url" ] }, { "attachments": {}, "cell_type": "markdown", "id": "ff2d1ff8-d459-4c8c-a70a-4a1a5ce084dd", "metadata": {}, "source": [ "## Clean up deployment related artifacts and the endpoint\n", "\n", "Endpoints should be deleted when no longer in use, since they're billed by time deployed, according to the [SageMaker pricing page](https://aws.amazon.com/sagemaker/pricing/)." ] }, { "cell_type": "code", "execution_count": null, "id": "193abc8f-d887-41bf-a253-7b96a4f6a0b5", "metadata": { "tags": [] }, "outputs": [], "source": [ "predictor.delete_endpoint()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "137e4f8a-dd3e-42bd-8f72-3012497779cb", "metadata": {}, "source": [ "Also remove the cloned directory and the model object." ] }, { "cell_type": "code", "execution_count": null, "id": "f295a778-2179-4e53-9610-3ebac97894d3", "metadata": { "tags": [] }, "outputs": [], "source": [ "!rm -rf ./instruct-pix2pix" ] }, { "cell_type": "code", "execution_count": null, "id": "51bbb705-baaf-4371-a39c-6f07edb39386", "metadata": { "tags": [] }, "outputs": [], "source": [ "!rm model.tar.gz" ] }, { "cell_type": "code", "execution_count": null, "id": "a8d3528f-0c75-4faf-97b0-9047f62d39cf", "metadata": { "tags": [] }, "outputs": [], "source": [ "!aws s3 rm --quiet $pretrained_model_data" ] }, { "attachments": {}, "cell_type": "markdown", "id": "3c9c3d3b-888e-4a4e-94fd-00b1b97f443f", "metadata": {}, "source": [ "## Notebook CI Test Results\n", "\n", "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n", "\n", "\n", "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-2/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ca-central-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/sa-east-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-2/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-3/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-central-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-north-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-2/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-2/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n", "\n", "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-south-1/advanced_functionality|huggingface_deploy_instructpix2pix|deploy-instructpix2pix-to-sagemaker-ir.ipynb)\n" ] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 }, { "_defaultOrder": 55, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 56, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4de.24xlarge", "vcpuNum": 96 } ], "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Data Science 3.0)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:eu-central-1:936697816551:image/sagemaker-data-science-310-v1" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.6" } }, "nbformat": 4, "nbformat_minor": 5 }