{ "cells": [ { "cell_type": "markdown", "id": "aad18d80", "metadata": {}, "source": [ "# amazon-sagemaker-inference-using-tenstorhub-ready-model\n", "\n", "This notebook is for building and deploying the code-sample as a BYOC (Bring your own container) as an Amazon SageMaker endpoint for object detection.\n", "\n", "The model is a ready model downloaded directly from [Tensor Hub](https://www.tensorflow.org/hub/)\n", "\n", "To use this notebook, you need to run it in AWS, preferably in Amazon SageMaker, with permissions to Amazon Elastic Container Registry, and Amazon Sagemaker" ] }, { "cell_type": "markdown", "id": "11386b42", "metadata": {}, "source": [ "## Prerequisite\n", "\n", "Lets configure some variables we will use later\n", "\n", "`SVC` will be the name of the ECR container repository name\n", "`TAG` will be the tag of the container image for proper versioning" ] }, { "cell_type": "code", "execution_count": null, "id": "723084d7", "metadata": {}, "outputs": [], "source": [ "# ECR Repository name\n", "SVC='amazon-sagemaker-inference-using-tenstorhub-ready-model'\n", "TAG='1.0'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will add additional variables that will be used next" ] }, { "cell_type": "code", "execution_count": null, "id": "393a7ef8", "metadata": {}, "outputs": [], "source": [ "import boto3\n", "import sagemaker\n", "\n", "sess = sagemaker.Session()\n", "s3_bucket = sess.default_bucket()\n", "sts = boto3.client(\"sts\")\n", "account_id = sts.get_caller_identity()[\"Account\"]\n", "aws_region = boto3.session.Session().region_name\n", "ecr_url = \"{}.dkr.ecr.{}.amazonaws.com\".format(account_id, aws_region)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create a docker repository in [AWS ECR](https://aws.amazon.com/ecr/), the repository will store the docker image that we are about to build and push to the repository." ] }, { "cell_type": "code", "execution_count": null, "id": "187faaaf", "metadata": {}, "outputs": [], "source": [ "!aws ecr --region $aws_region create-repository --repository-name $SVC > /dev/null" ] }, { "cell_type": "markdown", "id": "11febb64", "metadata": {}, "source": [ "## Build" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will now build the a docker image according to [`Dockerfile`](./Dockerfile) and we will tag the container image in the build command" ] }, { "cell_type": "code", "execution_count": null, "id": "f8c361c9", "metadata": {}, "outputs": [], "source": [ "full_repository_url = \"{}/{}:{}\".format(ecr_url, SVC, TAG)\n", "!docker build -t $full_repository_url ." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In order to push to Amazon ECR, we will need to authenticate using a temporary login password from IAM, this password will allow us to securely push the docker image to Amazon ECR" ] }, { "cell_type": "code", "execution_count": null, "id": "f2529fb2", "metadata": {}, "outputs": [], "source": [ "!aws ecr get-login-password --region $aws_region | docker login --username AWS --password-stdin $ecr_url" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Push docker image to the ECR repository" ] }, { "cell_type": "code", "execution_count": null, "id": "2b152be0", "metadata": {}, "outputs": [], "source": [ "!docker push $full_repository_url" ] }, { "cell_type": "markdown", "id": "d338f260", "metadata": {}, "source": [ "## Deploy\n", "\n", "After we finished building the container image, we can start working on Amazon SageMaker.\n", "We will need to follow couple of steps:\n", "\n", "1. Creating a model that points to the container image\n", "2. Creating a model endpoint configuration that contains the model, instance type etc...\n", "3. Deploy - creating an endpoint from the model and endpoint configuration\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First we will use the `create_model` to create the docker image that we stored in ECR and a model in Amazon SageMaker.\n", "We will print out the result to see what Amazon SageMaker API returns.\n", "\n", "For more details on `create_model` go to the following [documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_model)" ] }, { "cell_type": "code", "execution_count": null, "id": "7251c0a8", "metadata": {}, "outputs": [], "source": [ "from sagemaker import get_execution_role\n", "\n", "role = get_execution_role()\n", "\n", "client = boto3.client('sagemaker')\n", "\n", "response = client.create_model(\n", " ModelName=SVC,\n", " PrimaryContainer={\n", " 'ContainerHostname': SVC,\n", " 'Image': full_repository_url,\n", " 'ImageConfig': {\n", " 'RepositoryAccessMode': 'Platform',\n", " 'RepositoryAuthConfig': {\n", " 'RepositoryCredentialsProviderArn': role\n", " }\n", " }\n", " },\n", " ExecutionRoleArn=role,\n", " EnableNetworkIsolation=False\n", ")\n", "print(response)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After we've created the model using `create_model`, now we will create a model config using `create_endpoint_config`.\n", "A model config will have the `ModelName` that we've created before, with additional settings like `InstanceType` the amount of instances to Launch.\n", "\n", "We will print out the conf response to see what the API returns.\n", "\n", "For more details on `create_endpoint_config` go to the following [documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_endpoint_config)" ] }, { "cell_type": "code", "execution_count": null, "id": "8849b74a", "metadata": {}, "outputs": [], "source": [ "conf = client.create_endpoint_config(\n", " EndpointConfigName=SVC,\n", " ProductionVariants=[{\n", " 'VariantName': 'default',\n", " 'ModelName': SVC,\n", " 'InitialInstanceCount': 1,\n", " 'InitialVariantWeight': 1,\n", " 'InstanceType': 'ml.g4dn.xlarge'\n", " }]\n", ")\n", "print(conf)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After we have created a model and an endpoint config, we can launch a new Amazon SageMaker that will use the container image we've built, with the endpoint config we have set.\n", "\n", "We will print the endpoint response to see what the API returns\n", "\n", "For more details on `create_endpoint` go to the following [documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_endpoint)" ] }, { "cell_type": "code", "execution_count": null, "id": "0fa5a4dd", "metadata": {}, "outputs": [], "source": [ "endpoint = client.create_endpoint(\n", " EndpointName=SVC,\n", " EndpointConfigName=SVC\n", ")\n", "print(endpoint)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The deployment of a new endpoint takes couple of minutes, lets execute the following code block to wait until the endpoint is ready.\n", "\n", "A ready SageMaker endpoint responds `200 OK` on `/ping` requests from SageMaker, once the endpoint is ready, it's status will be `InService` than we can start invoking the endpoint for inference " ] }, { "cell_type": "code", "execution_count": null, "id": "f4e65d4e", "metadata": {}, "outputs": [], "source": [ "# Wait until the endpoint is InService\n", "import time\n", "\n", "endpoint_info = client.describe_endpoint(EndpointName=SVC)\n", "endpoint_status = endpoint_info[\"EndpointStatus\"]\n", "\n", "while endpoint_status == \"Creating\":\n", " endpoint_info = client.describe_endpoint(EndpointName=SVC)\n", " endpoint_status = endpoint_info[\"EndpointStatus\"]\n", " print(\"Endpoint status:\", endpoint_status)\n", " if endpoint_status == \"Creating\":\n", " time.sleep(10)\n", "\n", "print(\"Endpoint is in {}\".format(endpoint_status))" ] }, { "cell_type": "markdown", "id": "f39024a7", "metadata": {}, "source": [ "## Testing the endpoint\n", "\n", "The endpoint was built to download an images from Amazon S3 as an input.\n", "\n", "So let's upload the `./Naxos_Taverna.jpg` image to the default Amazon SageMaker bucket that we configured when starting this notebook as `s3_bucket` and then we will invoke the endpoint with the appropriate information." ] }, { "cell_type": "code", "execution_count": null, "id": "bb873aeb", "metadata": {}, "outputs": [], "source": [ "!aws s3 cp ./Naxos_Taverna.jpg s3://$s3_bucket/test/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can invoke the endpoint, the endpoint will do as follow:\n", "\n", "1. Download the image to its temp storage\n", "2. Run inference on the image\n", "3. Delete the image\n", "4. Return a `json` with all the objects detected, their confidence store, and their detection boxes bounding box coordinates.\n", "\n", "We will print the model prediction response.\n", "\n", ">Note, that the first invocation is slow, its the first time that `tensorflow_hub` loads the model. You can invoke it twice, to see how it will perform." ] }, { "cell_type": "code", "execution_count": null, "id": "3f1926e3", "metadata": {}, "outputs": [], "source": [ "import json\n", "\n", "runtime = boto3.client('sagemaker-runtime')\n", "\n", "payload = {\n", " \"s3_bucket\": s3_bucket,\n", " \"key_prefix\": 'test',\n", " \"file_name\": 'Naxos_Taverna.jpg'\n", "}\n", "\n", "response = runtime.invoke_endpoint(\n", " EndpointName=SVC,\n", " Body=json.dumps(payload),\n", " ContentType=\"application/json\"\n", ")\n", "\n", "model_prediction = json.loads(response[\"Body\"].read())\n", "print(model_prediction)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's see the image we are going to use for this sample" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.core.display import HTML\n", "\n", "HTML('\"Naxos_Taverna.jpg\"'\n",Naxos_Taverna.jpgThe image has been downloaded from '\n", " 'https://commons.wikimedia.org/wiki/File:Naxos_Taverna.jpg, '\n", " 'License')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we have the results, let's test how the model performed detecting objects in the image.\n", "\n", "We will use `pillow` to draw on the image, and we will only draw a bounding box for objects with a confidence score higher than 0.5" ] }, { "cell_type": "code", "execution_count": null, "id": "a411bd57", "metadata": {}, "outputs": [], "source": [ "!pip install pillow\n", "\n", "from PIL import Image, ImageDraw, ImageColor, ImageFont\n", "import numpy as np\n", "\n", "conf_score = 0.5\n", "\n", "def draw_bounding_box_on_image(image, ymin, xmin, ymax, xmax, color, font, thickness=1, display_str_list=()):\n", " draw = ImageDraw.Draw(image)\n", " im_width, im_height = image.size\n", " (left, right, top, bottom) = (xmin * im_width, xmax * im_width,\n", " ymin * im_height, ymax * im_height)\n", " draw.line([(left, top), (left, bottom), (right, bottom), (right, top),\n", " (left, top)],\n", " width=thickness,\n", " fill=color)\n", "\n", " # If the total height of the display strings added to the top of the bounding\n", " # box exceeds the top of the image, stack the strings below the bounding box\n", " # instead of above.\n", " display_str_heights = [font.getsize(ds)[1] for ds in display_str_list]\n", " # Each display_str has a top and bottom margin of 0.05x.\n", " total_display_str_height = (1 + 2 * 0.05) * sum(display_str_heights)\n", "\n", " if top > total_display_str_height:\n", " text_bottom = top\n", " else:\n", " text_bottom = top + total_display_str_height\n", " # Reverse list and print from bottom to top.\n", " for display_str in display_str_list[::-1]:\n", " text_width, text_height = font.getsize(display_str)\n", " margin = np.ceil(0.05 * text_height)\n", " draw.rectangle([(left, text_bottom - text_height - 2 * margin),\n", " (left + text_width, text_bottom)],\n", " fill=color)\n", " draw.text((left + margin, text_bottom - text_height - margin),\n", " display_str,\n", " fill='red',\n", " font=font)\n", " text_bottom -= text_height - 2 * margin\n", "\n", "\n", "def draw_bounding_box(image_path, inferred_image_result):\n", " colors = list(ImageColor.colormap.values())\n", " font = ImageFont.load_default()\n", " \n", " with Image.open(image_path) as img:\n", " for i in range(0, len(inferred_image_result)):\n", " if float(inferred_image_result[i]['confidence']) >= conf_score:\n", " ymin = float(inferred_image_result[i]['ymin'])\n", " xmin = float(inferred_image_result[i]['xmin'])\n", " ymax = float(inferred_image_result[i]['ymax'])\n", " xmax = float(inferred_image_result[i]['xmax'])\n", " obj_class = inferred_image_result[i][\"class\"]\n", " obj_confidence = float(inferred_image_result[i][\"confidence\"])\n", "\n", " # set bounding box display string\n", " display_str = \"{}: {}%\".format(obj_class, obj_confidence * 100)\n", " \n", " draw_bounding_box_on_image(img, ymin, xmin, ymax, xmax, colors[i], font, display_str_list=[display_str])\n", "\n", " img.save('./Naxos_Taverna-boxed.jpg')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we will call the functions we configured with the image [`/Naxos_Taverna.jpg`](./Naxos_Taverna.jpg) and the `model_prediction` results" ] }, { "cell_type": "code", "execution_count": null, "id": "b805a055", "metadata": {}, "outputs": [], "source": [ "draw_bounding_box('./Naxos_Taverna.jpg', model_prediction)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lets see the image with the bounding boxes" ] }, { "cell_type": "code", "execution_count": null, "id": "b7e01c6c", "metadata": {}, "outputs": [], "source": [ "from IPython.core.display import HTML\n", "\n", "HTML('\"Naxos_Taverna-boxed.jpg\"'\n",Naxos_Taverna-boxed.jpgThe image has been downloaded from '\n", " 'https://commons.wikimedia.org/wiki/File:Naxos_Taverna.jpg, '\n", " 'License')" ] }, { "cell_type": "markdown", "id": "08182cc1", "metadata": {}, "source": [ "### Clean up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Make sure to run clean up to avoid unplanned costs of the running endpoint." ] }, { "cell_type": "code", "execution_count": null, "id": "de2ad2f4", "metadata": {}, "outputs": [], "source": [ "# Amazon SageMaker\n", "\n", "client.delete_endpoint(EndpointName=SVC)\n", "client.delete_endpoint_config(EndpointConfigName=SVC)\n", "client.delete_model(ModelName=SVC)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "534a1f84", "metadata": {}, "outputs": [], "source": [ "# ECR\n", "!aws ecr delete-repository --region $aws_region --repository-name $SVC --force" ] } ], "metadata": { "kernelspec": { "display_name": "conda_python3", "language": "python", "name": "conda_python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.13" } }, "nbformat": 4, "nbformat_minor": 5 }