{ "cells": [ { "cell_type": "markdown", "id": "d5845c0a-25c6-40e6-a3ba-2b3316bd38c2", "metadata": { "tags": [] }, "source": [ "# Step 5: Add a deployment pipeline\n", "In previous four steps you implemented an automated data processing and model building pipeline. Each run of the pipeline produces a new version of the model. This notebook implements the automated model deployment step in our ML workflow.\n", "\n", "You can use a [SageMaker MLOps project template](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-projects-templates.html) to provision a ready-to use model deployment CI/CD pipeline.\n", "\n", "This template automates the deployment of models in the SageMaker model registry to SageMaker endpoints for real-time inference. This template recognizes changes in the model registry. When a new model version is registered and approved, it automatically initiates a deployment.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "id": "6cd36377-d862-4d18-a575-860bf0c15b31", "metadata": { "tags": [] }, "outputs": [], "source": [ "import boto3\n", "import sagemaker \n", "from time import gmtime, strftime, sleep\n", "\n", "sagemaker.__version__" ] }, { "cell_type": "code", "execution_count": null, "id": "aa13831e-7aad-49ac-9ea9-10ebb9fb16c9", "metadata": { "tags": [] }, "outputs": [], "source": [ "%store -r \n", "\n", "%store\n", "\n", "try:\n", " initialized\n", "except NameError:\n", " print(\"+++++++++++++++++++++++++++++++++++++++++++++++++\")\n", " print(\"[ERROR] YOU HAVE TO RUN 00-start-here notebook \")\n", " print(\"+++++++++++++++++++++++++++++++++++++++++++++++++\")" ] }, { "cell_type": "code", "execution_count": null, "id": "d0bb83a8-521f-4405-bb4f-eee77ab56094", "metadata": { "tags": [] }, "outputs": [], "source": [ "sm = boto3.client(\"sagemaker\")" ] }, { "cell_type": "markdown", "id": "f9e91bef-93a8-4265-ad0c-d6f72fde9099", "metadata": {}, "source": [ "## Create an MLOps project\n", "Follow the same procedure as in the step 4 notebook to create a model deployment MLOps project. " ] }, { "cell_type": "markdown", "id": "7c8f3124-56f3-4d5a-b354-9e7930ba1c4d", "metadata": {}, "source": [ "### Option 1: Create project programmatically\n", "Use `boto3` to create an MLOps project via a SageMaker API." ] }, { "cell_type": "code", "execution_count": null, "id": "e23f268b-1dbe-40f9-884a-7ce63f7d1352", "metadata": { "tags": [] }, "outputs": [], "source": [ "sc = boto3.client(\"servicecatalog\")\n", "\n", "sc_provider_name = \"Amazon SageMaker\"\n", "sc_product_name = \"MLOps template for model deployment\"" ] }, { "cell_type": "code", "execution_count": null, "id": "b0eb4137-e7c7-447e-9f87-2012963ea2d6", "metadata": { "tags": [] }, "outputs": [], "source": [ "p_ids = [p['ProductId'] for p in sc.search_products(\n", " Filters={\n", " 'FullTextSearch': [sc_product_name]\n", " },\n", ")['ProductViewSummaries'] if p[\"Name\"]==sc_product_name]" ] }, { "cell_type": "code", "execution_count": null, "id": "9b8ca253-eca3-4de2-9d8a-7dcfd8dc1c0f", "metadata": { "tags": [] }, "outputs": [], "source": [ "p_ids" ] }, { "cell_type": "code", "execution_count": null, "id": "3b3c3f09-ca7d-43de-9432-c106f32d0c21", "metadata": { "tags": [] }, "outputs": [], "source": [ "# If you get any exception from this code, go to the Option 2 and create a project in Studio UX\n", "if not len(p_ids):\n", " raise Exception(\"No Amazon SageMaker ML Ops products found!\")\n", "elif len(p_ids) > 1:\n", " raise Exception(\"Too many matching Amazon SageMaker ML Ops products found!\")\n", "else:\n", " product_id = p_ids[0]\n", " print(f\"ML Ops product id: {product_id}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "38294903-3f86-4a3a-9849-40af57753c34", "metadata": { "tags": [] }, "outputs": [], "source": [ "provisioning_artifact_id = sorted(\n", " [i for i in sc.list_provisioning_artifacts(\n", " ProductId=product_id\n", " )['ProvisioningArtifactDetails'] if i['Guidance']=='DEFAULT'],\n", " key=lambda d: d['Name'], reverse=True)[0]['Id']" ] }, { "cell_type": "code", "execution_count": null, "id": "70b48d04-b558-4727-8e1f-1ac6b38dc7a4", "metadata": { "tags": [] }, "outputs": [], "source": [ "provisioning_artifact_id" ] }, { "cell_type": "code", "execution_count": null, "id": "8ae543a2-a20e-40e2-b251-c87462be9507", "metadata": { "tags": [] }, "outputs": [], "source": [ "project_name = f\"model-deploy-{strftime('%-m-%d-%H-%M-%S', gmtime())}\"" ] }, { "cell_type": "code", "execution_count": null, "id": "7ee8a72b-53d3-461a-95d4-6af781d7ce44", "metadata": { "tags": [] }, "outputs": [], "source": [ "project_parameters = [\n", " {\n", " 'Key': 'SourceModelPackageGroupName',\n", " 'Value': model_package_group_name\n", " },\n", "]" ] }, { "cell_type": "markdown", "id": "d1fef1a7-0396-4501-85c6-18ad874cabb9", "metadata": {}, "source": [ "Finally, create a SageMaker project from the service catalog product template:" ] }, { "cell_type": "code", "execution_count": null, "id": "acf05f97-ad6a-4503-9e47-b220f257cb58", "metadata": { "tags": [] }, "outputs": [], "source": [ "# create SageMaker project\n", "r = sm.create_project(\n", " ProjectName=project_name,\n", " ProjectDescription=\"Model build project\",\n", " ServiceCatalogProvisioningDetails={\n", " 'ProductId': product_id,\n", " 'ProvisioningArtifactId': provisioning_artifact_id,\n", " 'ProvisioningParameters': project_parameters\n", " },\n", ")\n", "\n", "print(r)\n", "project_id = r[\"ProjectId\"]" ] }, { "cell_type": "markdown", "id": "1f2e65a3-0185-4b33-b8b4-f3db9689dab8", "metadata": {}, "source": [ "