{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "6bfa072a-5d8b-4a9e-9058-f7e01bf5a8e1", "metadata": { "tags": [] }, "source": [ "# Amazon SageMaker Model Governance - Model Cards Model Registry integration" ] }, { "attachments": {}, "cell_type": "markdown", "id": "f6a7daa0-105e-4976-9eb9-363e8c6f78c9", "metadata": {}, "source": [ "---\n", "\n", "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n", "\n", "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-2/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "---" ] }, { "attachments": {}, "cell_type": "markdown", "id": "63d1e3d2-e7f6-4f47-b170-5afd811f48e1", "metadata": {}, "source": [ "This notebook walks you through the new feature of Amazon SageMaker Model Cards Model Registry Integration. To learn about the existing features and for more information on model cards, see [Model Cards](https://docs.aws.amazon.com/sagemaker/latest/dg/model-cards.html) in the _Amazon SageMaker Developer Guide_.\n", "\n", "Amazon SageMaker Model Cards give you the ability to create a centralized, customizable fact-sheet to document critical details about your machine learning (ML) models. Use model cards to keep a record of model information, such as intended uses, risk ratings, training details, evaluation metrics, and more for streamlined governance and reporting. \n", "\n", "In this example, you will create a model package along with a model card to document model package details along the way. Learn how to create a model card by associating model package using the Amazon SageMaker Python SDK.\n", "\n", "---\n", "## Contents\n", "\n", "1. [Setup](#Setup)\n", "1. [Create a model package](#ModelPackage)\n", "1. [Create Model Card](#ModelCard)\n", "1. [Cleanup](#Cleanup)\n", "\n", "---\n", "## Setup\n", "To begin, you must specify the following information:\n", "- The IAM role ARN used to give SageMaker training and hosting access to your data. The following example uses the SageMaker execution role.\n", "- The SageMaker session used to manage interactions with Amazon SageMaker Model Card API methods.\n", "- The S3 URI (`bucket` and `prefix`) where you want to store training artifacts, models, and any exported model card PDFs. This S3 bucket should be in the same Region as your Notebook Instance, training, and hosting configurations. The following example uses the default SageMaker S3 bucket and creates a default SageMaker S3 bucket if one does not already exist.\n", "- The S3 session used to manage interactions with Amazon S3 storage." ] }, { "cell_type": "code", "execution_count": null, "id": "6ff3c234", "metadata": {}, "outputs": [], "source": [ "! pip install --upgrade sagemaker" ] }, { "cell_type": "code", "execution_count": null, "id": "c461da94-8746-44da-8f9d-21fe2a3c8652", "metadata": {}, "outputs": [], "source": [ "import boto3\n", "from sagemaker.session import Session\n", "from sagemaker import get_execution_role\n", "\n", "role = get_execution_role()\n", "\n", "sagemaker_session = Session()\n", "\n", "bucket = sagemaker_session.default_bucket()\n", "prefix = \"model-card-registry-sample-notebook\"\n", "region = sagemaker_session.boto_region_name\n", "\n", "print(bucket)\n", "print(region)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "32cb6670-d9d5-449e-a01c-c7ddb774af2f", "metadata": {}, "source": [ "Next, import the necessary Python libraries." ] }, { "cell_type": "code", "execution_count": null, "id": "7144d8d4-1290-4290-baa9-4e41bbef9ec5", "metadata": {}, "outputs": [], "source": [ "import io\n", "import os\n", "import time\n", "import numpy as np\n", "from six.moves.urllib.parse import urlparse\n", "from pprint import pprint\n", "import sagemaker\n", "from sagemaker.image_uris import retrieve\n", "import sagemaker.amazon.common as smac\n", "from sagemaker.model_card import (\n", " ModelCard,\n", " ModelPackage,\n", " IntendedUses,\n", " ModelCardStatusEnum,\n", ")\n", "from sagemaker.model_card.model_card import ModelApprovalStatusEnum" ] }, { "attachments": {}, "cell_type": "markdown", "id": "c8c67dd2-f277-417c-bca1-f1e211a0d9a2", "metadata": {}, "source": [ "---\n", "## Prepare a Model (Model package)\n", "The following code creates an example model package trained on a synthetic dataset. The target variable (0 or 1) is the second variable in the tuple.\n", "\n", "### 1. Prepare the training data\n", "The code will upload example data to your S3 bucket." ] }, { "cell_type": "code", "execution_count": null, "id": "0bfcad27-81d4-4c95-80f0-424067237e37", "metadata": {}, "outputs": [], "source": [ "s3 = boto3.client(\"s3\", region_name=region)\n", "\n", "# synthetic data\n", "raw_data = (\n", " (0.5, 0),\n", " (0.75, 0),\n", " (1.0, 0),\n", " (1.25, 0),\n", " (1.50, 0),\n", " (1.75, 0),\n", " (2.0, 0),\n", " (2.25, 1),\n", " (2.5, 0),\n", " (2.75, 1),\n", " (3.0, 0),\n", " (3.25, 1),\n", " (3.5, 0),\n", " (4.0, 1),\n", " (4.25, 1),\n", " (4.5, 1),\n", " (4.75, 1),\n", " (5.0, 1),\n", " (5.5, 1),\n", ")\n", "training_data = np.array(raw_data).astype(\"float32\")\n", "labels = training_data[:, 1]\n", "\n", "# upload data to S3 bucket\n", "buf = io.BytesIO()\n", "smac.write_numpy_to_dense_tensor(buf, training_data, labels)\n", "buf.seek(0)\n", "boto3.resource(\"s3\").Bucket(bucket).Object(os.path.join(prefix, \"train\")).upload_fileobj(buf)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "53fdc706-1c26-4b7a-b7b6-77dc353d9e2c", "metadata": {}, "source": [ "### 2. Create a training job\n", "Train a binary classification model with the training data from the previous step." ] }, { "cell_type": "code", "execution_count": null, "id": "722aff01-defd-4a9e-8b06-c0d7b320cc0d", "metadata": {}, "outputs": [], "source": [ "s3_train_data = f\"s3://{bucket}/{prefix}/train\"\n", "output_location = f\"s3://{bucket}/{prefix}/output\"\n", "container = retrieve(\"linear-learner\", sagemaker_session.boto_session.region_name)\n", "estimator = sagemaker.estimator.Estimator(\n", " container,\n", " role=role,\n", " instance_count=1,\n", " instance_type=\"ml.m4.xlarge\",\n", " output_path=output_location,\n", " sagemaker_session=sagemaker_session,\n", ")\n", "estimator.set_hyperparameters(feature_dim=2, mini_batch_size=10, predictor_type=\"binary_classifier\")\n", "estimator.fit({\"train\": s3_train_data})\n", "print(f\"Training job name: {estimator.latest_training_job.name}\")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "eb31a7f3-3633-43f9-8220-a31cebdc14cc", "metadata": {}, "source": [ "### 2. Create a model package" ] }, { "cell_type": "code", "execution_count": null, "id": "13385e3b-898f-4272-8c6a-5b030d016465", "metadata": {}, "outputs": [], "source": [ "# create a model package group\n", "model_package_group_name = \"test-notebook-model-package-group\"\n", "sagemaker_session.sagemaker_client.create_model_package_group(\n", " ModelPackageGroupName=model_package_group_name\n", ")\n", "\n", "# describe training job to get model_data_url and image\n", "training_job_name = estimator.latest_training_job.name\n", "training_job = sagemaker_session.sagemaker_client.describe_training_job(\n", " TrainingJobName=training_job_name\n", ")\n", "\n", "model_data_url = training_job[\"ModelArtifacts\"][\"S3ModelArtifacts\"]\n", "image = training_job[\"AlgorithmSpecification\"][\"TrainingImage\"]\n", "\n", "# model package request input object\n", "create_model_package_input_dict = {\n", " \"ModelPackageGroupName\": model_package_group_name,\n", " \"ModelPackageDescription\": \"Test model package registered for integ test\",\n", " \"ModelApprovalStatus\": ModelApprovalStatusEnum.PENDING_MANUAL_APPROVAL,\n", " \"InferenceSpecification\": {\n", " \"Containers\": [{\"Image\": image, \"ModelDataUrl\": model_data_url}],\n", " \"SupportedContentTypes\": [\"text/csv\"],\n", " \"SupportedResponseMIMETypes\": [\"text/csv\"],\n", " },\n", "}\n", "\n", "model_pkg = sagemaker_session.sagemaker_client.create_model_package(\n", " **create_model_package_input_dict\n", ")\n", "print(\"Model package ARN:\", model_pkg[\"ModelPackageArn\"])" ] }, { "attachments": {}, "cell_type": "markdown", "id": "2fb11fce-ef8b-4db3-9dbe-79bcc6b781d6", "metadata": {}, "source": [ "---\n", "## Create Model Card\n", "Document your model package details in an Amazon SageMaker Model Card using the SageMaker Python SDK." ] }, { "attachments": {}, "cell_type": "markdown", "id": "bf1473e8-2ce2-4bbb-aea9-1b1f8f895091", "metadata": {}, "source": [ "### 1. Collect model package details\n", "Automatically collect basic model package information like model package ARN, model package group name, model package approval status, and model package's inference specification information." ] }, { "cell_type": "code", "execution_count": null, "id": "56af1e2b-688b-4ddb-980b-24098695a25a", "metadata": {}, "outputs": [], "source": [ "model_package_details = ModelPackage.from_model_package_arn(\n", " model_package_arn=model_pkg[\"ModelPackageArn\"],\n", " sagemaker_session=sagemaker_session,\n", ")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "46bc452c-55d8-4395-ad83-7403b2e48d7d", "metadata": {}, "source": [ "### 2. Initialize a model card\n", "Initialize a model card with the model package details collected in the previous step. When associating model package to a model card, model card will try to auto discover information like training job details and evaluation job details only if there are information like model artifacts and model metrics available in model package. Additionally, it will also try to carry over some additional information like business details to this model card from the previously created the most recent model card that is associated with this particular model package group." ] }, { "cell_type": "code", "execution_count": null, "id": "19966fff-86d1-4c06-bc84-05dca057c1ec", "metadata": {}, "outputs": [], "source": [ "model_card_name = \"sample-model-card-with-model-package\"\n", "my_card = ModelCard(\n", " name=model_card_name,\n", " status=ModelCardStatusEnum.DRAFT,\n", " model_package_details=model_package_details,\n", " intended_uses=IntendedUses(\n", " purpose_of_model=\"Test model card.\",\n", " intended_uses=\"Not used except this test.\",\n", " ),\n", " sagemaker_session=sagemaker_session,\n", ")\n", "\n", "# Check auto-discovered data\n", "print(\"Auto discovered training job details\")\n", "print(\" arn: \", my_card.training_details.training_job_details.training_arn)\n", "print(\n", " \" environment: \",\n", " my_card.training_details.training_job_details.training_environment.container_image,\n", ")\n", "print(\n", " \" metrics: \",\n", " [(m.name, m.value) for m in my_card.training_details.training_job_details.training_metrics],\n", ")\n", "print(\n", " \" hyper-parameters: \",\n", " [(h.name, h.value) for h in my_card.training_details.training_job_details.hyper_parameters],\n", ")\n", "\n", "my_card.create()\n", "print(f\"Model card {my_card.name} is successfully created with id {my_card.arn}\")\n", "\n", "time.sleep(\n", " 3\n", ") # sleep 3s to wait for model card being populated in the search service which is required by information inheritance" ] }, { "attachments": {}, "cell_type": "markdown", "id": "c891c7f9", "metadata": {}, "source": [ "#### Information inheritance\n", "Additionally, new model card will also try to carry over some additional information like business details, intended uses, additional information to this model card from the previously created the most recent model card that is associated with this particular model package group. In this example, check out the intended uses that is automatically carried over from the previous model card." ] }, { "cell_type": "code", "execution_count": null, "id": "36164c44", "metadata": {}, "outputs": [], "source": [ "# create another model package under the same model package group as sample-model-card-with-model-package\n", "create_model_package_input_dict2 = {\n", " \"ModelPackageGroupName\": model_package_group_name,\n", " \"ModelPackageDescription\": \"Test model package registered for integ test\",\n", " \"ModelApprovalStatus\": ModelApprovalStatusEnum.PENDING_MANUAL_APPROVAL,\n", " \"InferenceSpecification\": {\n", " \"Containers\": [{\"Image\": image, \"ModelDataUrl\": model_data_url}],\n", " \"SupportedContentTypes\": [\"text/csv\"],\n", " \"SupportedResponseMIMETypes\": [\"text/csv\"],\n", " },\n", "}\n", "\n", "model_pkg2 = sagemaker_session.sagemaker_client.create_model_package(\n", " **create_model_package_input_dict2\n", ")\n", "model_package_details2 = ModelPackage.from_model_package_arn(\n", " model_package_arn=model_pkg2[\"ModelPackageArn\"],\n", " sagemaker_session=sagemaker_session,\n", ")\n", "\n", "# create another model card with the new model package\n", "model_card_name2 = \"sample-model-card-with-model-package2\"\n", "my_card2 = ModelCard(\n", " name=model_card_name2,\n", " status=ModelCardStatusEnum.DRAFT,\n", " model_package_details=model_package_details2,\n", " sagemaker_session=sagemaker_session,\n", ")\n", "\n", "print(\n", " \"Information carried over from the latest model card, i.e. sample-model-card-with-model-package, associated with the same model package group\"\n", ")\n", "print(\" Intended uses: \")\n", "print(\" purpose_of_model: \", my_card2.intended_uses.purpose_of_model)\n", "print(\" intended_uses: \", my_card2.intended_uses.intended_uses)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "fe88e044", "metadata": {}, "source": [ "---\n", "## Cleanup (Optional)\n", "Delete the following resources:\n", "1. The model card\n", "2. The model package\n", "3. The model package group" ] }, { "cell_type": "code", "execution_count": null, "id": "7d447849", "metadata": {}, "outputs": [], "source": [ "my_card.delete()\n", "\n", "response = sagemaker_session.sagemaker_client.list_model_packages(\n", " ModelPackageGroupName=model_package_group_name\n", ")\n", "\n", "for package in response[\"ModelPackageSummaryList\"]:\n", " sagemaker_session.sagemaker_client.delete_model_package(\n", " ModelPackageName=package[\"ModelPackageArn\"]\n", " )\n", "\n", "sagemaker_session.sagemaker_client.delete_model_package_group(\n", " ModelPackageGroupName=model_package_group_name\n", ")" ] }, { "attachments": {}, "cell_type": "markdown", "id": "ba5c2fdd", "metadata": {}, "source": [ "## Notebook CI Test Results\n", "\n", "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n", "\n", "\n", "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-2/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ca-central-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/sa-east-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-2/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-3/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-central-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-north-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-2/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-2/sagemaker_model_governance|model_card_with_model_package.ipynb)\n", "\n", "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-south-1/sagemaker_model_governance|model_card_with_model_package.ipynb)\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "a2e1bf9e", "metadata": {}, "source": [] } ], "metadata": { "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" } }, "nbformat": 4, "nbformat_minor": 5 }