{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Deploy Models to the edge using SageMaker Edge Manager and IoT Greengrass v2\n", "\n", "1. [Introduction](#Introduction)\n", "2. [Prerequisites](#Prerequisites)\n", "3. [Setup](#Setup)\n", "4. [Create an ARM64 instance](#Create-an-ARM64-instance)\n", "5. [Prepare the Model for the Edge](#Prepare-the-Model-for-the-Edge)\n", " 1. [Model](#Model)\n", " 2. [Neo Compilation](#Neo-Compilation)\n", " 3. [Package the model for Edge Manager](#Package-the-model-for-Edge-Manager)\n", "6. [Setup Edge Device](#Setup-Edge-Device)\n", " 1. [Install GreenGrass Core](#Install-GreenGrass-Core)\n", " 2. [Create an Edge Manager device fleet](#Create-an-Edge-Manager-device-fleet)\n", " 3. [Creat the device](#Creat-the-device)\n", "7. [Build GreengrassV2 Component](#Build-GreengrassV2-Component)\n", "8. [Deploy Using Greengrass](#Deploy-Using-Greengrass)\n", "9. [SSH into the EC2 instance](#SSH-into-the-EC2-instance)\n", "10. [Clean Up](#Clean-Up)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction\n", "\n", "In this lab illustrate how you can optimize and deploy machine learning models for edge devices.\n", "\n", "1. Use an ARM64 EC2 instance to mimic an typical ARM based edge device.\n", "2. Config the edge device by installing Greengrass core to manage the edge deployment\n", "3. Prepare our model for edge using SageMaker Neo and Edge Manager (either use the model from the previous modules or run the `optional-prepare-data-and-model.ipynb` notebook to create a new model)\n", "4. Finally we will deploy a simple application that predicts a list of bird images and send the results to the cloud\n", "\n", "** Note: This Notebook was tested on Data Science Kernel in SageMaker Studio**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Prerequisites\n", "\n", "Download the notebook into your environment, and you can run it by simply execute each cell in order. To understand what's happening, you'll need:\n", "\n", "- Following Policy to your **SageMaker Studio Execution role** to properly executed this lab. Warning: the permissions set for this lab are very loose for simplicity purposes. Please follow the least privilege frame when you work on your own projects. These permissions allow user to interact with other AWS services like EC2, System Manager (SSM), IoT Core, and GreengrassV2.\n", "\n", " - AmazonEC2FullAccess\n", " - AmazonEC2RoleforSSM\n", " - AmazonSSMManagedInstanceCore\n", " - AmazonSSMFullAccess\n", " - AWSGreengrassFullAccess\n", " - AWSIoTFullAccess\n", " \n", "- Familiarity with Python\n", "- Basic understanding of SSM\n", "- Basic familiarity with AWS S3\n", "- Basic familiarity with IoT Core\n", "- Basic familiarity with GreengrassV2\n", "- Basic familiarity with AWS Command Line Interface (CLI) -- ideally, you should have it set up with credentials to access the AWS account you're running this notebook from.\n", "- SageMaker Studio is preferred for the full UI integration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup\n", "\n", "Setting up the environment, load the libraries, and define the parameter for the entire notebook." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import sagemaker\n", "from sagemaker import get_execution_role\n", "import boto3\n", "import botocore\n", "import json\n", "\n", "role = get_execution_role()\n", "sess = sagemaker.Session()\n", "region = boto3.Session().region_name" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# S3 bucket and folders for saving model artifacts.\n", "# Feel free to specify different bucket/folders here if you wish.\n", "bucket = sess.default_bucket()\n", "folder = \"BIRD-Sagemaker-Deployment\"\n", "compilation_output_sub_folder = folder + \"/compilation-output\"\n", "iot_folder = folder + \"/iot\"\n", "\n", "# S3 Location to save the model artifact after compilation\n", "s3_compilation_output_location = \"s3://{}/{}\".format(bucket, compilation_output_sub_folder)\n", "s3_compilation_output_location" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create an ARM64 instance" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will use an ARM64 EC2 instance to mimic the edge device. If you do not have permission to create/update IAM roles, you can also complete the steps below using AWS Console. Please follow the [AWS Documentation](https://docs.aws.amazon.com/efs/latest/ug/gs-step-one-create-ec2-resources.html) to create an EC3 instance.\n", "\n", "1. Select an `Arm 64 Ubuntu AMI`\n", "\n", "\n", "\n", "2. Select an `t4g.large instance type`\n", "\n", "\n", "\n", "\n", "3. Create an IAM role for your EC2 instance. follow this [AWS Documenation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html) to create a role for an AWS service.\n", "\n", "3.1 Here is teh trust policy for the role\n", "```\n", "{\n", " \"Version\": \"2012-10-17\",\n", " \"Statement\": [\n", " {\n", " \"Effect\": \"Allow\",\n", " \"Principal\": {\n", " \"Service\": \"ec2.amazonaws.com\"\n", " },\n", " \"Action\": \"sts:AssumeRole\"\n", " }\n", " ]\n", "}\n", "```\n", "3.2 Add following policies\n", "\n", "- CloudWatchAgentAdminPolicy\n", "- AmazonS3FullAccess\n", "- AmazonSSMManagedInstanceCore\n", "\n", "4. Configure the security group to allow SSH from your IP address\n", "\n", "5. Add a tag {\"Key\": \"Name\", \"Value\": \"edge-manager-notebook\"}\n", "\n", "## If you use the console, Please Skip to [next section](#neo-compile-the-model) \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create EC2 Instance in Code\n", "\n", "Again, skip this step if you created the instance using the console." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ec2_client = boto3.client(\"ec2\", region_name=region)\n", "iam_client = boto3.client(\"iam\", region_name=region)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### create the key pem file\n", "Generate key pair for EC2 instance, save the key PEM file. We can use this key with SSH to connect to the instance. But in this notebook example, we will not use SSH, instead, we will use AWS Systems Manager to send commands to the instance." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import datetime\n", "\n", "ec2_key_name = \"edge-manager-key-\" + str(datetime.datetime.now())\n", "ec2_key_pair = ec2_client.create_key_pair(\n", " KeyName=ec2_key_name,\n", ")\n", "\n", "key_pair = str(ec2_key_pair[\"KeyMaterial\"])\n", "key_pair_file = open(\"ec2-key-pair.pem\", \"w\")\n", "key_pair_file.write(key_pair)\n", "key_pair_file.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Get Ubuntu AMI Id" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ami = ec2_client.describe_images(Filters=[{'Name': 'name', 'Values': ['aws-parallelcluster-3.0.0b1-ubuntu-2004-lts-hvm-arm64-202108282001 2021-08-28T20-04-54.501Z']}])['Images'][0]['ImageId']\n", "ami" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a role and allow EC2 instance to assume role using boto3" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ec2_trust_policy = {\n", " \"Version\": \"2012-10-17\",\n", " \"Statement\": [\n", " {\n", " \"Effect\": \"Allow\",\n", " \"Principal\": {\n", " \"Service\": \"ec2.amazonaws.com\"\n", " },\n", " \"Action\": \"sts:AssumeRole\"\n", " }\n", " ]\n", "}\n", "\n", "\n", "role_name = f'Edge-Device-Role'\n", "\n", "create_role_res = iam_client.create_role(\n", " RoleName=role_name,\n", " AssumeRolePolicyDocument=json.dumps(ec2_trust_policy),\n", " Description='This is role for ec2 instance to assume',\n", " )\n", "\n", "ec2_arn= create_role_res['Role']['Arn']" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "create_inst_prof_res = iam_client.create_instance_profile(\n", " InstanceProfileName=role_name,\n", " )\n", "\n", "create_inst_prof_res" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "response = iam_client.add_role_to_instance_profile(\n", " InstanceProfileName=role_name,\n", " RoleName=role_name\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "policy_list = [\n", " 'arn:aws:iam::aws:policy/CloudWatchAgentAdminPolicy',\n", " 'arn:aws:iam::aws:policy/AmazonS3FullAccess',\n", " 'arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore',\n", " 'arn:aws:iam::aws:policy/AWSIoTFullAccess',\n", " 'arn:aws:iam::aws:policy/IAMFullAccess'\n", " \n", "]\n", "\n", "for p in policy_list:\n", " policy_attach_res = iam_client.attach_role_policy(\n", " RoleName=role_name,\n", " PolicyArn=p\n", " )\n", " \n", " print(policy_attach_res)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create the t4g large instance\n", "\n", "T4g instances are powered by Arm-based AWS Graviton2 processors. [Here](https://aws.amazon.com/ec2/instance-types/t4/) about this instance type. We are going to use it to mimic an Arm64 edge device.\n", "\n", "Previous step make take a few minutes to complete, please retry the code block below if yuo get an error about InvalidParameterValue. While the instance is spinning up, we can go to the model preparation steps next." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ec2_instance = ec2_client.run_instances(\n", " ImageId=ami,\n", " MinCount=1,\n", " MaxCount=1,\n", " InstanceType=\"t4g.large\",\n", " KeyName=ec2_key_name,\n", " IamInstanceProfile={\n", " \"Name\": role_name,\n", " },\n", " TagSpecifications=[\n", " {\n", " \"ResourceType\": \"instance\",\n", " \"Tags\": [{\"Key\": \"Name\", \"Value\": \"edge-manager-notebook\"}],\n", " }\n", " ],\n", ")\n", "\n", "instance_id = ec2_instance[\"Instances\"][0][\"InstanceId\"]\n", "print(f\"Instance ID is {instance_id}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prepare the Model for the Edge" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Model\n", "\n", "You can use the bird model use created from the previous modules or run the `optional-prepare-data-and-model.ipynb` notebook to create a new model. Update the path to your model below if necessary." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "bird_model_path = ''" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Neo Compilation\n", "Amazon SageMaker Neo enables developers to optimize machine learning (ML) models for inference on SageMaker in the cloud and supported devices at the edge.\n", "\n", "Amazon SageMaker Neo automatically optimizes machine learning models for inference on cloud instances and edge devices to run faster with no loss in accuracy. You start with a machine learning model trained in Amazon SageMaker or anywhere else. Then you choose your target hardware platform, which can be a SageMaker hosting instance or an edge device based on processors from Ambarella, Apple, ARM, Intel, MediaTek, Nvidia, NXP, Qualcomm, RockChip, Texas Instruments, or Xilinx.\n", "\n", "In this example we will be using our Bird model from previous lab.\n", "\n", "We are compiling a MobileNetV2 model in TensorFlow. The input tensor shape is `\"input_1\":[1,224,224,3]`, The edge device is our t4g instance which is a ARM64 device running Linux." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sagemaker_client = boto3.client(\"sagemaker\", region_name=region)\n", "\n", "s3_client = boto3.client(\"s3\", region_name=region)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The TF2 model is already provided with the lab. We need to upload the model to S3 first." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "bird_data_shape = '{\"input_1\":[1,224,224,3]}'\n", "bird_framework = \"TensorFlow\"\n", "\n", "target_os = \"LINUX\"\n", "target_arch = \"ARM64\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import time\n", "\n", "bird_compilation_job_name = \"Sagemaker-Bird-\" + str(time.time()).split(\".\")[0]\n", "print(\"Compilation job for %s started\" % bird_compilation_job_name)\n", "\n", "response = sagemaker_client.create_compilation_job(\n", " CompilationJobName=bird_compilation_job_name,\n", " RoleArn=role,\n", " InputConfig={\n", " \"S3Uri\": bird_model_path,\n", " \"DataInputConfig\": bird_data_shape,\n", " \"Framework\": bird_framework.upper(),\n", " },\n", " OutputConfig={\n", " \"S3OutputLocation\": s3_compilation_output_location,\n", " \"TargetPlatform\": {\"Arch\": target_arch, \"Os\": target_os},\n", " },\n", " StoppingCondition={\"MaxRuntimeInSeconds\": 900},\n", ")\n", "\n", "print(response)\n", "\n", "# Poll every 30 sec\n", "while True:\n", " response = sagemaker_client.describe_compilation_job(\n", " CompilationJobName=bird_compilation_job_name\n", " )\n", " if response[\"CompilationJobStatus\"] == \"COMPLETED\":\n", " break\n", " elif response[\"CompilationJobStatus\"] == \"FAILED\":\n", " print(response)\n", " raise RuntimeError(\"Compilation failed\")\n", " print(\"Compiling ...\")\n", " time.sleep(30)\n", "print(\"Done!\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Package the model for Edge Manager\n", "\n", "Once the model is neo compiled, we will use Edge Manager to re-package the model and create the component for Greengrass deployment." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pip install boto3 --upgrade" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "bird_packaged_model_name = \"bird-model\"\n", "bird_model_version = \"1.0\"\n", "bird_model_package = \"{}-{}.tar.gz\".format(bird_packaged_model_name, bird_model_version)\n", "model_component_version = \"1.0.0\"\n", "model_component_name = 'gg-bird-component'" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "bird_packaging_job_name = bird_compilation_job_name + \"-packaging1\"\n", "\n", "response = sagemaker_client.create_edge_packaging_job(\n", " EdgePackagingJobName=bird_packaging_job_name,\n", " CompilationJobName=bird_compilation_job_name,\n", " ModelName=bird_packaged_model_name,\n", " ModelVersion=bird_model_version,\n", " RoleArn=role,\n", " OutputConfig={\n", " 'S3OutputLocation': f\"s3://{bucket}/artifacts/{model_component_name}/{model_component_version}/\",\n", " \"PresetDeploymentType\":\"GreengrassV2Component\",\n", " \"PresetDeploymentConfig\":'{\"ComponentName\":\"'+ model_component_name +'\", \"ComponentVersion\":\"'+ model_component_version +'\"}'\n", " }\n", "\n", ")\n", "\n", "print(response)\n", "\n", "# Poll every 30 sec\n", "while True:\n", " job_status = sagemaker_client.describe_edge_packaging_job(\n", " EdgePackagingJobName=bird_packaging_job_name\n", " )\n", " if job_status[\"EdgePackagingJobStatus\"] == \"COMPLETED\":\n", " break\n", " elif job_status[\"EdgePackagingJobStatus\"] == \"FAILED\":\n", " print(job_status)\n", " raise RuntimeError(\"Edge Packaging failed\")\n", " print(\"Packaging ...\")\n", " time.sleep(30)\n", " \n", "bird_model_data = job_status[\"ModelArtifact\"]\n", "\n", "print(f'Package data is located here: {bird_model_data}')\n", "\n", "print(\"Done!\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ssm_client = boto3.client(\"ssm\", region_name=region)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "iot_thing_name = \"bird-edge-thing-\"+ str(time.time()).split(\".\")[0]\n", "iot_thing_group = f\"{iot_thing_name}-group\"\n", "tes_role = f'SageMaker-TES-Role-{str(time.time()).split(\".\")[0]}'\n", "tes_alias = f'{tes_role}-Alias'\n", "\n", "print(f'Thing group name: {iot_thing_group}')\n", "print(f'TES Alias: {tes_alias}')\n", "\n", "run_command = 'sudo -E java -Droot=\"/greengrass/v2\" -Dlog.store=FILE -jar ./GreengrassInstaller/lib/Greengrass.jar ' \n", "run_command += f'--aws-region {region} --thing-name {iot_thing_name} --thing-group-name {iot_thing_group} '\n", "run_command += f'--tes-role-name {tes_role} --tes-role-alias-name {tes_alias} '\n", "run_command += '--component-default-user ggc_user:ggc_group --provision true --setup-system-service true'\n", "\n", "response = ssm_client.send_command(\n", " InstanceIds=[instance_id],\n", " DocumentName=\"AWS-RunShellScript\",\n", " OutputS3BucketName=bucket,\n", " OutputS3KeyPrefix=folder,\n", " Parameters={\n", " \"commands\": [\n", " \"#!/bin/bash\",\n", " \"wget -O- https://apt.corretto.aws/corretto.key | sudo apt-key add -\",\n", " \"sudo add-apt-repository 'deb https://apt.corretto.aws stable main'\",\n", " \"sudo apt-get update; sudo apt-get install -y java-1.8.0-amazon-corretto-jdk\",\n", " \"sudo apt-get update && sudo apt-get install python3-venv -y\",\n", " \"curl -s https://d2s8p88vqu9w66.cloudfront.net/releases/greengrass-nucleus-latest.zip > greengrass-nucleus-latest.zip\",\n", " \"unzip greengrass-nucleus-latest.zip -d GreengrassInstaller && rm greengrass-nucleus-latest.zip\",\n", " run_command\n", " ]\n", " },\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Check the status of your command, they may take a few minutes." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ssm_client.get_command_invocation(\n", " CommandId=response[\"Command\"][\"CommandId\"],\n", " InstanceId=instance_id,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create an Edge Manager device fleet" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create an Edge Manager fleet to manage multiple devices at once. There is a per account limit of 10 fleet. If you need go beyond 10, please contact your account team to increase the limit.\n", "\n", "In addition to creating the fleet, you also need to configure an IAM role that will be assumed by the credentials provider on behalf of the devices in your device fleet. \n", "\n", "**Notice**: The name of the role must start with `SageMaker`.\n", "\n", "Go to [IAM console](https://console.aws.amazon.com/iam), create role for IoT, attach the following policies:\n", "\n", "- AmazonSageMakerEdgeDeviceFleetPolicy" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "iot_trust_policy = {\n", " \"Version\": \"2012-10-17\",\n", " \"Statement\": [\n", " {\n", " \"Effect\": \"Allow\",\n", " \"Principal\": {\"Service\": \"credentials.iot.amazonaws.com\"},\n", " \"Action\": \"sts:AssumeRole\"\n", " },\n", " {\n", " \"Effect\": \"Allow\",\n", " \"Principal\": {\"Service\": \"sagemaker.amazonaws.com\"},\n", " \"Action\": \"sts:AssumeRole\"\n", " }\n", " ]\n", "}\n", "\n", "update_role_res = iam_client.update_assume_role_policy(\n", " RoleName=tes_role,\n", " PolicyDocument=json.dumps(iot_trust_policy),\n", " )\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "policy_list = [\n", " 'arn:aws:iam::aws:policy/service-role/AmazonSageMakerEdgeDeviceFleetPolicy',\n", " 'arn:aws:iam::aws:policy/AmazonS3FullAccess'\n", "]\n", "\n", "for p in policy_list:\n", " policy_attach_res = iam_client.attach_role_policy(\n", " RoleName=tes_role,\n", " PolicyArn=p\n", " )\n", " \n", " print(policy_attach_res)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(tes_role)\n", "role_resp = iam_client.get_role(\n", " RoleName=tes_role,\n", ")\n", "\n", "fleet_arn = role_resp['Role']['Arn']\n", "\n", "fleet_arn" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It may take a second or two to generate a role, so if you get an role doesn't exist error, just run the code below again." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "device_fleet_name = \"bird-classification-device-fleet-\" + str(time.time()).split(\".\")[0]\n", "\n", "sagemaker_client.create_device_fleet(\n", " DeviceFleetName=device_fleet_name,\n", " RoleArn=fleet_arn,\n", " OutputConfig={\"S3OutputLocation\": s3_compilation_output_location},\n", ")\n", "\n", "print(f'Device Fleet Name: {device_fleet_name}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creat the device\n", "associate the device to the IOT Core and the fleet" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "device_name = (\n", " \"sagemaker-edge-demo-device\" + str(time.time()).split(\".\")[0]\n", ") # device name should be 36 charactors\n", "\n", "sagemaker_client.register_devices(\n", " DeviceFleetName=device_fleet_name,\n", " Devices=[\n", " {\n", " \"DeviceName\": device_name,\n", " \"IotThingName\": iot_thing_name,\n", " \"Description\": \"this is a sample virtual device\",\n", " }\n", " ],\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Build GreengrassV2 Component\n", "\n", "AWS IoT Greengrass components are software modules that you deploy to Greengrass core devices. Components can represent applications, runtime installers, libraries, or any code that you would run on a device. You can define components that depend on other components. For example, you might define a component that installs Python, and then define that component as a dependency of your components that run Python applications. When you deploy your components to your fleets of devices, Greengrass deploys only the software modules that your devices require.\n", "\n", "There are the compnents for this lab:\n", "- Bird-Model-Test-Script2\n", "- Bird-Model-ARM-TF2\n", "- aws.greengrass.SageMakerEdgeManager (public component)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "def upload_artifacts(file_name, object_name=None):\n", " if object_name is None:\n", " object_name = file_name\n", " try:\n", " s3_client.upload_file(file_name, bucket, object_name)\n", " \n", " except Exception as e:\n", " print(\n", " \"Failed to upload the artifacts to the bucket {} with key {}.\\nException: {}\".format(\n", " bucket, object_name, e\n", " )\n", " )\n", " exit(1)\n", " \n", " \n", " print(\n", " \"Successfully uploaded the artifacts to the bucket {} with key {}.\".format(\n", " bucket, object_name\n", " )\n", " )\n", " return f\"s3://{bucket}/{object_name}\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Upload component artifacts files to S3**\n", "\n", "Components can have any number of artifacts, which are component binaries. Artifacts can include scripts, compiled code, static resources, and any other files that a component consumes. Components can also consume artifacts from component dependencies." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import shutil\n", "\n", "# zip the script files in build folder.\n", "shutil.make_archive('build/image_classification', 'zip', 'build/image_classification')\n", "\n", "script_component_name = \"Bird-Model-Test-Script2\"\n", "script_component_version = \"1.0.1\"\n", "model_component_path = f\"artifacts/{script_component_name}/{script_component_name}\"\n", "\n", "\n", "s3_path = dict()\n", "\n", "for file in os.listdir('build'):\n", "\n", " file_path = os.path.join('build', file)\n", " object_name = f\"{model_component_path}/{file}\"\n", " \n", " if \".zip\" in file_path:\n", " s3_path['code'] = upload_artifacts(file_path, object_name)\n", " if \".sh\" in file_path:\n", " s3_path['build'] = upload_artifacts(file_path, object_name)\n", " else:\n", " pass\n", " \n", "s3_path" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Build the component Recipe**\n", "\n", "Every component contains a recipe file, which defines its metadata. The recipe also specifies the component's configuration parameters, component dependencies, lifecycle, and platform compatibility. The component lifecycle defines the commands that install, run, and shut down the component. For more information, see [AWS IoT Greengrass component recipe reference.](https://docs.aws.amazon.com/greengrass/v2/developerguide/component-recipe-reference.html)\n", "\n", "You can define recipes in JSON or YAML format.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import json\n", "\n", "application_recipe = {\n", " \"RecipeFormatVersion\": \"2020-01-25\",\n", " \"ComponentName\": script_component_name,\n", " \"ComponentVersion\": script_component_version,\n", " \"ComponentDescription\": \"Sagemaker edge manager python example of image classification deployed using Greengrassv2.\",\n", " \"ComponentPublisher\": \"AWS\",\n", " \"ComponentConfiguration\": {\n", " \"DefaultConfiguration\": {\n", " \"accessControl\": {\n", " \"aws.greengrass.ipc.mqttproxy\": {\n", " \"com.greengrass.SageMakerEdgeManager.ImageClassification:mqttproxy:1\": {\n", " \"policyDescription\": \"Allows access to publish via topic gg/sageMakerEdgeManager/image-classification.\",\n", " \"operations\": [\n", " \"aws.greengrass#PublishToIoTCore\"\n", " ],\n", " \"resources\": [\n", " \"gg/sageMakerEdgeManager/image-classification\"\n", " ]\n", " }\n", " }\n", " },\n", " \"ImageName\": \"Cardinal_0001_17057.jpg\",\n", " \"InferenceInterval\": \"3600\",\n", " \"PublishResultsOnTopic\": \"gg/sageMakerEdgeManager/image-classification\"\n", " }\n", " },\n", " \"Manifests\": [\n", " {\n", " \"Platform\": {\n", " \"os\": \"linux\",\n", " \"architecture\": \"aarch64\"\n", " },\n", " \"Lifecycle\": { \n", " \"setEnv\": {\n", " \"DEFAULT_SMEM_IC_IMAGE_DIR\": \"{artifacts:decompressedPath}/image_classification/image_classification/images/\",\n", " },\n", " \"run\": {\n", " \"script\": \"bash {artifacts:path}/installer.sh\\npython3 {artifacts:decompressedPath}/image_classification/image_classification/inference.py\"\n", " }\n", " },\n", " \"Artifacts\": [\n", " {\n", " \"Uri\": s3_path['code'],\n", " \"Unarchive\": \"ZIP\",\n", " \"Permission\": {\n", " \"Read\": \"OWNER\",\n", " \"Execute\": \"NONE\"\n", " }\n", " },\n", " {\n", " \"Uri\": s3_path['build'],\n", " \"Unarchive\": \"NONE\",\n", " \"Permission\": {\n", " \"Read\": \"OWNER\",\n", " \"Execute\": \"NONE\"\n", " }\n", " }\n", " ]\n", " }\n", " ]\n", "}\n", "\n", "\n", "greengrass_client = boto3.client(\"greengrassv2\", region_name=region)\n", "greengrass_resp = greengrass_client.create_component_version(inlineRecipe=json.dumps(application_recipe))\n", "\n", "greengrass_resp" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Deploy Using Greengrass\n", "\n", "To deploy your components (console)\n", "\n", "1. In the [AWS IoT Greengrass console navigation menu](https://console.aws.amazon.com/greengrass), choose Deployments, and then choose the deployment for your target device that you want to revise.\n", "\n", "2. On the deployment page, choose **Create**. If you are updating an existing deployment then choose Revise and then choose Revise deployment.\n", "\n", "3. On the Specify target page, select the target IoT group and choose Next.\n", "\n", "4. On the Select components page, do the following:\n", "\n", " - Under My components, select the following components:\n", "\n", " - Bird-Model-Test-Script2\n", "\n", " - Bird-Model-ARM-TF2\n", "\n", " - Under Public components, turn off the Show only selected components toggle, and then select the aws.greengrass.SageMakerEdgeManager component.\n", "\n", " - Choose Next.\n", "\n", "5. On the Configure components page, select the aws.greengrass.SageMakerEdgeManager component and do the following.\n", "\n", " - Choose Configure component.\n", "\n", " - Under Configuration update, in Configuration to merge, enter the following configuration.\n", "```\n", "{\n", " \"DeviceFleetName\": \"device-fleet-name\",\n", " \"BucketName\": \"BUCKET-Start-With-Sagemaker\",\n", " \"FolderPrefix\": \"BIRD-Sagemaker-Edge\"\n", "}\n", "```\n", " - Replace device-fleet-name with the name of the edge device fleet that you created, and replace DOC-EXAMPLE-BUCKET with the name of the S3 bucket that is associated with your device fleet.\n", "\n", " - Choose Confirm, and then choose Next.\n", "\n", "6. On the Configure advanced settings page, keep the default configuration settings, and choose Next.\n", "\n", "7. On the Review page, choose Deploy\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Deploy Using Code" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "iot_client = boto3.client('iot')\n", "response = iot_client.describe_thing_group(\n", " thingGroupName=iot_thing_group\n", ")\n", "\n", "target_arn = response['thingGroupArn']\n", "target_arn" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "response = greengrass_client.create_deployment(\n", " targetArn=target_arn,\n", " deploymentName=\"bird-edge-demo2\",\n", " components={\n", " script_component_name: {\n", " \"componentVersion\": script_component_version,\n", " },\n", " model_component_name:{\n", " \"componentVersion\":model_component_version,\n", " },\n", " \"aws.greengrass.SageMakerEdgeManager\":{\n", " \"componentVersion\":\"1.0.3\",\n", " 'configurationUpdate': {\n", " 'merge': json.dumps({\n", " \"DeviceFleetName\": device_fleet_name,\n", " \"BucketName\": bucket,\n", " \"FolderPrefix\":folder\n", " })\n", " },\n", " \n", " }\n", " },\n", ")\n", "response" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## View Your Deployment in GreenGrass Console\n", "\n", "1. Go to [Greengrass Console](https://console.aws.amazon.com/greengrass/)\n", "\n", "2. View the deployment you just made in the deployment dashoboard\n", " \n", " - This shows you the deployment details: components, target devices, and deployment status\n", " - You can also see the device status to indicate the device is running in a healthy state.\n", "\n", "\n", "\n", "3. Click inot Core device to get more detail on the device status\n", " - You can get status details on each components\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## SSH into the EC2 instance\n", "\n", "In addition to monitor the device in the console, you can also SSH and see how different component interact with each other. We recommend to use AWS instance connect to SSH into the device. [Here](https://www.youtube.com/watch?v=lxSNeF7BAII&ab_channel=StephaneMaarek) is a nice quick tutorial on how to do that.\n", "\n", "Basically you need to open up SSH inbound rule in your security group. To use instance connect the security group on the EC2 instance should allow inbound connections from the EC2 Instance Connect service, not your own IP address.\n", "\n", "You can obtain the relevant IP address range from [AWS IP Address Ranges](https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html).\n", "\n", "For example, here is the range for the Sydney region:\n", "```\n", "{\n", " \"ip_prefix\": \"13.239.158.0/29\",\n", " \"region\": \"ap-southeast-2\",\n", " \"service\": \"EC2_INSTANCE_CONNECT\"\n", "},\n", "```\n", "\n", "Your components deployed on the device will be located at `/greengrass/v2/packages/` directory.\n", "\n", "All the logs will be located at `/greengrass/v2/logs/`. There are 3 logs that are important to monitor:\n", "\n", "- Script Component Log: `/greengrass/v2/logs/Bird-Model-Test-Script2.log`\n", "- Greengrass log: `/greengrass/v2/logs/greengrass.log`\n", "- Edge Manager log: `/greengrass/v2/logs/aws.greengrass.SageMakerEdgeManager.log`\n", "\n", "All these directory needs admin user permission to access, so the easiest way is to `sudo su` when you SSH onto the device." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clean Up\n", "Don't Forget to clean up the resource you created after you are done." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Data Science)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/datascience-1.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 4 }