{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# End to End example to manage lifecycle of ML models deployed on the edge using SageMaker Edge Manager + GreenGrass v2\n",
"\n",
"**SageMaker Studio Kernel**: Data Science"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contents \n",
"\n",
"* Use Case\n",
"* Workflow\n",
"* Setup\n",
"* Building and Deploying the ML Model\n",
"* Deploy Wind Turbine application to EC2 with Greengrass V2\n",
"* Cleanup\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use Case\n",
"\n",
"The challenge we're trying to address here is to detect anomalies in the components of a Wind Turbine. Each wind turbine has many sensors that reads data like:\n",
" - Internal & external temperature\n",
" - Wind speed\n",
" - Rotor speed\n",
" - Air pressure\n",
" - Voltage (or current) in the generator\n",
" - Vibration in the GearBox (using an IMU -> Accelerometer + Gyroscope)\n",
"\n",
"So, depending on the types of the anomalies we want to detect, we need to select one or more features and then prepare a dataset that 'explains' the anomalies. We are interested in three types of anomalies:\n",
" - Rotor speed (when the rotor is not in an expected speed)\n",
" - Produced voltage (when the generator is not producing the expected voltage)\n",
" - Gearbox vibration (when the vibration of the gearbox is far from the expected)\n",
" \n",
"All these three anomalies (or violations) depend on many variables while the turbine is working. Thus, in order to address that, let's use a ML model called [Autoencoder](https://en.wikipedia.org/wiki/Autoencoder), with correlated features. This model is unsupervised. It learns the latent representation of the dataset and tries to predict (regression) the same tensor given as input. The strategy then is to use a dataset collected from a normal turbine (without anomalies). The model will then learn **'what is a normal turbine'**. When the sensors readings of a malfunctioning turbine is used as input, the model will not be able to rebuild the input, predicting something with a high error and detected as an anomaly.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Workflow\n",
"\n",
"In this example, you will create a robust end-to-end solution that manages the lifecycle of ML models deployed to a wind turbine fleet to detect the anomalies in the operation using SageMaker Edge Manager.\n",
"\n",
" - Prepare a ML model\n",
" - download a pre-trained model;\n",
" - compile the ML model with SageMaker Neo for Linux x86_64;\n",
" - create a deployment package using SageMaker Edge Manager;\n",
" - download/unpack the deployment package;\n",
" - Download/unpack a package with the IoT certificates, required by the agent; \n",
" - Download/unpack **SageMaker Edge Agent** for Linux x86_64;\n",
" - Generate the protobuf/grpc stubs (.py scripts) - with these files we will send requests via unix:// sockets to the agent; \n",
" - Using some helper functions, we're going to interact with the agent and do some tests.\n",
"\n",
"The following diagram shows the resources, required to run this experiment and understand how the agent works and how to interact with it. \n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We need an AWS account role with SageMaker access. This role is used to give SageMaker access to S3, launch an EC2 instance and create components and deployments in Greengrass."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import sagemaker\n",
"from sagemaker import get_execution_role\n",
"import boto3\n",
"import botocore\n",
"import json\n",
"import time\n",
"\n",
"role = get_execution_role()\n",
"sess = sagemaker.Session()\n",
"region = boto3.Session().region_name\n",
"\n",
"# S3 bucket and folders for saving model artifacts.\n",
"# Feel free to specify different bucket/folders here if you wish.\n",
"bucket = sess.default_bucket()\n",
"folder = \"DEMO-Sagemaker-Edge\"\n",
"compilation_output_sub_folder = folder + \"/compilation-output\"\n",
"iot_folder = folder + \"/iot\"\n",
"\n",
"# S3 Location to save the model artifact after compilation\n",
"s3_compilation_output_location = \"s3://{}/{}\".format(bucket, compilation_output_sub_folder)\n",
"\n",
"print('Role: ', role.split('/')[-1])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Locate the above printed sagemaker role from [IAM console](https://console.aws.amazon.com/iam), find and attach the following policies to role:**\n",
"\n",
"- IAMFullAccess\n",
"\n",
"**Also add the following trust relationship**\n",
"```\n",
"{\n",
" \"Version\": \"2012-10-17\",\n",
" \"Statement\": [\n",
" {\n",
" \"Effect\": \"Allow\",\n",
" \"Principal\": {\n",
" \"Service\": \"sagemaker.amazonaws.com\"\n",
" },\n",
" \"Action\": \"sts:AssumeRole\"\n",
" },\n",
" {\n",
" \"Effect\": \"Allow\",\n",
" \"Principal\": {\n",
" \"Service\": \"iam.amazonaws.com\"\n",
" },\n",
" \"Action\": \"sts:AssumeRole\"\n",
" }\n",
" ]\n",
"}\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"## setup all the IAM and Policies needed\n",
"from setup_utils import setup_roles_and_policies\n",
"\n",
"iot_device_role_name = 'SageMakerTESRole' + str(time.time()).split(\".\")[0]\n",
"ec2_role_name = setup_roles_and_policies(iot_device_role_name)\n",
"print('Done')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Launch EC2 Instance"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As mentioned earlier, this EC2 instance is used in place of an Edge device for running the agent software. Generate key pair for EC2 instance, save the key PEM file. We can use this key with SSH to connect to the instance. But in this notebook example, we will not use SSH, instead, we will use AWS Systems Manager to send commands to the instance."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ec2_client = boto3.client(\"ec2\", region_name=region)\n",
"\n",
"key_pairs = ec2_client.describe_key_pairs()\n",
"key_names = list(map(lambda x: x[\"KeyName\"], key_pairs[\"KeyPairs\"]))\n",
"\n",
"key_name = \"ec2-key-pair\"\n",
"\n",
"if key_name in key_names:\n",
" ec2_key_pair = ec2_client.delete_key_pair(\n",
" KeyName=key_name,\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ec2_key_pair = ec2_client.create_key_pair(\n",
" KeyName=key_name,\n",
")\n",
"\n",
"try:\n",
" key_pair = str(ec2_key_pair[\"KeyMaterial\"])\n",
" key_pair_file = open(\"ec2-key-pair.pem\", \"w\")\n",
" key_pair_file.write(key_pair)\n",
" key_pair_file.close()\n",
"except:\n",
" pass\n",
"\n",
"print('Done')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Launch an EC2 C5 instance. In this example we will use aws deep learning AMI."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ami = ec2_client.describe_images(Filters=[{'Name': 'name', 'Values': ['Deep Learning AMI (Ubuntu 18.04) Version 36.0']}])['Images'][0]['ImageId']\n",
"\n",
"ec2_instance = ec2_client.run_instances(\n",
" ImageId=ami,\n",
" MinCount=1,\n",
" MaxCount=1,\n",
" InstanceType=\"c5.large\",\n",
" KeyName=key_name,\n",
" IamInstanceProfile={\"Name\": ec2_role_name},\n",
")\n",
"\n",
"instance_id = ec2_instance[\"Instances\"][0][\"InstanceId\"] # will used for running inference later\n",
"print(instance_id)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Compiling/Packaging/Deploying our ML model to our edge devices\n",
"\n",
"In this below section you will :\n",
"\n",
" - Compile/Optimize your pre-trained model to your edge device (Linux X86_64) using [SageMaker NEO](https://docs.aws.amazon.com/sagemaker/latest/dg/neo.html)\n",
" - Create a deployment package with a signed model + the runtime used by SageMaker Edge Agent to load and invoke the optimized model\n",
" - Deploy the package using IoT Jobs\n",
"\n",
"\n",
"Invoking SageMaker NEO to compile the pre-trained model. To know how this model was trained please refer to the training notebook [here](https://github.com/aws-samples/amazon-sagemaker-edge-manager-workshop/blob/main/lab/02%20-%20Training%20with%20Pytorch.ipynb). \n",
"\n",
"Upload the pre-trained model to S3 bucket"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model_file = open(\"model/model.tar.gz\", \"rb\")\n",
"boto3.Session().resource(\"s3\").Bucket(bucket).Object('model/model.tar.gz').upload_fileobj(model_file)\n",
"print(\"Model successfully uploaded!\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It will compile the model for targeted hardware and OS with SageMaker Neo service. It will also include the [deep learning runtime](https://github.com/neo-ai/neo-ai-dlr) in the model package."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"sm_client = boto3.client(\"sagemaker\", region_name=region)\n",
"compilation_job_name = 'wind-turbine-anomaly-%d' % int(time.time()*1000)\n",
"sm_client.create_compilation_job(\n",
" CompilationJobName=compilation_job_name,\n",
" RoleArn=role,\n",
" InputConfig={\n",
" 'S3Uri': 's3://%s/model/model.tar.gz' % bucket,\n",
" 'DataInputConfig': '{\"input0\":[1,6,10,10]}',\n",
" 'Framework': 'PYTORCH'\n",
" },\n",
" OutputConfig={\n",
" 'S3OutputLocation': 's3://%s/wind_turbine/optimized/' % bucket, \n",
" 'TargetPlatform': { 'Os': 'LINUX', 'Arch': 'X86_64' }\n",
" },\n",
" StoppingCondition={ 'MaxRuntimeInSeconds': 900 }\n",
")\n",
"while True:\n",
" resp = sm_client.describe_compilation_job(CompilationJobName=compilation_job_name) \n",
" if resp['CompilationJobStatus'] in ['STARTING', 'INPROGRESS']:\n",
" print('Running...')\n",
" else:\n",
" print(resp['CompilationJobStatus'], compilation_job_name)\n",
" break\n",
" time.sleep(5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Package Keras Model\n",
"\n",
"It will sign the model and create a deployment package with:\n",
"1. The optimized model\n",
"1. Model Metadata\n",
"1. Create a Greengrass V2 model component\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import time\n",
"model_version = '1.0'\n",
"model_name = 'WindTurbineAnomalyDetection'\n",
"model_component_version = \"1.0.0\"\n",
"model_component_name = \"com.model.WindTurbineAnomalyDetection\"\n",
"\n",
"# remove component if it is already exists\n",
"ggv2_client = boto3.client('greengrassv2')\n",
"account_id = role.split(\":\")[4]\n",
"try:\n",
" ggv2_client.delete_component(arn='arn:aws:greengrass:{}:{}:components:{}:versions:{}'.format(region, account_id, model_component_name, model_component_version))\n",
"except:\n",
" print('component does not exists')\n",
" \n",
"\n",
"# edge package job\n",
"edge_packaging_job_name='wind-turbine-anomaly-%d' % int(time.time()*1000)\n",
"resp = sm_client.create_edge_packaging_job(\n",
" EdgePackagingJobName=edge_packaging_job_name,\n",
" CompilationJobName=compilation_job_name,\n",
" ModelName=model_name,\n",
" ModelVersion=model_version,\n",
" RoleArn=role,\n",
" OutputConfig={\n",
" \"PresetDeploymentType\": \"GreengrassV2Component\",\n",
" \"PresetDeploymentConfig\": json.dumps(\n",
" {\"ComponentName\": model_component_name, \"ComponentVersion\": model_component_version}\n",
" ),\n",
" 'S3OutputLocation': s3_compilation_output_location\n",
" }\n",
")\n",
"\n",
"# Poll every 30 sec\n",
"while True:\n",
" job_status = sm_client.describe_edge_packaging_job(\n",
" EdgePackagingJobName=edge_packaging_job_name\n",
" )\n",
" if job_status[\"EdgePackagingJobStatus\"] == \"COMPLETED\":\n",
" break\n",
" elif job_status[\"EdgePackagingJobStatus\"] == \"FAILED\":\n",
" raise RuntimeError(\"Edge Packaging failed\")\n",
" print(\"Packaging ...\")\n",
" time.sleep(30)\n",
" \n",
"print('Done')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### Setup Greengrass\n",
"\n",
"SageMaker Edge Manager can use AWS IoT Greengrass to deploy the agent, the model and the inference application to the edge device.\n",
"\n",
"AWS IoT Greengrass provides all the necessary features to manage applications on remote devices in a secure and scalable way. To learn more about Greengrass, head to the [What is AWS IoT Greengrass?](https://docs.aws.amazon.com/greengrass/v2/developerguide/what-is-iot-greengrass.html). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ssm_client = boto3.client(\"ssm\", region_name=region)\n",
"device_name = \"demo-device\" + str(time.time()).split(\".\")[0]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = ssm_client.send_command(\n",
" InstanceIds=[instance_id],\n",
" DocumentName=\"AWS-RunShellScript\",\n",
" OutputS3BucketName=bucket,\n",
" OutputS3KeyPrefix=folder,\n",
" Parameters={\n",
" \"commands\": [\n",
" \"#!/bin/bash\",\n",
" \"sudo apt update\",\n",
" \"sudo apt-get install python3-venv -y\",\n",
" \"curl -s https://d2s8p88vqu9w66.cloudfront.net/releases/greengrass-nucleus-latest.zip > greengrass-nucleus-latest.zip && unzip greengrass-nucleus-latest.zip -d GreengrassCore\",\n",
" f'sudo -E java -Droot=\"/greengrass/v2\" -Dlog.store=FILE -jar ./GreengrassCore/lib/Greengrass.jar --aws-region {region} --thing-name {device_name} --tes-role-name {iot_device_role_name} --tes-role-alias-name {iot_device_role_name}Alias --thing-group-name GreengrassSMEdgeManagerGroup --component-default-user ggc_user:ggc_group --provision true --setup-system-service true --deploy-dev-tools true',\n",
" ]\n",
" },\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Need to wait for one or two minutes before below command will show `'Status': 'Success'`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"time.sleep(100)\n",
"ssm_client.get_command_invocation(\n",
" CommandId=response[\"Command\"][\"CommandId\"],\n",
" InstanceId=instance_id,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Create Edge Manager Device Fleet"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from setup_utils import modify_device_role\n",
"\n",
"role_arn = modify_device_role(iot_device_role_name)\n",
"\n",
"device_fleet_name = \"demo-device-fleet\" + str(time.time()).split(\".\")[0]\n",
"\n",
"sm_client.create_device_fleet(\n",
" DeviceFleetName=device_fleet_name,\n",
" RoleArn=role_arn,\n",
" OutputConfig={\"S3OutputLocation\": \"s3://{}\".format(bucket)}\n",
")\n",
"\n",
"print(device_fleet_name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Register device to the fleet"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"sm_client.register_devices(\n",
" DeviceFleetName=device_fleet_name,\n",
" Devices=[\n",
" {\n",
" \"DeviceName\": device_name,\n",
" \"IotThingName\": device_name,\n",
" \"Description\": \"this is a sample virtual device\",\n",
" }\n",
" ],\n",
")\n",
"\n",
"print('Done')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup Wind turbine application as Greengrass component\n",
"\n",
"Next, upload the python script, dataset and the protobuf libraries to an S3 bucket so that they can be referenced by the component recipe and from where they will be downloaded by Greengrass on the device. We use the SageMaker bucket for this."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!mkdir -p data\n",
"!curl https://aws-ml-blog.s3.amazonaws.com/artifacts/monitor-manage-anomaly-detection-model-wind-turbine-fleet-sagemaker-neo/dataset_wind_turbine.csv.gz -o data/dataset_wind.csv.gz"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"folder = \"com.sagemaker.windturbine/1.0.0\"\n",
"sess.upload_data(\"app_ggv2/entry_point.py\", bucket, folder)\n",
"sess.upload_data(\"app_ggv2/agent_pb2.py\", bucket, folder)\n",
"sess.upload_data(\"app_ggv2/agent_pb2_grpc.py\", bucket, folder)\n",
"sess.upload_data(\"app_ggv2/edgeagentclient.py\", bucket, folder)\n",
"sess.upload_data(\"app_ggv2/turbine.py\", bucket, folder)\n",
"sess.upload_data(\"app_ggv2/windfarm.py\", bucket, folder)\n",
"sess.upload_data(\"data/dataset_wind.csv.gz\", bucket, folder)\n",
"sess.upload_data(\"statistics/raw_std.npy\", bucket, folder)\n",
"sess.upload_data(\"statistics/mean.npy\", bucket, folder)\n",
"sess.upload_data(\"statistics/std.npy\", bucket, folder)\n",
"sess.upload_data(\"statistics/thresholds.npy\", bucket, folder)\n",
"\n",
"recipe = f'''\n",
"---\n",
"RecipeFormatVersion: 2020-01-25\n",
"ComponentName: com.sagemaker.windturbine\n",
"ComponentVersion: 1.0.0\n",
"ComponentDescription: Sagemaker Edge Manager Python example\n",
"ComponentPublisher: Amazon Web Services, Inc.\n",
"ComponentDependencies:\n",
" aws.greengrass.SageMakerEdgeManager:\n",
" VersionRequirement: '>=1.0.0'\n",
" DependencyType: HARD\n",
" com.model.WindTurbineAnomalyDetection:\n",
" VersionRequirement: '~1.0.0'\n",
" DependencyType: HARD\n",
"Manifests:\n",
" - Platform:\n",
" os: linux\n",
" architecture: \"/amd64|x86/\"\n",
" Lifecycle:\n",
" Install: |-\n",
" python3 -m venv venv\n",
" . venv/bin/activate\n",
" pip install pip --upgrade\n",
" pip install wheel \n",
" pip3 install grpcio==1.38.1\n",
" pip3 install grpcio-tools==1.38.1\n",
" pip3 install protobuf\n",
" pip3 install Pillow\n",
" pip3 install numpy\n",
" pip3 install pandas\n",
" pip3 install PyWavelets\n",
" Run:\n",
" Script: |- \n",
" export MODEL_PATH=\"{{com.model.WindTurbineAnomalyDetection:work:path}}\"\n",
" export ARTIFACTS_PATH=\"{{artifacts:path}}\"\n",
" . venv/bin/activate \n",
" python3 -u {{artifacts:path}}/entry_point.py \n",
" Artifacts:\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/entry_point.py\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/agent_pb2.py\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/agent_pb2_grpc.py\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/edgeagentclient.py\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/turbine.py\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/windfarm.py\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/dataset_wind.csv.gz\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/raw_std.npy\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/mean.npy\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/std.npy\n",
" - URI: s3://{bucket}/com.sagemaker.windturbine/1.0.0/thresholds.npy\n",
"'''\n",
"\n",
"print(recipe)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### As a final step, create the component. You can use the [AWS Greengrass Console](https://console.aws.amazon.com/iot/home?#/greengrass/v2/components/create) to create a new component. Go to **Component>Create component**, select `Enter recipe as YAML` and copy and paste the YAML above. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Deploy the application"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"## Run this cell in case some of the dependencies was not setup correctly on the EC2\n",
"cat_log_out = ssm_client.send_command(\n",
" InstanceIds=[instance_id],\n",
" DocumentName=\"AWS-RunShellScript\",\n",
" OutputS3BucketName=bucket,\n",
" OutputS3KeyPrefix=folder,\n",
" Parameters={\n",
" \"commands\": [\n",
" \"sudo apt-get install python3-venv -y\",\n",
" ]\n",
" },\n",
")\n",
"\n",
"time.sleep(15)\n",
"output = ssm_client.get_command_invocation(\n",
" CommandId=cat_log_out[\"Command\"][\"CommandId\"],\n",
" InstanceId=instance_id,\n",
")[\"StandardOutputContent\"]\n",
"print(output)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once the application component has been created, it can be deployed to the device. \n",
"\n",
"1. In the [AWS Console](https://console.aws.amazon.com/iot/home#/greengrass/v2/components/private) select the `com.sagemaker.windturbine` component, and choose **Deploy**.\n",
"2. Select the deployment named `Deployment for GreengrassSMEdgeManagerGroup` and choose `Next`\n",
"3. Toggle the selector on the `Public components` pane, search for `SageMakerEdgeManager`, and selct it\n",
"4. Choose **Next**\n",
"5. Select the `aws.greengrass.SageMakerEdgeManager` component and choose **Configure component**\n",
"6. Replace the **Configuration to merge** content with the following json. Don't forget to change the placeholder to the actual values.\n",
"```json\n",
"{\n",
"\t\"DeviceFleetName\": ,\n",
"\t\"BucketName\": \n",
"}\n",
"```\n",
"7. Choose **Confirm**\n",
"3. Choose **Next** until you reach the last screen. \n",
"4. Choose **Deploy**.\n",
"\n",
"You can check the status of the deployment by clicking on the device name `GreengrassSMEdgeManagerDevice` in the Target core devices pane and then selecting Deployments tab."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"cat_log_out = ssm_client.send_command(\n",
" InstanceIds=[instance_id],\n",
" DocumentName=\"AWS-RunShellScript\",\n",
" OutputS3BucketName=bucket,\n",
" OutputS3KeyPrefix=folder,\n",
" Parameters={\n",
" \"commands\": [\n",
" \"sudo tail -30 /greengrass/v2/logs/com.sagemaker.windturbine.log\",\n",
" ]\n",
" },\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"output = ssm_client.get_command_invocation(\n",
" CommandId=cat_log_out[\"Command\"][\"CommandId\"],\n",
" InstanceId=instance_id,\n",
")[\"StandardOutputContent\"]\n",
"print(output)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clean Up"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Stop the EC2 instance"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ec2_client.stop_instances(InstanceIds=[instance_id])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Deregister device and delete device fleet"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"sm_client.deregister_devices(DeviceFleetName=device_fleet_name, DeviceNames=[device_name])\n",
"\n",
"sm_client.delete_device_fleet(DeviceFleetName=device_fleet_name)"
]
}
],
"metadata": {
"instance_type": "ml.t3.medium",
"interpreter": {
"hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
},
"kernelspec": {
"display_name": "Python 3 (Data Science)",
"language": "python",
"name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/datascience-1.0"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.10"
}
},
"nbformat": 4,
"nbformat_minor": 4
}