{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# End-to-end Edge Computer Vision (CV) Machine Learning (ML) Orchistration\n", "---\n", "**Table of Contents:**\n", "\n", "1. [Introduction](#Introduction)\n", "2. [Prerequisites](#Prerequisites)\n", "3. [Setup](#Setup)\n", "4. [Dataset](#Dataset)\n", "5. [Start a GroundTruth labeling workflow for CV](#Start-A-GroundTruth-Labeling-Workflow)\n", "6. [Pipeline Automation with S3 Lambda](#Pipeline-Automation-with-S3-Lambda)\n", "7. [Build a Training Pipeline](#Build-a-Training-Pipeline)\n", " 1. [Preprocessing Script](#Preprocessing-Script)\n", " \n", " 1. [Bring Your Own Container (BYOC)](#Bring-Your-Own-Container-(BYOC))\n", " \n", " 2. [Generate TFRecord for pipe mode](#Generate-TFRecord-for-pipe-mode)\n", " \n", " 3. [Use Fast File Mode](#Use-Fast-File-Mode)\n", " \n", " 2. [Training Script](#Training-Script)\n", " \n", " 1. [SageMaker Debugger](#SageMaker-debugger)\n", " \n", " 3. [Evaluation Script](#Evaluation-Script)\n", " \n", " 4. [Pipeline Code](#Pipeline-Code)\n", " \n", " 5. [Trigger the Pipeline](#Trigger-the-Pipeline)\n", " \n", "8. [Manual approval for deployment](#Manual-approval-for-deployment)\n", "\n", "9. [Edge Deployment W/ AWS Panorama](#Edge-Deployment-W/-AWS-Panorama)\n", " \n", " 1. [Setup device](#Setup-device)\n", " \n", " 2. [Deploy applications and models](#Deploy-applications-and-models)\n", " \n", " 3. [Setup Test Utility](#Setup-Test-Utility)\n", " \n", " 4. [Run Panorama Application](#Run-Panorama-Application\n", ")\n", " \n", "10. [Cleanup](#Clean-Up)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction\n", "\n", "In this workshop, we will walk you through a step by step process to prepare, build and train computer vision models with Amazon SageMaker and deploy them to AWS panorama for edge inference. The workshop image classification of different species of birds, but the concept can be applied to a wide range of industrial applications. We will complete the CV MLOps lifecycle covering: data labeling, continuous integrated (CI/CD) training pipeline, model updates to the edge w/ approval, and, fianlly, edge inferencing and data capture back to the cloud.\n", "\n", "## Architecture\n", "---\n", "![MLOps Pattern](statics/end2end.png)\n", "\n", "The architecture we will build during this workshop is illustrated on the right. Several key components can be highlighted:\n", "\n", "1. **Data labeling w/ SageMaker GroundTruth**: We are using the CUB_MINI.tar data which is a subset of [Caltech Birds (CUB 200 2011)](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html) dataset. The code below will spin up a GoundTruth labeling job for private/public workforce to label. The output is a fully labeled manifest file that can be ingested by the atutomated training pipeline.\n", "\n", "2. **Automated training Pipeline** Once the dataset is fully labelled, we have prepared a SageMaker pipeline that will go through steps: preprocessing, model training, model evaluation, and storing the model to model registry. At the same time, the example code will provide advance topics like streaming data with pipe mode and fast file mode, monitor training with SageMaker debugger, and distributed training to increase training effeciency and reduce training time.\n", "\n", "3. **Model Deployment & Edge Inferencing w/ AWS Panorama** With manual approval of the model in Model registry, we can kick off the edge deploymnet with a deployment lambda. That can be accomplish using the AWS Panorama API. We will also illustration the edge application development and testing using the AWSW Panorama Test Utility (Emulator)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequisites\n", "\n", "To run this notebook, you can simply execute each cell in order. To understand what's happening, you'll need:\n", "\n", "- Access to the SageMaker default S3 bucket or use your own\n", "- The S3 bucket that you use for this demo must have a CORS policy attached. To learn more about this requirement, and how to attach a CORS policy to an S3 bucket, see [CORS Permission Requirement](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-cors-update.html)\n", "- The notebook use public workforce (Mechanical Turk) for the groundTruth labeling job. That will incur some cost at **$0.012 per image**. If want to use a prviate workforce, then you must have a private workforce already created and update the code with your private workforce ARN. Please follow this [AWS documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-workforce-create-private-console.html) on how to create a private workforce\n", "- Access to Elastic Container Registry (ECR)\n", "- Familiarity with Training on Amazon SageMaker\n", "- Familiarity with SageMaker Processing Job\n", "- Familiarity with SageMaker Pipeline\n", "- Familiarity with Python\n", "- Familiarity with AWS S3\n", "- Basic familiarity with AWS Command Line Interface (CLI) -- ideally, you should have it set up with credentials to access the AWS account you're running this notebook from.\n", "- SageMaker Studio is preferred for the full UI integration" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup\n", "\n", "---\n", "import the libraries and intialize the parameters for this workshop" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import sagemaker\n", "import boto3\n", "import os\n", "import time\n", "import datetime\n", "import json\n", "import numpy as np\n", "import pprint as pp\n", "\n", "from collections import namedtuple\n", "from collections import defaultdict\n", "from collections import Counter\n", "\n", "\n", "\n", "import matplotlib.pyplot as plt\n", "\n", "sagemaker_session = sagemaker.Session()\n", "\n", "default_bucket = sagemaker_session.default_bucket() # or use your own custom bucket name\n", "region = sagemaker_session.boto_region_name\n", "account = sagemaker_session.account_id()\n", "role = sagemaker.get_execution_role()\n", "\n", "sess = sagemaker.Session()\n", "\n", "base_job_prefix = 'BirdEnd2End'" ] }, { "cell_type": "markdown", "metadata": { "jp-MarkdownHeadingCollapsed": true, "tags": [] }, "source": [ "## Dataset\n", "The dataset we are using is from [Caltech Birds (CUB 200 2011)](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html) dataset contains 11,788 images across 200 bird species (the original technical report can be found here). Each species comes with around 60 images, with a typical size of about 350 pixels by 500 pixels. Bounding boxes are provided, as are annotations of bird parts. A recommended train/test split is given, but image size data is not.\n", "\n", "![Bird Dataset](statics/birds.png)\n", "\n", "If you kept the data set from previous lab, just copy the files here. Otherwise, run the cell below to download the full dataset or download manually [here](https://course.fast.ai/datasets). Note that the file size is around 1.2 GB, and can take a while to download. If you plan to complete the entire workshop, please keep the file to avoid re-download and re-process the data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!wget 'https://s3.amazonaws.com/fast-ai-imageclas/CUB_200_2011.tgz'\n", "!tar xopf CUB_200_2011.tgz\n", "!rm CUB_200_2011.tgz" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Generating a test video for our Panorama Test Utility. We will use this video to test our Panorama application. To do this, we will need opencv. Run the cell below to install the appropriate packages if your kernel does already have them.\n", "\n", "The video `bird.mp4` will be created in `2_deployment` folder. You can review the output there." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "!pip install opencv-python\n", "!apt-get update\n", "!apt-get install ffmpeg libsm6 libxext6 -y" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import cv2\n", "import glob\n", "import random\n", "\n", "img_array = []\n", "image_dir = 'CUB_200_2011/images'\n", "\n", "subset_data = [\"013.Bobolink\", \"017.Cardinal\", \n", " \"035.Purple_Finch\", \"036.Northern_Flicker\",\n", " \"047.American_Goldfinch\",\"068.Ruby_throated_Hummingbird\",\n", " \"073.Blue_Jay\",\"087.Mallard\"]\n", "\n", "# create an array of available image files\n", "for sub_dir in os.listdir(image_dir):\n", " if sub_dir in subset_data:\n", "\n", " for filename in os.listdir(f'{image_dir}/{sub_dir}'):\n", " img_array.append(f'{image_dir}/{sub_dir}/{filename}')\n", " \n", "# all the images should be the same size, so grab teh image size from first image\n", "img = cv2.imread(img_array[0])\n", "height, width, layers = img.shape\n", "size = (width,height)\n", "\n", "output_path = '../2_deployment/bird.mp4'\n", "out = cv2.VideoWriter(output_path,cv2.VideoWriter_fourcc(*'MP4V'), 1, size)\n", "\n", "#random sample images and append as frames into the video\n", "for i in range(200):\n", " rand_index = random.randint(0,len(img_array)-1)\n", " img = cv2.imread(img_array[rand_index])\n", " out.write(img)\n", "out.release()\n", "\n", "print(f'video generated at {output_path}')" ] }, { "cell_type": "markdown", "metadata": { "jp-MarkdownHeadingCollapsed": true, "tags": [] }, "source": [ "## Start A GroundTruth Labeling Workflow\n", "\n", "---\n", "We will create a GroundTruth labeling job. This involves several steps: collecting the images we want labeled, specifying the possible label categories, creating instructions, and writing a labeling job specification. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "**Create and upload an input manifest file.** Each line in an input manifest file is an entry containing an object, or a reference to an object, to label. An entry can also contain labels from previous jobs and for some task types, additional information. [AWS Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-input-data-input-manifest.html)\n", "\n", "If you do not want to go through the manual labeling process, the code below also generate a synthetic labelled output manifest file that is ready for training. In that case, ** after you run the code below just **[skip to the next section](LambdaTrigger)**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Copy the images to our local bucket.\n", "\n", "label_map = dict()\n", "for i in range(len(subset_data)):\n", " label = subset_data[i].split('.')[-1]\n", " label_map[label] = i\n", " \n", "s3 = boto3.client('s3')\n", "\n", "input_manifest_name = 'input.manifest'\n", "output_manifest_name = 'output.manifest'\n", "\n", "# remove if file already exist\n", "if os.path.exists(input_manifest_name):\n", " os.remove(input_manifest_name)\n", " \n", "if os.path.exists(output_manifest_name):\n", " os.remove(output_manifest_name)\n", "\n", "f = open(input_manifest_name, 'a')\n", "g = open(output_manifest_name, 'a')\n", "\n", "json_body = {'labels':[]}\n", "\n", "print('Processing......\\n')\n", "\n", "for raw_label in os.listdir(image_dir):\n", " \n", " if raw_label in subset_data:\n", " \n", " label_name = raw_label.split('.')[-1]\n", "\n", " print(label_name, label_map[label_name])\n", " json_body['labels'].append({\"label\": label_name})\n", "\n", " for filename in os.listdir(f'{image_dir}/{raw_label}'):\n", " if '.jpg' in filename:\n", " key = f\"{base_job_prefix}/unlabeled/images/{filename}\"\n", " s3.upload_file(f'{image_dir}/{raw_label}/{filename}', default_bucket, key)\n", "\n", " img_path = f\"s3://{default_bucket}/{key}\"\n", " f.write('{\"source-ref\": \"' + img_path + '\"}\\n')\n", "\n", " #build output manifest, if you don't want to go through the label \n", " output_manifest = dict()\n", "\n", " output_manifest['source-ref'] = img_path\n", " output_manifest['label'] = label_map[label_name]\n", " label_metadata = dict()\n", " label_metadata['class-name'] = label_name\n", " # these are just placeholders to mimic an actual manifest file\n", " label_metadata['job-name'] = \"labeling-job/bird-image-classification-1634678978\"\n", " label_metadata['type'] = \"groundtruth/image-classification\"\n", " label_metadata['human-annotated'] = \"yes\"\n", " label_metadata['creation-date'] = str(datetime.datetime.now().isoformat(timespec='microseconds'))\n", "\n", " output_manifest['label-metadata'] = label_metadata\n", "\n", " g.write(f\"{json.dumps(output_manifest)}\\n\")\n", " \n", " \n", "f.close()\n", "g.close()\n", " \n", " \n", "input_manifest_key = f\"{base_job_prefix}/unlabeled/manifest/{input_manifest_name}\"\n", "s3.upload_file(input_manifest_name, default_bucket, input_manifest_key)\n", " \n", "s3_input_manifest = f\"s3://{default_bucket}/{input_manifest_key}\"\n", "\n", "print(\"\\nInput manifest file location:\")\n", "print(s3_input_manifest)\n", "\n", "output_manifest_key = f\"{base_job_prefix}/pipeline/manifest/{output_manifest_name}\"\n", "s3.upload_file(output_manifest_name, default_bucket, output_manifest_key)\n", " \n", "s3_output_manifest = f\"s3://{default_bucket}/{output_manifest_key}\"\n", "\n", "print(\"\\nSynthetic output manifest file location: \")\n", "print(s3_output_manifest)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "remove data to save disc space" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!rm -rf ./CUB_200_2011\n", "!rm -f attributes.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "**Specify the categories with a `label.json` file** To run an image classification labeling job, you need to decide on a set of classes the annotators can choose from. \n", "In our case, this list is `[\"Bobolink\", \"Cardinal\", \"Purple Finch\"]`. In your own job you can choose any list of up to 10 classes. We recommend the classes to be as unambiguous and concrete as possible. The categories should be mutually exclusive, with only one correct label per image. In addition, be careful to make the task as *objective* as possible, unless of course your intention is to obtain subjective labels." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pp.pprint(json_body)\n", "\n", "class_file_name = \"class_labels.json\"\n", "with open(class_file_name, \"w\") as f:\n", " json.dump(json_body, f)\n", "\n", "classes_key = f\"{base_job_prefix}/unlabeled/classes/{class_file_name}\"\n", "s3.upload_file(class_file_name, default_bucket, classes_key)\n", "\n", "s3_classes = f\"s3://{default_bucket}/{classes_key}\"\n", "\n", "s3_classes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Create the instruction template** Part or all of your images will be annotated by human annotators. It is **essential** to provide good instructions that help the annotators give you the annotations you want. he labeling instruction template" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def make_template(test_template=False, save_fname=\"instructions.template\"):\n", " template = r\"\"\"\n", " \n", " \n", " \n", "
  1. Read the task carefully and inspect the image.
    2. \n", "
  2. Read the options and review the examples provided to understand more about the labels.
    3. \n", "
  3. Choose the appropriate label that best suits the image.
\n", " \n", " \n", "

Dear Annotator, please tell me whether what you can see in the image. Thank you!

\n", " \n", " \n", " \"\"\"\n", "\n", " with open(save_fname, \"w\") as f:\n", " f.write(template)\n", " if test_template is False:\n", " print(template)\n", "\n", "template_name = \"instructions.template\"\n", "# make_template(test_template=True, save_fname=\"instructions.html\")\n", "\n", "make_template(test_template=False, save_fname=template_name)\n", "templates_key = f\"{base_job_prefix}/unlabeled/templates/{template_name}\"\n", "s3.upload_file(template_name, default_bucket, templates_key)\n", "\n", "s3_templates = f\"s3://{default_bucket}/{templates_key}\"\n", "print(f\"S3 url: {s3_templates}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "Assign key labeling job resources (4 components)\n", "\n", "**Assign pre-built resources in the labeling job** There are the ARNs for the four key components of the labeling job: 1) the workteam, 2) the annotation consolidation Lambda function, 3) the pre-labeling task Lambda function, and 4) the machine learning algorithm to perform auto-annotation.\n", "\n", "**Note: put your own private workteam arn here or uncomment the code and use mechanical turk.**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Specify ARNs for resources needed to run an image classification job.\n", "ac_arn_map = {\n", " \"us-west-2\": \"081040173940\",\n", " \"us-east-1\": \"432418664414\",\n", " \"us-east-2\": \"266458841044\",\n", " \"eu-west-1\": \"568282634449\",\n", " \"ap-northeast-1\": \"477331159723\",\n", "}\n", "\n", "prehuman_arn = \"arn:aws:lambda:{}:{}:function:PRE-ImageMultiClass\".format(\n", " region, ac_arn_map[region]\n", ")\n", "acs_arn = \"arn:aws:lambda:{}:{}:function:ACS-ImageMultiClass\".format(region, ac_arn_map[region])\n", "\n", "labeling_algorithm_specification_arn = \"arn:aws:sagemaker:{}:027400017018:labeling-job-algorithm-specification/image-classification\".format(\n", " region\n", ")\n", "\n", "#Update this code block if you want to use your own private workforce.\n", "PRIVATE_WORKFORCE = False\n", "\n", "public_workteam_arn = \"arn:aws:sagemaker:{}:394669845002:workteam/public-crowd/default\".format(region)\n", "\n", "# private_workteam_arn = \"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "Submit the Ground Truth job request" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "job_name = \"bird-image-classification-\" + str(int(time.time())).split('.')[0]\n", "\n", "# define human task configuration\n", "human_task_config = {\n", " \"AnnotationConsolidationConfig\": {\n", " \"AnnotationConsolidationLambdaArn\": acs_arn,\n", " },\n", " \"PreHumanTaskLambdaArn\": prehuman_arn,\n", " \"MaxConcurrentTaskCount\": 200, # 200 images will be sent at a time to the workteam.\n", " \"NumberOfHumanWorkersPerDataObject\": 3, # 3 separate workers will be required to label each image.\n", " \"TaskAvailabilityLifetimeInSeconds\": 21600, # Your worteam has 6 hours to complete all pending tasks.\n", " \"TaskDescription\": 'Carefully inspect the image and classify it by selecting one label from the categories provided.',\n", " \"TaskKeywords\": [\"image\", \"classification\", \"birds\"],\n", " \"TaskTimeLimitInSeconds\": 300, # Each image must be labeled within 5 minutes.\n", " \"TaskTitle\": 'What bird is this',\n", " \"UiConfig\": {\n", " \"UiTemplateS3Uri\": s3_templates,\n", " },\n", "}\n", "\n", "# Using public or private workforce. Public workforce require price info\n", "if not PRIVATE_WORKFORCE:\n", " human_task_config[\"PublicWorkforceTaskPrice\"] = {\n", " \"AmountInUsd\": {\n", " \"Dollars\": 0,\n", " \"Cents\": 1,\n", " \"TenthFractionsOfACent\": 2,\n", " }\n", " }\n", " human_task_config[\"WorkteamArn\"] = public_workteam_arn\n", "else:\n", " human_task_config[\"WorkteamArn\"] = private_workteam_arn\n", " \n", "ground_truth_request = {\n", " \"InputConfig\": {\n", " \"DataSource\": {\n", " \"S3DataSource\": {\n", " \"ManifestS3Uri\": s3_input_manifest\n", " }\n", " },\n", " \"DataAttributes\": {\n", " \"ContentClassifiers\": [\"FreeOfPersonallyIdentifiableInformation\", \"FreeOfAdultContent\"]\n", " },\n", " },\n", " \"OutputConfig\": {\n", " \"S3OutputPath\": f's3://{default_bucket}/{base_job_prefix}/labeled',\n", " },\n", " \"HumanTaskConfig\": human_task_config,\n", " \"LabelingJobName\": job_name,\n", " \"RoleArn\": role,\n", " \"LabelAttributeName\": \"category\",\n", " \"LabelCategoryConfigS3Uri\": s3_classes,\n", "}\n", "\n", "sagemaker_client = boto3.client(\"sagemaker\")\n", "sagemaker_client.create_labeling_job(**ground_truth_request)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "**Monitoring the labelling job status**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# job_name = 'bird-image-classification-1647438119'\n", "# sagemaker_client = boto3.client(\"sagemaker\")\n", "sagemaker_client.describe_labeling_job(LabelingJobName=job_name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "**Analyze Ground Truth labeling job results**\n", "\n", "After the job finishes running (**make sure `sagemaker_client.describe_labeling_job` shows the job is complete!**), it is time to analyze the results. In this section, we will gain additional insights into the results, all contained in the `output manifest`. You can find the location of the output manifest under `AWS Console > SageMaker > Labeling Jobs > [name of your job]`. We will obtain it programmatically in the cell below." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Load the output manifest's annotations.\n", "output_manifest = f\"s3://{default_bucket}/{base_job_prefix}/labeled/{job_name}/manifests/output/output.manifest\"\n", "\n", "!aws s3 cp {output_manifest} 'output.manifest'\n", "\n", "with open(\"output.manifest\", \"r\") as f:\n", " output = [json.loads(line.strip()) for line in f.readlines()]\n", "\n", "# Create data arrays.\n", "img_uris = [None] * len(output)\n", "confidences = np.zeros(len(output))\n", "groundtruth_labels = [None] * len(output)\n", "human = np.zeros(len(output))\n", "\n", "# Find the job name the manifest corresponds to.\n", "keys = list(output[0].keys())\n", "metakey = keys[np.where([(\"-metadata\" in k) for k in keys])[0][0]]\n", "jobname = metakey[:-9]\n", "\n", "# Extract the data.\n", "for datum_id, datum in enumerate(output):\n", " img_uris[datum_id] = datum[\"source-ref\"]\n", " groundtruth_labels[datum_id] = str(datum[metakey][\"class-name\"])\n", " confidences[datum_id] = datum[metakey][\"confidence\"]\n", " human[datum_id] = int(datum[metakey][\"human-annotated\"] == \"yes\")\n", "groundtruth_labels = np.array(groundtruth_labels)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Compute the number of annotations in each class.\n", "n_classes = len(set(groundtruth_labels))\n", "sorted_clnames, class_sizes = zip(*Counter(groundtruth_labels).most_common(n_classes))\n", "\n", "# Find ids of human-annotated images.\n", "human_sizes = [human[groundtruth_labels == clname].sum() for clname in sorted_clnames]\n", "class_sizes = np.array(class_sizes)\n", "human_sizes = np.array(human_sizes)\n", "\n", "plt.figure(figsize=(9, 3), facecolor=\"white\", dpi=100)\n", "plt.title(\"Annotation histogram\")\n", "plt.bar(range(n_classes), human_sizes, color=\"gray\", hatch=\"/\", edgecolor=\"k\", label=\"human\")\n", "plt.bar(\n", " range(n_classes),\n", " class_sizes - human_sizes,\n", " bottom=human_sizes,\n", " color=\"gray\",\n", " edgecolor=\"k\",\n", " label=\"machine\",\n", ")\n", "plt.xticks(range(n_classes), sorted_clnames, rotation=90)\n", "plt.ylabel(\"Annotation Count\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Pipeline Automation with S3 Lambda\n", "\n", "---\n", "There are several popular ways to trigger a SageMaker pipeline\n", "\n", "1. Use a lambda triggered by S3 event, where we monitor a S3 bucket for a new lablled manifest file. We demonstrate an example of that in [week 4 MLOps training](https://master.d2g2roqxvwdnfj.amplifyapp.com/orchestration.html)\n", "\n", "2. Use Source control. CodeCommit and especially [third-party source controls](https://aws.amazon.com/blogs/machine-learning/create-amazon-sagemaker-projects-using-third-party-source-control-and-jenkins/). \n", "\n", "3. Use EventBridge. Essentially any events support by EventBridge event bus (AWS and third party) [AWS Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/pipeline-eventbridge.html)\n", "\n", "4. Last but not least, you can trigger an existing pipeline from the SageMaker Studio UI." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# image location\n", "s3_input_data = f\"s3://{default_bucket}/{base_job_prefix}/unlabeled/images\"\n", "# labelled manifest location\n", "s3_input_manifest = f\"s3://{default_bucket}/{base_job_prefix}/pipeline/manifest\"" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## Build a Training Pipeline\n", "---\n", "\n", "The pipelines configured includes python package under SageMaker Pipelines together with the defined code for preprocessing, training, and model evaluation to automate the model training. It is easy to use such that you can simple drop in whatever input data for image classification you want and have it train a model automatically." ] }, { "cell_type": "markdown", "metadata": { "jp-MarkdownHeadingCollapsed": true, "tags": [] }, "source": [ "### Preprocessing Script\n", "\n", "---\n", "Here is teh preprocessing script. we are using script processor from sageMaker processing to split the data into train, valid, and test channels, and then build the TFRecord file for pipe mode training." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pygmentize 'pipeline/preprocess.py'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Generate TFRecord for pipe mode\n", "\n", "---\n", "TFRecord is TensorFlow's custom data format that uses Protocol Buffers to serialize/deserialize the data and store them in bytes. It takes less space to hold and transfer an ample amount of data, and it is also cross-platform supported with libraries across all program languages.\n", "\n", "With TFRecord data format, the user can also uncover the Pipe Mode training on SageMkaer. It is an input mechanism that streams the data to the training conatiners, which allows data to be downloaded from S3 to the container while training is running. For larger datasets, this dramatically improves the time to start training, as the data does not need to be first downloaded to the container. To learn more, please reference [AWS Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html#your-algorithms-training-algo-running-container-trainingdata)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Use Fast File Mode\n", "\n", "---\n", "Newly release in Oct, 2021. Amazon SageMaker now supports Fast File Mode for accessing data in training jobs. This enables high performance data access by streaming directly from Amazon S3 with no code changes from the existing File Mode. [AWS Documentation](https://aws.amazon.com/about-aws/whats-new/2021/10/amazon-sagemaker-fast-file-mode/)" ] }, { "cell_type": "markdown", "metadata": { "jp-MarkdownHeadingCollapsed": true, "tags": [] }, "source": [ "### Training Script\n", "\n", "---\n", "We are training a Tensorflow(TF) 2.x model for image classification. This is a transfer learning example using Keras MobileNetV2 model. We will be bring in our own script to training on SageMaker standard TF containers. Couple things to highlight about this custom script:\n", "\n", "1. the training uses pipe mode to stream data directly from Amazon S3 allowing faster start to model training.\n", "\n", "2. We provide an distributed training example using xxxxx to boost training effeciency for large model and datasets, and cut down on training time.\n", "\n", "3. Sagemaker debugger profiler and rules alerts are attached to the training to monitor the training for abnormal behavior" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pygmentize 'pipeline/code/train_debugger.py'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### SageMaker debugger\n", "\n", "---\n", "[Amazon SageMaker Debugger](https://docs.aws.amazon.com/sagemaker/latest/dg/train-debugger.html) debugs, monitors and profiles training jobs in real time thereby helping with detecting non-converging conditions, optimizing resource utilization by eliminating bottlenecks, improving training time and reducing costs of your machine learning models.\n", "\n", " \n", "\n", "1. **Profile config** - configure how to collect system metrics and framework metrics from your training job and save into your secured S3 bucket URI or local machine.\n", "\n", " 1. [Monitoring hardware system resource utilization](https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-configure-system-monitoring.html)\n", " \n", " 2. [Framework profiling](https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-configure-framework-profiling.html)\n", " \n", "2. **Debugger hook config** - configure how to collect output tensors from your training job and save into your secured S3 bucket URI or local machine. For more info, refer [here](https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-configure-hook.html).\n", "\n", "3. **Rules** - configure this parameter to enable Debugger built-in rules that you want to run in parallel. The rules automatically analyze your training job and find training issues. The ProfilerReport rule saves the Debugger profiling reports in your secured S3 bucket URI. For more info, refer [here](https://docs.aws.amazon.com/sagemaker/latest/dg/use-debugger-built-in-rules.html).\n", "\n", "![Debugger](statics/debugger_access.png)" ] }, { "cell_type": "markdown", "metadata": { "jp-MarkdownHeadingCollapsed": true, "tags": [] }, "source": [ "### Evaluation Script\n", "\n", "---\n", "Evaluation is also using script processor from SageMaker processing with the same container as preprocessing. It loads the test dataset and the model to calculate model metrics like Accuracy, Precision, Recall, F1, and confussion matrix. Only model with accuracy >= 80% will be registered in Model Registry.\n", "\n", "If model passes the evaluation criteria, all the metrics will be captured in Model Registry for manual approvel to deploy. The models are also automatic are automatically versioned with full lineage history, and you can compare different version of the models side-by-side as shown in image below:\n", "\n", "![Model Registry](statics/model_registry.png)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pygmentize 'pipeline/evaluation.py'" ] }, { "cell_type": "markdown", "metadata": { "jp-MarkdownHeadingCollapsed": true, "tags": [] }, "source": [ "### Pipeline Code\n", "\n", "---\n", "SageMaker Pipelines is generating a directed acyclic graph (DAG) in the form of a JSON pipeline definition. The provided python code using SageMaker Python SDK is an easy way to generate the JSON pipleine definition for you. In general, the code goes through and define each step of the pipeline, feed output from one step to the input for another. In the end, all the parameters and steps are chained together to formulate the pipeline.\n", "\n", "Couple new features to highlight:\n", "\n", "1. **Caching Pipeline Steps** SageMaker will attempt to find a previous execution of a step that was called with the same arguments in order to save time on repeated pipeline runs. [AWS Documenation](https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-caching.html)\n", "\n", "2. **Retry Policy** is a recent released feature that help you automatically retry pipeline steps after an error occurs. [AWS Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-retry-policy.html)\n", "\n", "3. **Tuning Step** is a new pipeline step that allows you to run tuning jobs in your pipeline across ranges of hyperparameters and pickout the best model combination to deploy. We provide an example in this workshop. [AWS Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-steps.html)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pygmentize 'pipeline/pipeline_tuning.py'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Trigger the Pipeline\n", "---\n", "Here is how you create/update a pipeline in code\n", "\n", "Uncomment the code to try the pipeline with the tuning step. Note: SageMaker debugger is not integrated with Tuning jobs (HPO). Therefore when you run this pipeline, debugger will not work. (Please reachout to the account team for a feature request if this is important to you)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# from pipeline.pipeline import get_pipeline\n", "from pipeline.pipeline_tuning import get_pipeline\n", "\n", "model_package_group_name = f\"{base_job_prefix}ModelGroup\" # Model name in model registry\n", "pipeline_name = f\"{base_job_prefix}Pipeline\" # SageMaker Pipeline name\n", "\n", "pipeline = get_pipeline(\n", " region=region,\n", " role=role,\n", " default_bucket=default_bucket,\n", " model_package_group_name=model_package_group_name,\n", " pipeline_name=pipeline_name,\n", " base_job_prefix=base_job_prefix\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pipeline.upsert(role_arn=role)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "execution = pipeline.start(\n", " parameters=dict(\n", " InputDataUrl=s3_input_data, # loaction of the raw data\n", " InputManifestUrl=s3_input_manifest,\n", " ProcessingInstanceCount=1,\n", "# ProcessingInstanceType=\"ml.m5.xlarge\",\n", "# TrainingInstanceCount=1,\n", "# TrainingInstanceType=\"ml.p3.2xlarge\",\n", " ModelApprovalStatus=\"PendingManualApproval\"\n", " )\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Manual approval for deployment\n", "\n", "---\n", "After you create a model version, you typically want to evaluate its performance before you deploy it. So the pipeline default the approval status to `PendingManualApproval`. You can manually update or update using API to change the status to Approved or Rejected. Here is how you mnaully update from SageMaker studio UI:\n", "\n", "![Manual Approval](statics/manual_approval.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Amazon EventBridge monitors status change events in Amazon SageMaker. EventBridge enables you to automate SageMaker and respond automatically to events such as a training job status change, endpoint status change, or **Model package state change**.\n", "\n", "Please reference [AWS Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/automating-sagemaker-with-eventbridge.html#eventbridge-model-package) documentation for the entire event of Model package state change.\n", "\n", "To automate the deployment process, You use event bridge to Invoke a **deployment Lambda function** that checks the `ModelApprovalStatus` attribute in the event. If teh status is **Approved** the Lambda will continue with the edge deployement." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Edge Deployment W/ AWS Panorama\n", "\n", "---\n", "AWS Panorama is a machine learning appliance and SDK, which enables you to add computer vision (CV) to your on-premises cameras or on new Panorama enabled cameras. AWS Panorama gives you the ability to make real-time decisions to improve your operations, by giving you compute power at the edge.\n", " \n", "To try out the full Panorama experience, you need a Camera that support RTSP stream and Panorama appliance to deployment your model and application setup on the same local network. **Knowing that not everyone will have access to the equipment, we will talk about the setup and deployment experience, and practice developing Panorama application in the Panorama test utility.**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setup device\n", "\n", "---\n", "To get started using your AWS Panorama Appliance, register it in the AWS Panorama console and update its software. During the setup process, you create an appliance resource in AWS Panorama that represents the physical appliance, and copy files to the appliance with a USB drive. The appliance uses these certificates and configuration files to connect to the AWS Panorama service. Then you use the AWS Panorama console to update the appliance's software and register cameras.\n", "\n", "[AWS Documentation](https://docs.aws.amazon.com/panorama/latest/dev/gettingstarted-setup.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Deploy applications and models\n", "\n", "---\n", "[AWS Documentation](https://docs.aws.amazon.com/panorama/latest/dev/gettingstarted-deploy.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setup Test Utility\n", "\n", "Panorama Test Utility is a set of python libraries and commandline commands, which allows you to test-run Panorama applications without Panorama appliance device. With Test Utility, you can start running sample applications and developing your own Panorama applications before preparing real Panorama appliance. Sample applications in this repository also use Test Utility.\n", "\n", "For more about the Test Utility and its current capabilities, please refer to [Introducing AWS Panorama Test Utility document](https://github.com/aws-samples/aws-panorama-samples/blob/main/docs/AboutTestUtility.md).\n", "\n", "To set up your environment for Test Utility, please refer to **[Test Utility environment setup](https://github.com/aws-samples/aws-panorama-samples/blob/main/docs/EnvironmentSetup.md)**." ] }, { "attachments": { "test_util_folder.png": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ0AAAEdCAYAAADXb7p6AAABSGlDQ1BJQ0MgUHJvZmlsZQAAKJFj\nYGASSSwoyGFhYGDIzSspCnJ3UoiIjFJgf8bAzCDIwMvAxMCTmFxc4BgQ4ANUwgCjUcG3awyMIPqy\nLsgsleJbu9b3Wt7nfs0fc3Nm6lFM9SiAKyW1OBlI/wHi9OSCohIGBsYUIFu5vKQAxO4AskWKgI4C\nsueA2OkQ9gYQOwnCPgJWExLkDGTfALIFkjMSgWYwvgCydZKQxNOR2FB7QYDXKTUvUME1wtjUwDCI\ngHtJBiWpFSUg2jm/oLIoMz2jRMERGEqpCp55yXo6CkYGRkYMDKAwh6j+HAQOS0axfQix/CUMDBbf\nGBiYJyLEkqYwMGxvY2CQuIUQU5nHwMDfwsCw7VBBYlEi3AGM31iK04yNIGweewYG1rv//3/WYGBg\nn8jA8Hfi//+/F////3cx0PzbDAwHKgH5KmGZI2SzZwAAAFZlWElmTU0AKgAAAAgAAYdpAAQAAAAB\nAAAAGgAAAAAAA5KGAAcAAAASAAAARKACAAQAAAABAAABnaADAAQAAAABAAABHQAAAABBU0NJSQAA\nAFNjcmVlbnNob3QnyiTGAAAB1mlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPHg6eG1wbWV0YSB4\nbWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iWE1QIENvcmUgNS40LjAiPgogICA8cmRm\nOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1u\ncyMiPgogICAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIgogICAgICAgICAgICB4bWxu\nczpleGlmPSJodHRwOi8vbnMuYWRvYmUuY29tL2V4aWYvMS4wLyI+CiAgICAgICAgIDxleGlmOlBp\neGVsWERpbWVuc2lvbj40MTM8L2V4aWY6UGl4ZWxYRGltZW5zaW9uPgogICAgICAgICA8ZXhpZjpV\nc2VyQ29tbWVudD5TY3JlZW5zaG90PC9leGlmOlVzZXJDb21tZW50PgogICAgICAgICA8ZXhpZjpQ\naXhlbFlEaW1lbnNpb24+Mjg1PC9leGlmOlBpeGVsWURpbWVuc2lvbj4KICAgICAgPC9yZGY6RGVz\nY3JpcHRpb24+CiAgIDwvcmRmOlJERj4KPC94OnhtcG1ldGE+CoFHoDEAAEAASURBVHgB7Z0JvFXj\n+sffRlFUGlGSrkRlKkMyhAYul7p0TXVVhNBNmkyVzHKjrsxTdSNJlPgXkSIZkpSi0ECikrEiSf3P\n9+HZd53V2nuvfc46e5999vOczz5reqf1W+/7jO9abynn3PaDD26WtzEyBAwBQ8AQMASKFoGyFN9w\n/4ZFW4uVbggYAoaAIWAI5CFQKu+3PY8MDEPAEDAEDAFDoMgRKF3kNVgFhoAhYAgYAobAnwiY0LGu\nYAgYAoaAIZA2BCSmUxS13Xnnna5UqVKuR48ermLFikVRhZVpCBgChoAhkGUIFNrS2bbxB7d17cp8\nt71y5Ur33nvvublz57rPP/883zU7MAQMAUPAEMhdBEJNJBg9erT75ptvXN++fQWpXxfOdJumj3a/\nvDXJIXSUyjU4xO3a/kpXsc0FInDI89e//lUv29YQMAQMAUMgxxEI5V7bZZddmOImUP3wwJVuw6QR\ngbD9tuwD992wLnnXh7tDrnrclTvcBE4gUHbSEDAEDIEcRSCUpaPYIFCwcMrWrOd2OriVK1NrH1d+\n30Nc6UpV3JY8gbNt0w/ulzmT3G/LF8i5mne85rB+jAwBQ8AQMAQMARAILXQ2PDdcBErlzje4nQ5q\nlRA9XG4/jb3Bbc5zw9W+74OEae2iIWAIGAKGQO4gEEroIERwmVXudENKyOBuY5LBzke3TylfosRv\nv/22+/XXX/Ml2W233dyhhx7q+vfv74444gh31llnuenTp7uPPvrI9erVK1/aojwIalv58uVdixYt\nAqv9v//7P5lowQy/J554wn366afuhhtuCEwbxcl169a5jz/+2B133HEys9Bb5oIFCxxu1OrVqwtm\nAwYMcI0bN/Ymyfj+mjVr3NKlS2Pt4Lk3atTI7bzzzrFzxWGHflC/fn1Xq1atWHPmz5/vfvvtN+mf\nsZNp2Ln33ntdgwYN3MknnxxJbRs2bHDvv/9+YFlly5Z1LVu2DLzGyajbEreiJBeY3MQ9fP/99+7A\nAw+UZ1K6dKHnVEmtlPuf//zH3XfffTKekjQl45eD+iqN+vDDD2XW8b777pu0jVdccYW78sor3V/+\n8pekaUkQKqaD+yxVgUPhuNZwwUVJPXv2dJs3b3YVKlSIFXvYYYe5Bx980G3dujV27osvvnDz5s2L\nHadjJ6hte+yxh3v++ecDq1+2bJlbtGiRXNu2bZv7/fffY+muvfZad8opp7hjjz02dq6wOwjrrl27\nuokTJ4qQ1vI436lTJzdkyBB3wgkniFDXGJ6mKQ7b2bNni2JRu3ZtaQ5CCPrvf/+bkNlJojT+ox8M\nGjTI/e1vf5NaX3jhBfevf/1L+mgamyFVwQSjYqgUuHr16tiEIsbhpk2bXLVq1aQuXo2YOXOm7Af9\ni7otQXUkO6fPAmUFBevmm28WgXzPPfe4MmXKJMyOwL300kvdXXfdFVMouKfHH3/ckR9i3GzZsiVh\nOcXpor+vatuGDx/uDjjggNiz1vNBW5Rn+EpYCiV0whYWlA6BFTXdeuutsQHtLZvOkGmK17Zk7erc\nuXO+JAjM5s2b5ztX2IO6deu6o446yr344ov5hM4bb7whzKNt27auUqVKbvz48YWtqsjyI3DmzJkj\n5X/77bfu+uuvFy2L6fnFkd555x0ROLfddptr06ZNcWxiSm2CWSvW3Nu5557r0JaTMeyUKinCxIzP\niy66yKHUQVj+p556qnv33XfjeiS0OSi1b731Vj5PC32Q+1dq1qyZe/LJJ/XQtgEIRGNTBhSciVOY\neM8880xg1WgkXbp0cZiL5513XmzgBCYuopMw944dO0obLrnkEkeHVUJbHzhwoBwyzRxX2+DBg137\n9tG5Jin873//uwgVr1U1ZcoU16FDBxE4nG/VqpX75JNPpC0c33HHHe7oo492hx9+uGiGDL6ff/5Z\n0qm7i3ezyLdw4ULJ9/XXX8vx+vXr5bgo/qFh8yzBEdfhd999J3WuXbs2Vh2uDl5Uhh577DFpf79+\n/VzTpk2lP8BEiorAEAZ31VVXubPPPjtWDa8S4PalDTzrcePGxa7hqkB54rlzHeLcI488ImVwDjey\n9/234tC39QZWrVrl+vTpI22nz4wYMcJhxSv99NNPDu2a+2AsvPnmm3rJLV++3P3zn/+U8cE1+mWU\nhHsT69jrMkKbnzRpkrhDqStef6dtCCfozDPPlH40efJkebb0P/o+VhRu6pNOOknSYQly/qmnnnLt\n2rWT8TN06FD3yy+/yHX+Pfzww5IGPG655RaHW1t5WFHjEWtEiB3adPrpp8tzoz96x5g3O9fGjBnj\nunXrFuO19E8vlfYeZMs+DxOmpz9loDAeTGA/ffnll+6CCy6Qh/7SSy+Jtg8oMMao6bPPPhO3HpYK\nPzoOBJOgDYcccogMJlyCPBylH3/8URgnxw888IDbe++93eWXXy7+YU0TxZbOj0tEtbONGzdKexBG\nSrgm1UVw++23u5dfftmNHDlS3EOvvPKK+/e//y3+6ho1aoiGSD4EKvnUvcLLwWi/uDCKktC2ET60\nBWFIG7xM7ocffhDfPW0AYwQPfnwYPfmwlIqCYL7nn3++MCoGohJ99cILLxTmNmHCBBEo1113nXv1\n1VclCX0SrFGQnn766dg5mDeKyqOPPuqWLFkiLh0uprNvS2OS/CMmyT3CpNin3bNmzYrlwg1+8MEH\nCyNmi4XPM4NQBrDGsWRx12CNqAs1VkAhdsqVK+f+8Y9/SLkoeCgc8JKDDjrIqcs2Xn+nXcSkIFxP\nF198sbiiERL0o1GjRrnjjz9erKAVK1ZIOlxt3BuWD4obrjyUy9dee02uI+ywgFFA6I/0VfqE8rCi\nxkMakfcPBZfx6v15FWLi4yg6xJ5pH8+3e/fusddotBy29F+eO4IXpWGfffYRvgdvVspKocODbtKk\nSeyHJpGIYIh77rmnxC1q1qwpIOy6664OhhU1wdTQ5PQ3duxYqYI2EFxmIBGgh4EQPwkiBM5OO+3k\naCv7URL3jVWjWiQDgEGD281PDBqwxYIkGM0Pn7bGqE488cSYq4tyYLIzZsyQYhBquOuiJpgQ2PFD\niyRgS/yETy6FIRgMDI3+wzOCQSjTC5M/bBoEMwwNy8+r2aKEEMejDxPvI/CO5szAVoIJYemghSuh\nsIA3E2WwBnCRQuns29qWRFuEIky5Xr16DsWKCT5qDZMPFyPWH2OAsUD/VmuH2BPBfWKMWIAEs1UY\nJKozlWtYEwhCyqa/ooCg5MFIE/V3BBYMFEIAMTaZyKKTRbjG2AoihAfKJgofY0IVs6lTp4pFwHik\nPyKAvJ8MSwcetJeYFDFH7089FlzHgqPPYbnSZ3v37i19mPheEGGlgi3P+KabbpIkuC+VyupOUW2Z\n+RZ1XIcb8X7pgFlXiQhJzu+YY47JlyweaPkSpXgQL6YDs4GJeJkj/nEso3QTDO2yyy4TjQRtC+0v\nyCePtoNVBBP0DgbaCyNlkgNMH2uJgYSvHxcc+RBCMN6oiXbAfGEQmO1YrBqwD1NXnTp1YsmUYXiF\nQuxiIXfon1hRtA03KW4VSC1fb//lPLEApSDrkMGuxD4CDUpn39b6E20JKmOpYY3xrOg/rVu3jmXx\nClKYKseKCRoy4wdlDGGDgoMl5B0zsYIKuEM/55nww82JFcKzwb21//77J+zvBaxSBJTmRWDpmGfr\nnSjE7D8UO6V04EFdQTwLa1wJJQnljAlISjxbrGzveNJr4KgE3gh2tf44H0roFKcp0zQaDaNq1ars\nhiK0EDRcGGymiIejLi1tQzIN2+sm0jxRbNFYmP337LPPinBA6w6i3XffXU7zGSTv4NC0ykC4jkDH\nYoKZYh1hkXgZqeYp7BZtUuMjMK2rr75aBA+WLIMWIrajTBrGEk8DLWxbEuVHq4Vx4pI555xzROMn\n6K6DFNcOkzYKS8Whb+s9EL/DlcgsSBQZrHWsOC8R+/MSAof+CKEZ8+oAblDcjXx2iz6GIhMF4fLE\nQsR1Sf/HJYtChcsda0tfbYjX31XQo/BEQdyb1wpE+cHCOOOMM6T4osYj7D00bNhQZtLqZ9CS5fPy\nNbDiGdP3lUK517BUSles4tb1z5tOm/fCZzLi3ZzvhnV13w7rEuk7OsnqjXedzsTDRKvBXwrzb5Xn\nmtFZOPHyFeQ8Pk1A1p8+AAYW2h/uBwYnQUh1kQTVs9dee4nPmQEYNaF9YAKjiWPW06mCCKaO6U8g\nnvvAgsEXjmsLohymdQ8bNiwWPAVXjnGjwHSKkmBouCzVikBIooHBuBB6WF+JMC7KtmnZuMPwhxO3\nIcgM1ggjtEuEI0wYIUpAuSCUzr6drH30F4gtQXuUPN5P8hKuWVy7jAHuGUsN16661Jg5iZKAkghp\nbNFbRkH36R/0FTwlxBgQItOmTZNxiYKUrL/Tt6DXX389ZmlilTIucNd5X9kI00Y8DvRVcKCvemN/\n6cAjTBtJg2cBQYyLDP5Jm4nZxONNxKqJ/aLw4coEH+9M3FCWDhXv2uHKvM/bfCCCByG0c4v2rmze\nOzi8h1O29j7u1wUzSeZ+zvsIKC+Flq5Y2dUc+sc5uZCGf5jh2vG91e23337u/vvvF5+pBo6ZYROV\nBuWti0AkPyW0f4QbrjSYMcwGvzLHvBujs0D8LgR8osRSCOLToaMmOjy+bIRPEGl7GKAwTYQJhI9e\nmTzHnCdorJaQbjU9aaIi/7NF6PE8cQXgZoNREbDFcsPaQiDhRtF7Yav73jYFnfNeL+w+QWf6ALEM\nBiNxPma06UBkVhDWEOS/x3jnJHHevzB9u6jvT9sCU6evoJjwQ5hgAXvrBwsULywMxgZMiXuAcPle\nc8018uOYwHVQrJFrBSGEBpMziF14ZwzSZhgrlKi/86I37cZdyiQhYlf0Oe4R64SYjHdmnLbRe/+c\n02PcjsxUJDjPGKIPqKsXha2o8dD2xdtqX4RHfPXVVzLO9L0seEDlypVjWfWeOEF6Jk2gqBKzQ6iq\nhc91oq95FlB4czHRBz8pECq378GuWp9RxfK7a0hn3HNekP5odXr+gzXTRr0PLF7NGtxUt1G8dOk4\nj8ZJe4rb2/9B945bEo0sDMZB+dN1jtmXMLKonm+m+7biRj+BOTHO4hFxQIRA0DjkPrB2lOnFK6Mw\n57F0YPAwQ5QXPyXq79wf5M1Hep5jKm1evHixMHN9fwu+wIQDXLJ4EJTSgYfWlWjLuOK5JXqueB+Y\nCcurDPH4XGhLB1OJ34GXDs9buuCPL0nzcc9tm/7n/tmp6fGuYtsucj1R4zN5LdOMiEEWtg3eTp1J\nzKgb5pgtxMAPi3Em7ynZBJhU21Zc7pl+m4gxcV+J4lnpuA9mnyWiRP09aFwmSh+vHgQVrmrioFgE\nxJbYaoxL86UDD60r0ZZxley5av5EfC6UpYOwwWWB9gJATN/0UlHMUPOWb/uGgCFgCJREBIhpMWmC\n+B7fLmMCSnERMgXBm5luxLkSTXUPZekgbPhB3pd8tFFRT4nWcm1rCBgChkBJRoB4lsa0SsJ9MjEp\nGYUSOkzL5MvNzLSJF3hOVpFdNwQMAUPAEDAEQrnXDCZDwBAwBAwBQyAKBP6YWB9FSVaGIWAIGAKG\ngCGQBAETOkkAssuGgCFgCBgC0SEgMR2+l2RkCBgChoAhYAgUNQIW0ylqhK18Q8AQMARyCIFkRoy5\n13KoM9itGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9hoAhYAjk\nEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRCffAz040Mqn/8\n29+6myZ96b7duDXocs6eq1aprBvYvo47+6hqOYuB3bghYAgUXwSy1tIxgRPcqRDCYGNkCBgChkBx\nRCBrhY5ZOPG7k2ETHxu7YggYAplFIGuFTmZhs9oNAUPAEDAECoKACZ2CoGZ5DAFDwBAwBAqEgAmd\nAsFmmQwBQ8AQMAQKgoAJnYKgZnkMAUPAEDAECoRA1k6ZLtDdWqbQCHz00Uc7pD3wwAN3OFccT9x3\n332uVKlSrkePHsWxedKmb775xvGDatSoIT85yPu3adMmV7FiRT20rSFQohAwoRPB47zxrLpSyqBn\nVkVQWuaLmDBhgnvmmWd2aEirVq2KNSOnwQicWbNmxdpe3AQPAuX+++93c+fOjbWRncMPP1ywHT16\ntAPnbBHw+W7CDgyBEAiY0AkBUrIkTfbaOVmSrLoeZOVwAzNnzpRf0M307dtXGGfQtXSd8wsc2gsV\nF8GDwOnZs6fbZZddHHjts88+YuEggBA23bp1k/YidIwMgZKKQOmSemN2X+lFQBl8emv9X21+gaNX\naBeWRXEg2ojAueOOO0RA41aDsHLMsikOT8jakA4EQlk6DFyvy0IbhqZ2wQUX6KFtcwSBwYMHy50O\nGTIkdsc///xzbD/dO/RP4iMwbr+Vxrl169Y50mTSgqB97733nlg4/nhNPIGZbhytPkMgHQgkFTor\nV66MqykywGE2xcV9kQ7AqIMYTmOPS61xnV2k6om9GsaasHj1L66kxHhiN1VMdxAmKlDOPvvsfK1U\nAZnvZAYOdNJAkEVz2WWXOX5GhkAuIJBU6CTTYNEg+cUjBllxGfjx2liQ86UCMgWdC0iWtad4jl6m\nOX78eLEsvBZP1t6cNdwQMATSgkBSoVPYVmAN8fMyq8KWmen8fgvm2T8tnL+P+CTTTSvS+gl2E5NQ\nlyrHyZSSIm1QFhWu8RvGAjGcIEJ5I/702GOP2ZTpIIDsXIlAoMiFTolAyW5CEMDVCqmggYEWNyqu\nyg1Cp3nz5jJLjTb64zrMbGOa+vHHH7/DteKGsbXHECgMAiZ0CoNejubFwimuVJxducRtrrjiCjdg\nwADXpUsXV69ePbEcmWDAu1FeK7K44mvtMgQKi4AJncIimJd/Ud6kgZJEaOKJrBi1eLz3nEkLg5hS\novbSTtqXaYGEdTNy5Eh5gfXOO+/0wicWDm5LvwWUL5EdGAIlAAETOhE8RH+MJ4IiM1pEx44dXePG\njVNqQyaFDsxaXX7xGo0VURwIodKvXz+Z4q0z2rB4TNgUh6djbUgHAiZ00oFyFtaRSSGSKly8L5Zt\n5P/eWra139prCBQUAfsiQUGRs3yGgCFgCBgCKSNgQidlyCyDIWAIGAKGQEERyFqhU62SeQbjPXTD\nJh4ydt4QMAQyjUBSoRNFADaKMvxADWxfxxlz9aPiBBOwMTIEDAFDoDgiwJdbtudRwrbxpjSfX082\nQ8hfCMKGb2LFewPbn96ODQFDwBAwBLIbgf/7v/9LeAOhhE7CEuyiIWAIGAKGgCHwJwLJhE5S95oh\naQgYAoaAIWAIRIWACZ2okLRyDAFDwBAwBJIiYEInKUSWwBAwBAwBQyAqBEzoRIWklWMIGAKGgCGQ\nFAETOkkhsgSGgCFgCBgCUSFgQicqJK0cQ8AQMAQMgaQImNBJCpElMAQMAUPAEIgKARM6USFp5RgC\nhoAhYAgkRcCETlKILIEhYAgYAoZAVAiY0IkKSSvHEDAEDAFDICkCJnSSQmQJDAFDwBAwBKJCIGvX\nB0j2fZ+oALJyDAFDwBDIZQT++te/Rnr7WSt0QCFqMCJF1gozBAwBQyDLESgK5d7ca1neKaz5hoAh\nYAhkEwImdLLpaVlbDQFDwBDIcgRM6GT5A7TmGwKGgCGQTQiY0Mmmp2VtNQQMAUMgyxEwoZPlD9Ca\nbwgYAoZANiEgs9eKYoZCqiDYTLRUEbP0hoAhYAjER2DTpk3u888/d998842rUaOGO/DAA+MnTuMV\nETpHHHFEGqvcsap33313x5N2xhAwBAwBQ6BACGBIPPPMMw7Bo4Tg6du3r9tnn330VEa2Wf2eTljE\nfvzxR7d9+/YdkpcqVcpVrlzZ/fzzz3K9YsWKbsuWLe6XX36R8ztkiOAE7fj++++lM1StWtVt3brV\n7brrrlLyhg0bZL9MmTIR1PRHEUV9P5E11AoyBAyBSBBA4IwePVosm44dO7pddtnFrVy5Us7deOON\nbtCgQRkVPDkhdKZMmeJ+++23wAfatWtX9/7774ugadOmjVuxYoWbM2eO4zx5sMIaN27sqlSpEpg/\n1ZMffPCB41ehQgV36KGHurfeesuddtpprly5cu65555zZ5xxhtt9991TLTZueszr2bNny/3ETWQX\nDAFDoEQggGWDhYMrbfDgwbF7wrqBj/Xv399NmDDB9evXL3Yt3Ts5IXQAtWnTpu6QQw7Jhy+WDsT5\nbdu25bvGAQ/wk08+cQcccMAO1wp6Ao2jUaNGrkWLFu73338XXyvW1saNGwtapOUzBAwBQ0AQQMmE\nb2Hh+An3WqtWrVymY/g5I3QQMGXLBt/uZ5995jZv3uyOOuqo2HPCLfXqq6/K8YwZM1zz5s3FJP32\n22/d3Llz3fr1612lSpVckyZN3F/+8hdJN2/ePNl+/fXXUt5xxx3natasGSvzxRdfdD/88IN0CrbH\nH3+8e+ONN2SrAlAT4+LDyvrqq68c7rYGDRqIcFTX26JFi9xHH30k7kCssMMOO8ztueeemn2HLdbV\nxx9/LBYVQo92f/fdd+711193xxxzjKtevbrk4b6nTZsm9+svb/78+ZIGrBCeWGeUQ3kQnR0M1q1b\n53AV4j4kXkg5XJs+fbqkp90c77vvvqIMYFmuWbNG8MX6w80JJcJaEtg/Q8AQyIcAYw+KN2kAV1um\nKWemTMPEYWLeH/EUCCsDJuklBJQyU7bVqlUTRvn888+LhYKlArNHaMCAIRjpwoULRUjAxBFKXjro\noIPErbbXXnsJ88XSIb6j7dC0WF0w/rVr14qggTlTLsIOomOxv99++zkE20477eReeumluC5E8iBw\nsOgQguT99NNPpf3ggktRadWqVYKRCiE9z5b7Q3jRZoQw9497EAEK4cZDeBx88MEiyIhfIbC5H71X\n8KLd3BPCZ/z48YIJwgkcUQAg6kqEtSSyf4aAIZAPgX3+nCSgvCLfxbwDZrJlWvAEq/7+lpaAY5gs\nPy+deuqp+SwR77XSpUs7hAOEpk6w/7333pPj1q1bC6PH+kBgfPjhh6KlcxHtn9hQkFVVt25dV758\neYnZsO8XdFJ43r/Vq1cLI2/btm2sDTvvvLNYPlg0mq9evXpSFgICYRHkItQysWaoE1fhTz/9JEII\n5s8PXBAiWFvLly939evXl3ZqXu+W+2vXrp0I1jp16rinnnpK2go+u+22m/iNOQ+BwcyZM8Xq0zIO\nP/xwaQMCCaFDHtoGoRB8+eWXIrQQklAirCWB/TMEDIEYAggd3Gga11GvAQkYb7NmzRLPSixDBnZy\nRujAXP2xGWIpqRAaPoRGr4SWr0KAc7iUggSOpg+zZbYdBONdsmSJ7DPDDqKuPfbYQ6yDyZMnS30I\nCSwHLJ54VLt27dglhCkWC4yfvAhNrCfwgOmfdNJJsbT+HSw+dfEhCCEEL+eOPPJI98UXX4hwxv2I\nmxGiHiXwgRBwWEoq2DnH5Arce1AYrCWh/TMEDIF8CDAtesiQIW7AgAESw0EI4UXQWE6mX5HJGaED\ng4RhFoaYzYbm4C1H99XKgHEWltTd5nVxUY9aSpjH7du3F7cYnYnZd/w6dOggjDyofq8gVKFBOsrF\n2iAAiRDAkvEKAn9ZXA8i7n/q1KkivBCKuPFq1aolws2bHkvPS/HKS4Y1lqiRIWAI7IgA1s7QoUPd\nqFGjZKYaKeAZZ511luPanXfeKcKoR48eO2ZOw5mcETpRYIklQJyF4LkyceIhxB+iZIL63g7MG8YN\nYTngQoNJY0FovIeAIdYW062xUhAcQUTcSsvFfUc6nbzQsGFDsaqYqo3L0CuUgsoKOodrDGsJK2nv\nvfeWJGqleS2doLxB59KFdVDdds4QyHYEsG7iTYtmAhNubygTgsfURYE++J8yX5g5gkVnqTHbCvcP\nQoCHR5A8SsKiQbgQDETAIHDwxSJosKRoFzPJcL/RLoKDUCJ3IfEoYjnMeiPYr5MkyIf2QzncD+62\ngpAGJxE+uMhot8bA1HJLpdx0YZ1KmyytIVASELjsssskrgPvuv/++9N+Szlj6ahWnwxhbzpcabiJ\nmAaMu6dZs2auZcuWEtBftmyZCAZiRVg+URIuKCYRMJ2ZWWwQgkh9sbSJ945o19tvvy3XOdYAvpzw\n/EOAwfgnTpwoZ7GO9t9//1gKLCDKRCipZRW7GGIHzMCK2XGLFy8Wlxp1ghftQxBRPuTFN6hovU47\n0oF1UBvsnCFQ0hFA8EC8loHbDcsoXVQqr6Ltqimnq1J/Pdx4qh/8JCiWah5/vWGP0dxhosoQcRcx\n1RirI0q3WlB7eCcGV56687xpiKNgocDww7QjUVm457By/C/QeusLs0+bqEctnzB5EqVJJ9aJ2mHX\nDIGSiIB+DDTevRWEz+qEhXhl5oylEw+AMOf9wW+ET1RMNVn9iSYmIGg0TpOsHK4HlYXFhruNuBBW\nW2GJNkWJTTqxLuy9W35DINsQSKeFo9iY0FEkcnSrkxJOOOGE2JcAchQKu21DwBBIAwImdNIAcnGu\nQl/MLM5ttLYZAoZAyUGgQEKHFxTRkPkuGO+JQMyA4s19pvmm4vKRzPbPEDAEDAFDICcQSFnoECTi\nw5V+8n7rh8/LpCvI72+HHRsChoAhYAgUXwRSek9n+PDhgQLHf3sIJdIaGQKGgCFgCBgCXgRCC53L\nL798hw9megvy7/MRSfIYGQKGgCFgCBgCikAooTNu3DhNn/K2MHlTrswyGAKGgCFgCBRrBJLGdFj/\nxPtV5VTvhry8KX/sscemmjVp+mQvISUtwBIYAoaAIWAIpBWBhEKHz6KwXoqfWNOFxcOCiM/k64qb\nep0yWNiLrxlHRTZRISokrRxDwBAwBNKHQEKhw4cugwiBk+jtdb/QoQzKireEalAdds4QMAQMAUOg\n5CGQMKYTT+gUBIYoyypI/ZbHEDAEDAFDIPMImNDJ/DOwFhgChoAhkDMIJBQ6Yb5cHBapKMsKW6el\nMwQMAUPAECheCCQUOnzaJiqKsqyo2mTlGAKGgCFgCKQXgYRCh2+pRUVRlhVVm6wcQ8AQMAQMgfQi\nkHD2Gh/vDCKmRccjvkQQRPHKCkpr5wwBQ8AQMARKJgIJhQ5fi+bjnf4PfDIlOmhadDyIKMO+PB0P\nHTtvCBgChkDuIJDQvQYMvISZ6J2cZFCR117kTIaSXTcEDAFDIDcQSCp0gOHKK68sMBqFyVvgSi2j\nIWAIGAKGQLFEIJTQoeX33nuvS2WVSdKSJ1300UcfOf+vsHXfcsstbsmSJYUtJrL8zzzzjJsyZUpk\n5UVZ0PTp092IESOiLDJW1s8//+wuu+wy99tvv8XO6c4VV1zhPvvsMz3Mt33//fddly5dHPnDEt/z\nu//++8Mml3SffPKJ69WrV0p5ijJxce4nRXnfVnZ2IJAwpuO/hXPPPVc+3hn0PTZv2nPOOadIPvDp\nrcO7P2HCBMdA81OrVq1cjx49/KdDHf/444/u0UcfLVbMZMyYMW7AgAGh2p/uRF988YWbN29ekVT7\nzjvvuLVr17py5crtUD5ComvXrjuc58T27dvdli1bAq/FO7ls2TK3aNGieJcDz3///feiDBSV0A2s\nNMHJ4txPEjTbLuUIAikJHTDha9F8vJPP2uiP83xJWn9RftiTspMRFk4QzZw50/ELor59+7rDDz88\n6JKce+utt1yLFi1cpUqV4qZJ54V169YJM2zWrFk6qy0Wdb3++uvupJNOSrktYPXkk0+mnC+bM+Ry\nP8nm55ZLbQ/tXvOCglDh451t27Z13bp1kx/7nEu3wPG2K5X9eMJIy+B6QRid5o96O2fOHGlPhQoV\ndih6w4YNYgE1bdpU0tx1112i5ZNw1apVrk+fPo5rRx99tLjAtm3bJmXgmnrwwQfdmWee6fbdd1/X\nv39/t3DhQte+fXs55tmixUOPPfaYw914ww03SFnt2rVzTz/9tFwL+oflefrpp0ta6sFSUZoxY4ZM\nLqHO7t27u0RT8Mnz0ksvxf2qOddxgXbs2FHafMkll8g9c37BggWxZ7h582aH5Yv1ylbdYSzd4c37\n7bffkjUhYdVRD+3nHj/44IN86b/55hspH8yZRONdUwocb7zxRtezZ0/JT920H4uc8kjvtbTAkedB\nWRdeeKFbvHhxvrr8B4n6iT+tHRsCmUCgQEInEw2Nus5Efn6YMtPEi2INoILeB0LwhBNOCMz+8MMP\nu48//ti98MIL7u6773bPP/+8e+WVVyQtQuL3338X9yP7uIBmzZol177++mv3yCOPiFCiDBjc+eef\nLxNHxo4dK8KALaTuRrDBvUq6q6++2sG0/UR8BwEGI8X1Sf0IF9xdLJdx0UUXuQsuuEDWaUJRGThw\noFNB6C+LeA1C9YADDvBfih3ffvvtUv7EiRPdL7/8IuVT56+//upWrFgh6agbYYGgvOmmm+SeP//8\nc2nHIYccIu4xluzANZWItm7dKnVRD/WxOu4dd9wRy0K9CAe23DsC97rrrou9YgCOo0aNci1bthS8\nwQNBg0IwadIkV6VKFXfrrbdKeQhbcOzUqZNgzgvWuK4RavEoUT+Jl8fOGwLpRKBsOivLlrpg4LxX\nlMpU8fXr1zviAUceeWTsNtXt513SgfhEgwYNXPXq1WPpku3AwHgv6qqrrgpMynftNm3a5L777jt3\n0EEHORiPEpo9hKZfrVo1d+ihh7qlS5fGBFjnzp2F4ZEGy26vvfYSS4Djk08+2a1cuZJdodq1a7vB\ngwe7MmXKuMaNG7u5c+e6l19+eQfhjPD75z//GSu3d+/eUtbq1atjljBuoJ133tlxjV88Qqidcsop\nUme8NLhKsbShoUOHSr3Lly8PTE77YfAQArV+/fru2muvdaVKlZJ7evfddwPz6UmEIC9AI5xq1aol\npxEqWIEQ9WKpMIkGq5+XorEkEcRqOePWRXhAZ599tnviiScczwEiboolBCmOZ511lhyjNEybNk1w\nD3oNIVk/kULsnyGQYQRy1tJJhDuMTplYonTeazAK3FqjR4+W0wicIUOGyE+FD9dIo0zFmz/RPm6i\nmjVrur333jswGQyrefPmwtzQ2mGAaNQQgXYYFIIPS2n+/Pkx1xvXEURKuO687lEELxaCEoIGgaOE\nUEbQ+gmmC1NmBiM/mG7FihUlBkj5MGS0emIuuI6CrCUt87XXXosJQT3n3/7lL3+JnUIwQlgxQeS9\nXwQEFhQCR6lRo0a6G7ilXO5FBQ6JvPWrsANzvX8ExZo1a2Ll1ahRI7YP5lg3SsQQURAg+o23bLBv\n2LBh3Nl6yfqJ1mFbQyCTCJjQCUAfqyLeyqgByYv8FEy5TZs2ceuBkeLiITbwwAMPiGsQLR7rC/cO\n2jPWG7ETLJ2CkjJUzc9EkiBBCGNkijP1eX9HHXWUZMVyAeM333xTJnPgasNS8xPnWO6cCR2JiLiV\nEtYexKSWZEQa/z3hgktEWIK0S4U6ab1rRWm9TETx3jsutVQJHL1lowDgLgzCnLKT9ZNU67f0hkBR\nIGBCx4cqgXOm/nrdZL4kgYeDBg0S9xcMFMKywJXDT91rXMNFRtpUKFkgHb8/ZRIXwRrBKmKqsC4n\nwZZ3XLAusHQKSjC8kSNHijCjTcSAWrVqtUNxJ554olh8uKqIx+A+wrUEo8YKYp8JC1gLTZo0kfxB\nMR1ckVhDVatW3aEO74lhw4bJpAFiVFiXWDu4MJMRbjYENS5IBPTkyZN3+OSTvwwsD4Q8zxXrhUkE\nxJSUEBTUT1wGAYh7EhcaMbNUCZweeughESa075577pE6jzjiiMCikvWTwEx20hBIMwIW0/EBjoaK\nWwQXSipEjMYfp1Fh4y0nVWHGrC8YY6Kp0rynwpcftD5mlnXp0kWYNQKJQD0/LA3uTd1JKpS87dNr\nes57jMUBPrgIwYc6ifv4iRlZX331lQTUsQpg0sRaKleuLLEyXJe41SCuIciCvs0Xdqo0M8mYnMDM\nM9xjzBDjnR5v27WN3nOkRWAhIHBJckzQ3jvTTvPpFncYVku/fv0kNgQOTJIYPny4JKFerEyUC1ye\nEDPcNIZD/d42SALPP67pDEViOQhSyuLecGciwIO+2B6mn3iqsV1DIGMI4Mzenmg2TDpahkYcFBgN\nW3e8l0MT5WdAwxz9xMuXMJ94Lxz60xf18XPPPSfBY6Y2JyMYPEyvfPny+ZISYOaaN16TL0GIA2bF\nEb9BQGC97LLLLvniO0FFYL1s3LgxsF7axDUEURDhSmKGF5q+WkNB6bznKK8g71VRF7PIvG3B9ceX\nBvyElYE1CZGH+oKEN9eZIcmzKFu2cLod7eP5Jbq3VPoJbTMyBIoKAeLIiahwoyFRyWm8hvBQRhC2\nWrUK/Olx6aC5FhdihliieI63nfGsMwLQhRE43jrYD7JK/Gk4hhnHq5c2eZm8Pz+xFRh2vOfkT89x\nIqYclF7PYV3428LUaH5+QgAoxbs3vY5gjoJoX7J7S6WfRNEmK8MQKCgCJcLSKejNW77wCCAEeDdl\n//33D5/JUhoChkDOIZATlk7OPdUM3HC8GVMZaIpVaQgYAlmMgM1ey+KHZ003BAwBQyDbEDChk21P\nzNprCBgChkAWI2BCJ4sfnjXdEDAEDIFsQ8CETrY9MWuvIWAIGAJZjIAJnSx+eNZ0Q8AQMASyDQET\nOtn2xKy9hoAhYAhkMQImdLL44VnTDQFDwBDINgRM6GTbE7P2GgKGgCGQxQiY0Mnih2dNNwQMAUMg\n2xAoNt9eY0VGI0PAEDAEDIGSjUCxETreFRJLNuR2d4ZA9iKAcmhjNXufXzpaHvR1dm+95l7zomH7\nhoAhYAgYAkWKgAmdIoXXCjcEDAFDwBDwImBCx4uG7RsChoAhYAgUKQImdIoUXivcEDAEDAFDwIuA\nCR0vGrZvCBgChoAhUKQImNApUnitcEPAEDAEDAEvAlktdObMmeM+/PBD7/3I/ltvveXmz5+/w3k7\nYQgYAiUfgZ9//tnNmjXLrV27Nq03+/XXX0u9q1evDqz3nXfecW+88UbgtTAnx40b53r27ClJv/nm\nG3f44Ye7p59+OpaVZaJ79Ojhbr75Zhd0PZYwxI63rhDJU0qS1ULnkksucWeccYZbvHhxvpv+17/+\nJcDnO2kHhoAhkBMIwPy7du3qUD6jJITG2Wef7dasWRNY7Ouvvy71wvT99MUXX7hzzz3XXXDBBf5L\noY+XL1/uZs6cKenLly/v9thjD7frrrvK8fr1690VV1zhtmzZ4o455hjnvx66kj8TeutKNW+y9MXm\n5dBkDU10HSHzwgsvuJ133jlRMrtmCOQEAt9++60rVaqU23333XPiftN1kz/88IObO3eu+/XXXxNW\n+dJLL4lgql27diyd1yKJnSzETuXKld3zzz8fK+GDDz6Q/f79+7v9999f9r3XYwmLwU5WWzqK34oV\nK9ydd96phztsJ0yY4M4//3y37777unbt2rlnnnkmlqZbt26iIQwdOtQ1bdrUtWrVys2bN8/deuut\ncnz00Ue7SZMmxdK///77jjyUdfrppzvMUCNDoLggsG7dOnfbbbe522+/3X333XfFpVnFqh1vv/22\nuKF0vD/00ENu+/bt0kY0/C5dusj4/utf/+ruv/9+OT9lyhR3/fXXyz7WSiJ+Q6KJEydKWv5hfYwe\nPTp2rDu4wK666ipxk8F3brzxxnwCbcaMGe7MM88UPtSrVy+3YcMGzep++eUX4VWTJ08WfnbNNdfI\nNbw/t9xyS77rmgk+2L59e7m3Cy+80C1dulQvuUR1xRJFtJP1Queggw6ShzZq1KhAfyk+1AEDBrhK\nlSqJy22nnXZyaAP6rbfPP//c4Qv99NNPxXTGDO7YsaNbsGCBmMI8aDoGfmI65FlnnSVaDIN67733\ndtddd51YWRE9DyvGECgwAriVEDirVq1y9GsTPDtCiRV43nnnua1bt7oRI0a4Y489VnDCdQZdeeWV\nIqyxTEiHcMGlBZ+BYUO47tq2bSv7Qf9QcOFHv//+u1x+5ZVX3KZNm0Tx1fS//fab8BflTwgy8sCr\nID4lc9FFF0kZQ4YMES+O11ratm2bg1fBnw477DDXoUMHyUc5CEvvdS7A4ygb19ujjz7qNm7c6Dp1\n6uR+/PHHpHVJwRH+y3r3Wrly5dzw4cOlEyAcXn755XzwHHXUUSJASAfQdIRFixa5JUuWxL4hVbFi\nRffAAw+4MmXKiDDCN8tx1apVRdjQGXjAGgTs06eP22+//Vzz5s3lYWJOn3baafnqtQNDIJ0IfPnl\nlw5r3Rs8V8Fz9dVXm6vtz4dRrVo1UTA5hB/sueee7oknnhCtH16B6ww+gHuS+A1xmNKlS8sx4x2G\njVVSr169uI8XoUOZTGY48cQT3dixY91JJ53k6tevH8uDlQEPGjlypAgJLiAA/vOf/4jiQF4IPoSb\nDqFC+mXLlsl57z+8LrTtkUcecccff7xYMgg5L2ERNWnSxHXu3FlOE5JgnwlXKNxQmLokYSH/Zb3Q\n4f4JqN19993u4osvdtdee63bvHlzDBb8sGh/XhcZF9WcZr9mzZrS0dinU0L4TKHq1avLln/6Ibvu\n3bvHzrHz0Ucf5Tu2A0MgnQigEPXr1881bNgw1m+99WPxMOupbt263tM5uY8FAK8YP368w+pRUqsE\nF9egQYPE28G1c845R7BFAQ1LCKQTTjhBBA/7uPNwrylzpxwVHiivSjw/CGWB6yjDGhdCCOIO1Hya\nJ+yWWb5MgGjRokW+LIQmoq4rXwUBByVC6HBfrVu3lljLY489lu820RwQOGgUmJavvfaa6927d740\nYQ9wp0GYy16tJWx+S2cIFAUC9EuYqFFyBKZPn+7uu+8+UUT/9re/iYKKlaB05JFHOjwXuChxqw0e\nPFiUWp2qTDoVUJonaIsVQey3bNmy4oZv2bJlPqGjvAQLVQUPygNUp04dyYO1gtJcpUoVOY97v6CE\nQOPr4GPGjNmhCPCIsq4dKvCdCB3TAZziTmh7jRo1Cmwmc+fRONByCkrHHXecZCWgOHv2bHmADRo0\nkABgQcu0fIaAIVA0CDDph1iG/nAlqcAgFoKWT9BdibgtE4eIfeBi01lguNcgPCIQE5FUQMiJgH/E\nihAsCDmC9lqGJoVPYcUwvRq3/dSpU929994rijHxZ9oB4crnfUTiT4WZAo5SDs/C4sK1Bx/ELcfr\nJlHXpfcYbxvK0mEmF0wb6Y0U9s7+ilewnsdPyq+oyPswmSSARQPASgTj3n33XdFsMFcJBuJvDUuY\ntUoHH3yw+Fz//e9/x+bbUx7BRyNDwBAoHgjomCWWwk8JywZXI4F2eBp06aWXypY8u+yyi7jnmQnG\nTC+IPBoHYfzjNiP2wXsxxNC8pPWyRWhddtllUj/vEvqJ1ztoG4qyvrtD3AfeAh1yyCHSxmHDhknA\nn4kM//jHP9yLL74o1711yYk//yk/9F8nzoR7jfKxauCFCNzGjRtLTvCIV5e3/Cj24ajbmbqXiIpa\n6CAU6AhFSWg2dCo6QxT0/fffy4tZmM9GhkCuIFBSFnFjyjGMuUKFCoGPjunmu+22m7jH/Amwlsir\nDN5/PdVjLCz4CC90+onYMxMe9CVQ//VUj4lp4bILeocrqrqwLBNRKKFDAbjXsHKKgtIhdIqi3Vam\nIZBrCJQUoZNrzy2d95tM6ISO6RSVwEknGFaXIWAIGAKGQGYRCC10smEiQWahtNoNAUPAEDAEkiEQ\nKiBR1DGdZI2064aAIWAIGAIlA4HQlk7JuF27C0PAEDAEDIFMIhDK0uEtf+9EApsinMlHZnUbAoaA\nIZC9CISevVaUt8jsNSNDwBAwBAyBko9AKEsHGLyWTlHAwhu8fHnV+020oqjHyjQEDAFDwBCIHgHe\nXeLDyvph5Hg1hBI66ZhIwIubJnDiPSY7bwgYAoZA8UdAv4SQqKWhhE6iAqK6FtWXAqJqj5VjCBgC\nhoAhED0CoWM6Relesy8SRP9grURDwBAwBDKBgH2RIBOoW52GgCFgCBgCgQjYezqBsNhJQ8AQMAQM\ngaJAoNjEdAp7c0Grdx544IGFLdbyGwKGgCFgCESIQIkQOqx9EbTGD2uZ9+jRo8BwXXHFFbJWDivu\n+YkFoliVlFX3mHkXhvB1shRtKm3iM+q0gzU+dLXBMHVZGkPAEEg/AizExsKOJ598cvorz5IaS4TQ\nCbJywJ/lZvkFUd++fd3hhx8edCl2DiHRtWvX2LF3h+ndW7Zs8Z5Kus9a5IsWLUqazpuAelg+9/LL\nL/eetn1DwBAohgigjEa1zk4xvL1ImpSzMZ14wigsqs2aNXNPPvlkaCsnbLmWzhAwBAyBkoxAzgod\nVusLQ0uWLHEdO3aU9cQvueQSt2rVKsm2YMECx/Ky0ObNmx2uvEcffVS2vXr1kvO8mevN++2338r5\nRP9+/fVXN3DgQNe0aVNZu3zSpEn5krPKK+VzndVWx40bF7v+2GOPuRtvvNH17NlT2kvdtB93Huuh\nk95raaGVdenSJZY22VRH2saLwtwr9dOOn376SeqfN2+eO/vss9348ePFgsSKZDlcXZOetnF8ww03\nSN527dq5p59+OtZ22zEEMonAW2+95S688EIZC/TNqVOnSnN0bD/11FOO8/Rrlqlm5dF4xKqj8ArG\nCGNv8eLFsaSJxhzu+rvuuiuWltWJGWtr166Ny2NiifN2Eo1P0j388MOxsctYHDBgQCwswTi94447\n5Dr32L9/f/fjjz96i49sP2eFTlgEWVO9e/fubuLEidLRLrroImGkPOAVK1ZIMbjAvvjiC2GiN910\nk+vTp4/Eblj7nLXOp0yZ4g477DA3ZsyYpNXefPPNbtq0abIO+/Dhwx3MWomOwcBgSxyLWM91113n\nXn31VUlCJxk1apRr2bKldCYEAoLm6KOPdgivKlWqxNaGp720r2HDhrLu+umnny7lMfji0eOPP+5m\nz57taNfo0aNFgLFePMTgnDt3rnv22WfdQw89JAOTgcpAgmgbQpnlcjnPmu1XX3110k9mSGb7ZwgU\nIQL0TcZs8+bNxZV92mmniTub8zq28WrAlBmf//3vf91rr70Wt0X0c8ogHXxCBUmyMYewQtAoMc7J\ns3Xr1lg7UNSUx2g63SYan4z/2267TRRFFFWWrIaHbNiwQbJzbfLkyaIYPvjgg8LbLr74Yi060m2J\niOlEioivMGI/bdu2lbNoODDw5cuX+1L9cTh48GC5ztHYsWNd/fr1HV/o5tMQjRs3dsk+bEoHp3PQ\nuREcEEKvffv2sk+9WCoEK1m/fY899nBnnnmmmz59eszqQks555xzJD2WxxNPPOE6d+4sx+eee65Y\nQhwgPGrWrCkTFGjfAQcc4LBW0PBatGgh6f3/Lr30UsePQUBnxdLzx9PuvPPO2IQHhAraVe/evaWo\n2rVrOzDi6xPggZB6+eWXHd/dMzIEMoVA5cqV3Zw5c4SxY8F06NDB3X333Y4YLOMC6tevnyiQ7NNn\ncc+j0AURCtXf/vY3uYTCyjHlFmTM+cv38hj/tUTjk3HdrVs3uTfyIWRQbiH4Dh4K+JvyHa7D97Cy\natWqJemi+mdCJwmS3plrME2IGWgwfT9Vq1YtdgoBQYf1fouoUaNGjjXm4xHut02bNomJr2mYCaOk\nws7f2YkvKdWoUUN3XYUKFcS60ROVKlUSi4Rj2uFv33777edef/11Tb7DFjfBsGHDZMBVrFhRrh96\n6KH50nln2NF2NDU+5AohaLyfO6K+t99+O19+OzAE0o0ATBeLHEWRMajjmPNKKGhKdevWTTiO99xz\nT03qdJ9JRwUZc7GC/tzRtvnPc5xofFK3V7krW7aszLIjn/IdlGQl3YfnRC10zL2mKMfZagyHy5i/\nUJ06dWSb6B9pVEhoOhhwIqpatapc/uqrr2LJVq9eHdvXenGBffjhh7EfLrVUiU7lbx/3qp0tqDym\nbVevXt3hl6Z+NCc/oRkp8ekkBDVfnoX89XHdK6Q0n20NgXQigNUyYsQIcQvDnGfNmlUk1ScbcwiC\ndevWxer2utpiJxPsJBqfKJhLly6N5cbyWrhwoRzvvvvuDiXSy3d0HwEbNZnQSYIomj2TBr7++ms3\nZMgQYaJe6yNedtxwBPHx765fv178pS+++GK85HIeK+DUU08Vv+qnn34qVgJ1KhF/gYkTzEcArly5\nUoL3uLBSpaOOOkraR8wITQeXAf5inRwRVB7+aQjrbf78+TJ7z5+OiQIISrQufNkEX5WIgY0cOVLw\nYBo471YRKDUyBDKJAO5iCKZPHFRjMFG3KdmYY7IPrnLczmvWrInFQ8O2I9H4xEWPqx1egZAlHqzE\nFG/eK8KtD99BGWRCEp4ZVXQ1bRTbEiF0CvLlgbB5mIXC5AF8nTwQmDSau9dtpg/Ce44HhsAi0H7E\nEUc4gnOdOnXSpHG3BCrr1asnzBqGrCYxZVMvLgDiKAQ9TzzxRInraAyHNN42+CvhGi43CNcWHZCO\nSByICQnXX3+9a9OmjT9b7HjQoEES88GlxntDfgGFtrTXXntJm/FpM3mCGTJKxIqw0sCDWBkr0NpL\ndIqObTOFAGMMZe+MM86QPks76MveseTd57r/mHNK8a4lG3MIhmOOOUYUSZRW9Tp4y/Pua326TTQ+\nW7duLcIUgUO8hnoQgkoqZFASjzvuOHHzeycxabootqXyCtnONNxMUhRfmfYHtJPdT1iho+Vs3LjR\nERNJlfALoz0RrFR688033SeffKKHsS3MmLgHhPmL5VO+fPnYde8OU765hnZWWGJSwK677irFcJ/M\navETdREQ5X6Y1cP9eAcA90QgE7cbbeeaCjjK0sAslg718RUHb3zHX58dGwLpRoB+C+28885FXrV3\nzPkrY2wzruONfX9673G88YnnAZeZKpXwJGbWMinplFNOiRWB1UcMtjAYJHv1ovAcK9bczO6kKkRS\nbW1BBA51wHy9AodzPFg16TlW8gYukz30sJ/e0bITbVXgaJqgtulb1twPU68TUbK2++tLVJZdMwTS\nhUCyfhtlOxKNgcKM7Xjjk4kMeG2YhEQcFfc2WywqLyHsolBkvWX690uMpeO/MTtOLwLM+8f9iKsu\niJhEgSa5//77B122c4aAIVDECDA+eaePeDCzcnGl+RXiKJqQM5ZOFGBZGQVHAOsnnsChVJulVnBs\nLachEAUCxJT4ZZpKxESCTINo9RsChoAhYAiEQ8CETjicLJUhYAgYAoZABAiY0IkARCvCEDAEDAFD\nIBwCJnTC4WSpDAFDwBAwBCJAwIROBCBaEYaAIWAIGALhEDChEw4nS2UIGAKGgCEQAQImdCIA0Yow\nBAwBQ8AQCIeACZ1wOFkqQ8AQMAQMgQgQMKETAYhWhCFgCBgChkA4BErEt9d+eOBKt2X5gnB3/Geq\n8vse7KpcOjylPJbYEDAEDAFDoHAIlAhLZ8uyD1hz1e100PGhfqSVPEmwY80JFnUKIhYy69Kli+OL\nsGGJbxLdf//9YZNLOtZJ79Gjh6ytk1LGCBOzmNQ///lPWSMnwmJ3KCoR3jskthM5jwBfQ37jjTdk\nrSsvGHznj7WaWLbDv3CgN10q++kaA6m0KVvTlghLB/B3OriVq9zphlDP4Ud3g/t1wcykaRESXbt2\nDUzHF6H5cmsqxJrrixYtSiWLLCXAF2FZvyaTxAJR3q9gF0VbEuFdFPVZmdmLAIIFJYXxxKKGuqbU\nK6+84i6++GLXpEkTWX9q4MCBsnCgf4n3gtx5VGMAhfXxxx9399xzT0GakfV5skLo/Pb7b65cmT+W\nPE6G+KqT+XB2fKo77X/rnsdPlfxKs2bNAlfOTJ4z+1KwjPb48eOzr+HW4hKJAMobC56xyCEr+irh\nFejdu7es69S/f385zSKKLCR4wgknFGqNmCjHACv1vv3229rsnNsWa/faZ98scx3uO8sdeVtLd9Tt\nx7ix7zyZ9gfEktMdO3Z0LCXLCFt0AAAgSUlEQVTLehSrVq2SNrCEta6cuXnzZhkALE3NQOjVq5ek\nwfT35qWzJSO0KbSzpk2byloXkyZNypeFBfcon+tob+PGjYtdZ6U/VgDs2bOntJe6aT/uOdpPeq+l\npS5CrpE23ifJGczcFwvP6b1OmTJFyqMdaJys7c7SBWDCgm5KLOpGXhZ+o32sjNqvXz9pP+5JVhL1\nUjy8vWlsP7cRoC9dc801YinstttuMTBwhW/atMmdd955sXPscy6Iyc+bN0/6PQuZ0Y/pp6QbOnRo\nbPzhpoO8Y4Bj+vwjjzwiq3ySFyH3+eefc8nNzFud09sGztHXWYp68uTJ7qqrrpIl4qnvhRde4LLz\njkXysmS10owZM2SsMU67d+8uCyXqNe8W3oHVR7m0CT7BYm1KrBSs12655RYRxt77Y7lqrvO1eO4H\nnIuCShdFoVGU+fu23133MZe6z7/7QorbsnWLu2v6cPfmZ3OiKD50Gbfffrs86IkTJwpTZelqOiAP\neMWKFVIObifM/aefftrddNNNrk+fPtIBL7jgAlmdDwbN0s1jxoxJWi9Medq0abLM9fDhw4VRaybq\nvfDCC6V+Vvek47PMNGtkQHSSUaNGydLadCY6HIKGhZoQXiw/QKeEaC/ta9iwoXvxxRfd6aefLuX5\nhYAk/jM97kS9V4QbHXP06NGibSLYWASLzs7AUkKQMVhYt0MFDwvuISyrVasmS2RrWrbx8Pamsf3c\nRoCx5GfqILL77rsLMChASrr/5Zdf6qnYFgUKwYMiR3+k71Iu68089dRTst4MfZx0EGNGXepYWCNG\njBBFFGUTZQmXGYTyxSqdXkJZJf6LxYXlRd9nrB5//PGOtjEWYfa40llGulu3bjKuGMPwHK7Pnj3b\nMXZQSrdt2+YtXvapnzTwDcYlCiaWHsT4Z5lqBBH3yvpX8BBWMIW4xrhFGD344IPC23BTFgUVW6Ez\n65PX3Y+/7ChpH33zjwdbFGAEldm3b1/Xtm1bd+ihh4oGxEJI8YKTgwcPdqy3ztoxWDmscX7ttdfK\nEtRYSXS4RARDp3MMGTJEBAfLV8OElaiXjkSn3WOPPSTNmWeeKRqUpqHj4t9mYJ599tnShs6dO7uD\nDjrInXvuubLQGmnpnDVr1hSN8YADDnCkQfBMnTpVi0q45b7QiqiHDjt//ny3Zs0aR3sQeAxUBgb7\nZ511Vqws2kGcDJ87FhmCm8GslAremse2hgAI1KhRQ8YYyhiKHgoP4w5iGfZ4xHiiPyqTxRJh2XjN\n6xcgWg6C4MQTT3SMUybaoLwlIyyzWrVqSbJ99tlHlomHV+y5556uU6dOMiYpl5VF33nnnVhxCE+U\nOtyH8AhdyTeWIG+H5eK5Z+4F3oPX4aOPPpIkjGsEWYcOHeQ6Y7ZixYpyDb6D+xxh1rJly9iYxtpa\nu3att4pI9ottTGfFtysDb/CLPy2fwItFcJIV9pRq164tu5jRXrNer6O9KCEgYOYsH6vUqFGjuLPh\nSIP7DVcAloFSgwYNdDcm7PxBUeJLSgw8pQoVKuRbWpolt1VrwxXhbx8LPL3++uuaPeHWuxgUHRwC\nF6wqcECooXlSn1fY1qlTJ1auDj5dm54L8fD21hcrwHYMAR8CI0eOFLcbrqSddtpJmPTdd9/t6tWr\n50v5xyGMV5epZrxAujR9smWjUfyU2NexpefCblFk+R1zzDH5sqxevVr4DO6/f//73477QGnDk4Jy\n66fFixe7YcOGiXtPBQrKMsR49+ZhSWrlLcp3dByTXvfhYzpOOR8FFVuhc2T9I9y9r+04vfio+keG\nuu/SFSu73fuMcmVr7ePW9W/ltm3a0WoKU5DGcEiL2Q3BOL2+Ujnp+0cavx/Zq9H7ksshwUoIzQoB\nBdHxlJRh4wLTgaHXUt3Sqfzt4161syUrj3tBo4JUE9xrr71cmTJlxKJC40LooFnpoE5WJtfj4R0m\nr6XJbQS2bt3qPv7443wTCXBdYU2H7ddRIMgYgJHjDmcfS0LdfEHlY/EgTLBgguiUU05x/HDpERfF\nEiJWqoJF8xDngm8QH8KVjpDCAwGhYC5dulSTigtw4cKF7owzzpBxSllevqNjum7durE8Ue2Ujqqg\nqMtpsmdj16fNlfmKPWCPRu6aUwbkOxd0UC7vxc9a933gdj66vSvX4BC3x+iVjnMFITQHJg3wwHF7\nYe2ohpCoPDR+/Lz4e9evXy/+0mTmNx301FNPFb8qmg+MnTqViL9QP3EZBODKlSvFhYZWlyrhN6Z9\ndGIGyMsvvywxKZ0cgS/4ueeei1sssScGOIOa+A6WCEIHoiMzgDDZETqpUEHxTqUOS1syEWD8wHhx\nQcHkGXfXX3+9uIH333//tN00Ag6PBXEh2sGEA46VqlevLmMOwYGgbNGihUMAPPnkkxJjQRnEdY17\nC3c6Y5LrWByq6AXFdIgzQ3hXEDaUp8RsP95bglcw0QEXpBKuupNPPtkxkQC+o2MaAaaKrqaNYlso\noQOzKko6/8jz3PTe09yIc+52T3V/wj1x4RhXcac//JDx6q3Y5gJXc+hMsXA0TelKVVztPCHEtVQJ\nvy6BPHydPBCYdLly5fK5zbRMvysNBgrzxudLcA6fbTKCmeMKaNeunXQ8NYkpm3rHjh0rftrmzZuL\nPxmzXt9RII23Df66uKYuBIQEHZCOSByICQkM0DZt2kg2LJVZs2blK8JbNvEfBORxxx0nA+ihhx6K\n1Y1QRnPDzXbIIYfEyojXPm+58fCOFWI7hoAPAe0/bP/zn/+IMoZSxbiDOTMbLSgG4ismdugtL3by\nzx29lqg8xhbxIWIktAPLA2GheRgbuNJQzpigRHpeGmcMHXzwwTKZgdmkjEsmDhBTRmjgeoY/4EIk\n5uOnQYMGSUwWlxrv9akCSbrWrVu7u+66SwQO8Rzqp21KKI4IGfgOYxohCa8rCiqVV+h2Zm+kSjBR\npC+ukyuvvLJQEvHdd9+VWVaptkHTr+vXKrWXQ8fe4H5fu1Lcb1pGsi2ByIK4tDCtccUxe0uJKcVM\nP/YTg4QAJkScA82tfPny/mRyzEwYruGbLSwxg8XfiRmsDDAdZFoH7aKNBCYZBBz7cUF7o/MyE4jZ\ndgWhguJdkLosT8lDgEkt8CbvuEv3XTLTja8m+N1g2g6uM35VGHGeGZ7Ei/3jDlcdYyLZ/cBvKIN0\n3jKI9+AyU6USnoRCSLwI150SY5c2p+IS17y6RWFNRAXiWGj8CBwIpsMMJQRPJunXhbPcT2P/54pK\n1BbS8smcVMjPWMPm5cH7OwoPlp+f6DBKyR56siCnlhNm6xc45PEOhHhlIBT9uBBvIuhJp2b2XEHJ\nX25By7F8uYmATvrJ5N2jFMZTGmlX0DU/r9D2M9biXdM0bOE3xHP8hIDDi8AkJGbXMjWbLWEALyEE\no1BkvWX69wskdPyFZPq4Ytsubmue5bI97y8MIXAqHNQqTNIiScPcfH7ZSMwIwiUXL8BIh2dqNm5B\nExzZ+IStzSURAVxuCBre6SMejADCGxFGkEWNRyj3GpYMwSUlrBvvMVq5P+DEuxn+c5rfvy2se81f\nnh0bAoaAIWAIZAaBQrvXcKW99tprCVuPECKdl4qDy83bHts3BAwBQ8AQyDwChZq9lvnmWwsMAUPA\nEDAEsgkBEzrZ9LSsrYaAIWAIZDkCJnSy/AFa8w0BQ8AQyCYEQk0k4IZ4EZQ4jU6VTnaT3skFySYV\n2ESCZGjadUPAEDAEsgOBZBMJQls6vF3Ol1TDkk4uYIIBn9o2MgQMAUPAEDAEQgsdoMJ60TUrUoEO\nAVTUn8xJpT2W1hAwBAwBQyAzCKQkdLBa9EvLqTSXb/x4P/ufSl5LawgYAoaAIVByEEj6RQIsFBYT\nYuv/FH4iGPjiMB+tg/ignZEhYAgYAoaAIZBU6GDZxPskP662eJYP3wZLp7BhhTxdJc//WPfZZx/H\nV5mNDAFDwBAwBDKLQFKhE695fOAToUK8ZsyYMaFntcUrrzDnWVfGu+5MUFmsT9GjR4+gS3bOEDAE\nDAFDIE0IFEjoEKNRK4bJBUyJDjuVuijui8/8Q6yoh1XjJQQSU/hm5i1cBKUieFjoCOHqXUJZCsn7\nxxoZrN1x3333ubBffKYdLOmcShuor6D5tK22NQRKIgJ8gh+XP+PTu3Q0q+3ytXP4AmvXeNd0KigO\n33//vevVq5cbMGBAbPmRgpaV6/mSTiTAhYZg8ZJ/BlvQJIGwH/v0llvYfQQOix55f5yrUaOGMHoE\nD4slhSWYPWtTBBHLEPC58FRo2bJl7oMPPkgli6QtaL6UK0qQASHbs2fPBCnskiGQPgRYVffMM88U\nRdO72CDv/PGF83HjxjnWrfr73/8uymEULWNlTu/yIwUtM9fHUlJLB4HCErDe2I1f6AC+fz0dtYQK\n+mCizod7DVKhk6q1IZk9/5o1a5ZvOVjPpRK5m+pEkhIJgt1UsUAAJYyVNBnTLCPvpVGjRsmKtngh\nWFuGZdOvuuoq+ZQ/y3IUlKpWrSrLrxc0vzdfro+lpJYOYCF4ECL6C7Js9JpuvSBneh+XFjEfNCJW\n8cPiiTfpwN/WJUuWyCy8fffdVzruqlWrJMmCBQtiy8Fu3rxZBsCjjz4qW8xw6I033siXl84WhpLl\n48sQLBfdtGlTWet87dq1Uqy2gyWtGZBcJ972wgsvyGJNHLPcLasQQmxZF520LI3bv3//QMtu8uTJ\nMnBpP2kpD2JdHbRNsGHBNv3SuLbDj4dk+vMfS2VTFm265ZZbxG3BfUEsNMU17482GBkCIID3AUX4\nnnvukVU2vajQ93C36aqZ6m7HFeenefPmyfhk9Uz6If0Ndx3LW3PMAmfaJxkrXNcVf3G9P/LII9Lv\nScvYgc9A8BdWzfVSly5d3PTp0128sYT1QxrGEnnnzp0byz5jxgwZE1zr3r27+/DDD2PXvDtYYrfe\nequ0kzbBh1hMUSnRmAvLC7SswmxLFyZzNuStV6+exJzU5ZbqLLbbb79dHjRrmTNp4qKLLhJmzQNe\nsWKFQIDJjbn/9NNPu5tuusn16dNHOiAxJvzJU6ZMcYcddpgIgGSY0XET5aPj0sGx1CZMmCBtoSPS\nBm0HHZsBSZobbrhBBgfLi7OGOuuev/fee9IM1konLUyf69wPa7v76YQTThChgLKBJskCdAwKvjTB\nYlC4IXFhct+QtsOLh7dMtE/qZlDgBvnhhx/kXlg2G3rggQekHurq1KmTTNdnESojQwAEGEt+pq7I\nsDw6Shd96KmnnpI+yZdUghYUREAheL755hvphzBqysWrQ14WOWOskQ5ijKtLHQtrxIgR0v9RrlBO\nH3/8cUkHn2BpaC+hrBJjChpLrE3GmEfxY6E1YubdunUTKw6hAc/h+uzZsyV0MHDgQMdy8n6iftIw\nLkePHu0WLVokOJAu2ZgLywv8dRbkOKl7rSCFFqc8WDZozkpYOF4fsJ6Pt+3bt69r27atXEYDQvtZ\nvnx5YPLBgwfHln+l49evX99de+21onU1btzY4W9WYoKD1zWA+d6oUSOxjhLlw8pgEOkys71793Yn\nn3yyI3iqFijMnwGEoGXpaAQDx/zQ6hg8RxxxhLgLuKeWLVtKs+h43CuWU61atbSpok3qsWqODBAE\nDxrSpk2bxKVx+eWX5/N5e/GIFZa3M3XqVBlUHTp0iNU7bdq0WBKW0YXAiIE9cuRIWVo3lsB2DIE4\nCDCpB/c/gmC33XZz69atk3EVJ7mcZnIAcWsULl4PwR1Xs2ZNGU8wbwQIyqufEAQnnniinGZM4jVA\nyUtEtMk/lhjTe+65pyhY5KVchB7vR2r53AdtZLzzC6JLL73U8du6datDgTvppJNiHp1EYw4lcfz4\n8WLhJeMFQfWmeq7EC51UAfGn985c03XXsUboPH5Sps95BNMBBxwQM/M5h1D57LPP2JXOjaWiRPAT\nKylZPrQXLBIsLyUEK9qS1o8Ag1hXnWteLY9208lwlSEsEHBKuk8bdGDoNf8W6wQhpfdAPRBlK2l7\n9Fi3YMD9KrEme4MGDfRQtrQNYYkgy9alvfPdkB0UOQJ4H2D+WDsaY8ZtxexalCUsCD/Rb3WiVIUK\nFeSyjpdks1K9M+bYV4vIX0eyY9zS/I455ph8SVEkGa8oiiiPd999t8zGQ6n0jh/NtHjxYjds2DBx\n7+l4VA9BojFXWF6g9YfdmtBJgpTGcEimkymYmef1lQYVQRr/FxywMJRgpvz8lCxfw4YN3SmnnOKw\nwPyEWR+W0AbpmGhxCENIXQJ169ZNWgzuOGbiYTWSnplCnTt3TpqPBAjjpUuXxtLSbqbcn3HGGXIO\n10G/fv0khojgMTIEwiCA4oWyctppp8WS44rDcsbDESR0Ygkj3EHZg5HjBWAfRQxLJR4hEJnajQss\niBjv/PCM4B7HEiKuo4JF8xDnYiwjaKtUqSJCav78+XI50ZgrLC/Q+sNuS3xMJywQ8dKhOTBpgAfO\nZASsHb9WHpQX9xd+Xsz89evXS+wk3pcdvPmT5cPcxuTHVYcJTTAfMzre1G5v2d790qVLi1sOlwBa\nFgP2xhtvlE6L4MPaufrqqx3vJ0DVq1eXgURnx3zXwGz58uUlL7OFEhE+9ueee06SMPOIdhPYJOhK\nUNZLCDSYBL5rBBK+cOo0MgQSIQDzhhE/9NBDInxg+jBylD3V+BPlj+oaHgOEHy4yhA0TDjhW8o+l\nFi1aiNL15JNPyphGWWXSAu5rPBuMb5QyvA9NmjSRYoJiOlh6EJMoEDaUp5RozCXjBVpGVNsSZekQ\nA0hGYdJ4y0DTJpCH5oIWgaZRrly5fG4zTa8zZjgmLQKL2SQE6jkmKK4zzTSPf5ssH9+zwyLBhUBH\nxoVFXKZy5crCoCnP2w6O6VRe0usIGQQLAVMITZD7g3AhMhGgS96MGtx1aGKY/1gjuNU4P2fOHMeA\ngfAlM0i0bM5595lswOwb4jitW7d2d911l7jmmB10zjnnxNpOPl645d60bM7hK8d1YmQI+BHQfoZV\nAaP/17/+JfFLTccYTEXoaHm61XLY6jn/mPKmYQYv8SGUJn6ML4SF5vGPJWZ+8ioH4+r666+XonCh\nETdFuBBnRWhAjHdinLvuuqsce/8xUYhxyJhCOUZYqfci2ZhLxAu8dUSxXyqvkO3M3sgkFXYRNwQJ\nwcBUCA0fzSgsbdy4MV9sJGw+TGtccQiFVChZPjojbQqKLaVSD2nVclHftuZX94Aes2X2DjEYHUDc\nG9olAz4R0V4GLD98zwyGNm3aSBbKYJYfvmvcCEaGQGERwLuAhYzVrn21sGWmmp+xgkfA7wbTcvxj\nifN4LBjTKtw0LWOR8Z6Mj8A3KIN03jLCjrl4vEDbEWaLgpmISoTQ4QYRPHSyMESAMBWBE6ZMSxMe\nAUx/3u9hViH+dqaJMqiYvp1sUIWvxVIaAoaAIpDOMZczQkfBtW12IEAc6dVXX5XJGcwQxAVhAic7\nnp21MjsRSNeYSyZ0SlRMJzu7Qm62ujh+uSI3n4Tdda4gUFzGXP4Ic66gb/dpCBgChoAhkBEETOhk\nBHar1BAwBAyB3ETAhE5uPne7a0PAEDAEMoKACZ2MwG6VGgKGgCGQmwiY0MnN5253bQgYAoZARhAw\noZMR2K1SQ8AQMARyEwETOrn53O2uDQFDwBDICAImdDICu1VqCBgChkBuImBCJzefu921IWAIGAIZ\nQaDEfJGAT+HzCyK+s5bqMtVB5dg5Q8AQMAQMgcIhUCKEDh/7ZK2bRMT6FD169EiUxK4ZAoaAIWAI\nFDECJULo6NelWVHP//VoBBIfoGOxMCgVwcPiYix7612yWgrJ+8fqfCxcxtovyZa11Ty0g3VqUmmD\n5i3oliUFWNWTxdaOPPLIghZj+QyBEo0AS23wQUwvsWYNK4/6afr06eJV6dWrl/+SHYdAoEQIHb1P\nBM6BBx6oh7FtjRo1ZJ10FkqCwjJ9hETXrl1j5Xh3WLeC9TBSoWXLlslKgKnkKWxaFmK79tprZVEn\nFl0zMgQMgR0ReOGFF9ztt98u40SvssQzK//6iZVI582b5z9txyERKFFCJ9E9416DUhU8kingX7Nm\nzfItBxuQJOOnWJyPVUtZQZBFnIwMAUMgGAGWlWaV4FQXgwwuzc4mQiAnZq/h0iLmM2vWLFnFD1db\nvEkHfrCWLFniWCKapZbplKtWrZIkCxYsEGbOwebNmx1CDa2IrZrdb7zxRr68LHkdj3DlPfjgg7K4\nGXX1799f1kVnmVqOu3Xr5r7//nvJzpLSCBOWcG7atKmsRYNF4yc0N5agZoloI0PAEIiPAMvIV69e\nXRKwoGAyIg3ji/HHgoQsRKjENVYmhhew5DRjmdU8IXjPeeedp0lly9LvuOwg+ABLuTPuKRuaMWOG\nLHgIH+jevbv78MMP5bz/36+//upYmpt6yQsfYlVepYcffjh2Df6BgGVpayhRmzV/VNvSURVUXMup\nV6+euNZwu/FLdRYbHYsHPXHiRPfLL7+4iy66SB4QD3jFihVy27jaMLlh/DfddJNjfXMEHTEmlmGe\nMmWK+IbHjBkTF6avv/7aPfLII5KXzkFnOP/88yWmNHbsWOlobCE6MAKOeA1rwpPu6quvdgg5JQTs\nc889J+3Rc7Y1BAyBYATWrFnj3n33XXf00Uc71p1hTDGm49Fbb73lWN6d8YfXo2/fvg6eAN12222y\nCi6MHUUSPnHxxRfLNXgI8SMvochqXBo+MHLkSIcggp8gNOA58JLZs2cLDxs4cKCMfW8Z7D/++OOS\nZvjw4W706NHiyn/ggQck2aRJk6RdCKJx48a5H374wU2YMMFt2LBBridqsySI8F+Jd6+xPjnLIith\n4cCQwxKdqW3btpJ86NCh0imXL18emH3w4MFynYsIiPr160s8hbXKGzduLJ06MOOfJzt37hzLj0ts\nr732Es2EyyeffLIsya35a9eu7aivTJkyUvbcuXPdyy+/7I499lgRjtdcc40IHNIZGQKGQGIEVq9e\nLQlQ+MqVKydWzDnnnCNMvHTpHXVzlllXjwbjHIWRyUV4FsaPH+/gFS1btpQyYejwEKypMES5WDqQ\nWiq4/xByvXv3ll9QOZdeeqnjt3XrVhEm8BD16EydOlW8Jer1oE3Tpk2TYlCaE7W5Vq1aQdUV+NyO\naBa4qJKZ0TtzTRk4VkwQVatWLXYawUQgEoGj1KhRI90N3HrzV6hQwe22226xdMykoXMoIcQQOEpo\nZ0xUgIhb4SrALYjZzO+3336TzqjpbWsIGAL/Q+DZZ591eCIYVw0bNnQ33nijw/rRMfW/lH/sIXSU\nmL3K2MWKwYW+adMmUTj1OkIJiqesajrdqpuPY3jAvffe67BUsKgQRl6PhuZhS9wWNzztP+6448Si\nwRsCffbZZ46JVkply5Z1DRo0kMMo2qzlhtma0EmCksZwSPbdd99J6jp16iTJ5Rxp/J0skbmetEBf\nAn/ZX375pdOB8Morr4hpjZBDGKE50bHojEuXLvWVZIeGQG4jgFKGK4uJN0ow5YLQ7rvvLnFjrwtN\n9+vWrSuKImOROiEUSayYRHTKKae4V199VV59IEaEqw3B5ie8GwgsLC7iPgggJRRg79hHQC5cuFAu\nJ2uzlhHV1oROEiSHDRvmmDSAr5XJCFg7qiEkyopvmEkIxF7Wr18vPt4XX3wxlgWhQRxGJwfELoTc\nwU+M75eyCWISA2rVqpXkRmNjerT+0NrQxDj2Wm4hq7JkhkCJRgCPAdYD8VjipRs3bnR33323jJkw\nY90LDq44XOFMJOC9H5RBxh8KIIooVg8Cg1gQwga3XJAA0TIXLVokE5YQELi5mjRpIpfUgtF0bDWm\nhHdl/vz5+WbXYiE98cQTDvchkxmYsKCUrM2aLqptwcR5VLVHXA4vgiajMGm8ZTBjjUAe2gkdh5lj\n+Hy9bjNN7z1HWgQWs0kIKHLcqVOnmF8XFx3aVZe8gGHVqlVdkN/YWx51eI9btGjhCGYy04W4FS+x\n0tkhr3nOMZoMbVb3IOeMDAFD4H8IMI4Y64ceeqicxGtAwD1oXP4vV/49HZ8IGRTKdu3aSQLiPPAN\nCM8DkwqYDMCPNAgSrUe3kjjvH5OfiAdpjAflEWUTd7ufBg0aJDEdFFDGOjEdtbJat24tvILJA1wn\nXoW1o5SozZomqi0Bh+1eszKqglMph1kj3mB/KnlJiyBJdX49mojXx5msTrSfSpUqJUu2w3XMZ4KB\nlStX3uEaJrY3LrNDgjgn0MLwNdP5mH2CT7kg5cQp3k4bAjmLALwQKyKK4DkBfWKpTADwEy+Wcw2F\nMQzBK+BBQXzEmx9+g7VGOhWCXCfegwBq06aNJIcnMbOWeBHuO6VEbdY0yba8VJ+ISoSlg/BAiOi0\nw0Q3zDWYdCoChzwFETjk48HH6yhRCIogjYd6jQwBQyB1BPh6SVREXChebIjPUvELS/CKeHzEWwb8\npkqVKt5Tso+Qw5JDuceKwyXPljCAlxK12ZuuMPslQugAQKpCpDCgZTovL6N5TeNMt8fqNwQMgeKN\nAG5DBA0TEpgQhQDCtRdGkEV9ZyVG6EQNTHEuT2epFec2WtsMAUOgeCFAPIlfpslmr2X6CVj9hoAh\nYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9\nhoAhYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6\nCVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRM\n6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQ\naQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKG\ngCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdutGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAImdHLoYdut\nGgKGgCGQaQRM6GT6CVj9hoAhYAjkEAL/D4vOrXLOxYTpAAAAAElFTkSuQmCC\n" } }, "cell_type": "markdown", "metadata": {}, "source": [ "### Run Panorama Application\n", "\n", "Test Utility enviornment setup will spin up an ARM64 EC2 instance with Jupyter environment pre-installed. To run our bird classification application, Please package all the files in 2_deployment directory using `tar -czvf bird-demo.tar.gz ../2_deployment` command, then upload and unpack in `aws-panorama-samples/samples` directory of the EC2 instance. Your test utility folder structure should look something like this:\n", "\n", "![test_util_folder.png](attachment:test_util_folder.png)\n", "\n", "Run the `bird_demo.ipynb` notebook to test your application code and model.\n", "\n", "![Panorama Test Utility](statics/panorama_test.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clean Up\n", "\n", "---\n", "Please do not forget to clean up after you are down with this lab.\n", "\n", "1) You can manually remove the label jobs in the SageMaker Console.\n", "\n", "2) To clean up your pipelines and model groups using the clean up code below.\n", "\n", "3) Goto Cloudformation and delete stack to remove your Panorama Test Utility setup." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def delete_model_package_group(sm_client, package_group_name):\n", " try:\n", " model_versions = sm_client.list_model_packages(ModelPackageGroupName=package_group_name)\n", "\n", " except Exception as e:\n", " print(\"{} \\n\".format(e))\n", " return\n", "\n", " for model_version in model_versions[\"ModelPackageSummaryList\"]:\n", " try:\n", " sm_client.delete_model_package(ModelPackageName=model_version[\"ModelPackageArn\"])\n", " except Exception as e:\n", " print(\"{} \\n\".format(e))\n", " time.sleep(0.5) # Ensure requests aren't throttled\n", "\n", " try:\n", " sm_client.delete_model_package_group(ModelPackageGroupName=package_group_name)\n", " print(\"{} model package group deleted\".format(package_group_name))\n", " except Exception as e:\n", " print(\"{} \\n\".format(e))\n", " return\n", "\n", "\n", "def delete_sagemaker_pipeline(sm_client, pipeline_name):\n", " try:\n", " sm_client.delete_pipeline(\n", " PipelineName=pipeline_name,\n", " )\n", " print(\"{} pipeline deleted\".format(pipeline_name))\n", " except Exception as e:\n", " print(\"{} \\n\".format(e))\n", " return\n", " \n", "def delete_sagemaker_project(sm_client, project_name):\n", " try:\n", " sm_client.delete_project(\n", " ProjectName=project_name,\n", " )\n", " print(\"{} project deleted\".format(project_name))\n", " except Exception as e:\n", " print(\"{} \\n\".format(e))\n", " return" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import boto3\n", "import time\n", "\n", "client = boto3.client(\"sagemaker\")\n", "\n", "delete_model_package_group(client, model_package_group_name)\n", "delete_sagemaker_pipeline(client, pipeline_name)" ] } ], "metadata": { "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Data Science)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/datascience-1.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 4 }