{ "cells": [ { "cell_type": "markdown", "id": "ae16b7f1", "metadata": { "papermill": { "duration": 0.011857, "end_time": "2022-04-18T00:22:50.574201", "exception": false, "start_time": "2022-04-18T00:22:50.562344", "status": "completed" }, "tags": [] }, "source": [ "# Hyperparameter Tuning with the SageMaker TensorFlow Container\n" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n", "\n", "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-2/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "---" ] }, { "cell_type": "markdown", "id": "ae16b7f1", "metadata": { "papermill": { "duration": 0.011857, "end_time": "2022-04-18T00:22:50.574201", "exception": false, "start_time": "2022-04-18T00:22:50.562344", "status": "completed" }, "tags": [] }, "source": [ "\n", "This tutorial focuses on how to create a convolutional neural network model to train the [MNIST dataset](http://yann.lecun.com/exdb/mnist/) using the SageMaker TensorFlow container. It leverages hyperparameter tuning to run multiple training jobs with different hyperparameter combinations, to find the one with the best model training result.\n", "\n", "## Runtime\n", "\n", "This notebook takes approximately 10 minutes to run.\n", "\n", "## Contents\n", "\n", "1. [Set Up the Environment](#Set-Up-the-Environment)\n", "1. [Data](#Data)\n", "1. [Run a TensorFlow Training Job](#Run-a-TensorFlow-Training-Job)\n", "1. [Set Up Channels for Training and Testing Data](#Set-Up-Channels-for-Training-and-Testing-Data)\n", "1. [Run a Hyperparameter Tuning Job](#Run-a-Hyperparameter-Tuning-Job)\n", "1. [Deploy the Best Model](#Deploy-the-Best-Model)\n", "1. [Evaluate](#Evaluate)\n", "1. [Cleanup](#Cleanup)\n", "\n", "## Set Up the Environment \n", "Set up a few things before starting the workflow:\n", "\n", "1. A boto3 session object to manage interactions with the Amazon SageMaker APIs. \n", "2. An execution role which is passed to SageMaker to access your AWS resources." ] }, { "cell_type": "code", "execution_count": 2, "id": "997c9297", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T00:22:50.599001Z", "iopub.status.busy": "2022-04-18T00:22:50.598452Z", "iopub.status.idle": "2022-04-18T00:22:52.560333Z", "shell.execute_reply": "2022-04-18T00:22:52.559646Z" }, "papermill": { "duration": 1.976681, "end_time": "2022-04-18T00:22:52.562437", "exception": false, "start_time": "2022-04-18T00:22:50.585756", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "import os\n", "import json\n", "\n", "import sagemaker\n", "from sagemaker.tensorflow import TensorFlow\n", "from sagemaker import get_execution_role\n", "\n", "sess = sagemaker.Session()\n", "role = get_execution_role()" ] }, { "cell_type": "markdown", "id": "337e9ddb", "metadata": { "papermill": { "duration": 0.012028, "end_time": "2022-04-18T00:22:52.586824", "exception": false, "start_time": "2022-04-18T00:22:52.574796", "status": "completed" }, "tags": [] }, "source": [ "## Data\n", "Download the MNIST data from a public S3 bucket and save it in a temporary directory." ] }, { "cell_type": "code", "execution_count": 3, "id": "e2a54d88", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T00:22:52.612241Z", "iopub.status.busy": "2022-04-18T00:22:52.611645Z", "iopub.status.idle": "2022-04-18T00:22:55.059906Z", "shell.execute_reply": "2022-04-18T00:22:55.057654Z" }, "papermill": { "duration": 2.463274, "end_time": "2022-04-18T00:22:55.061919", "exception": false, "start_time": "2022-04-18T00:22:52.598645", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "import logging\n", "import boto3\n", "from botocore.exceptions import ClientError\n", "\n", "public_bucket = \"sagemaker-sample-files\"\n", "local_data_dir = \"/tmp/data\"\n", "\n", "# Download training and testing data from a public S3 bucket\n", "def download_from_s3(data_dir=\"/tmp/data\", train=True):\n", " \"\"\"Download MNIST dataset and convert it to numpy array\n", "\n", " Args:\n", " data_dir (str): directory to save the data\n", " train (bool): download training set\n", "\n", " Returns:\n", " None\n", " \"\"\"\n", " # project root\n", " if not os.path.exists(data_dir):\n", " os.makedirs(data_dir)\n", "\n", " if train:\n", " images_file = \"train-images-idx3-ubyte.gz\"\n", " labels_file = \"train-labels-idx1-ubyte.gz\"\n", " else:\n", " images_file = \"t10k-images-idx3-ubyte.gz\"\n", " labels_file = \"t10k-labels-idx1-ubyte.gz\"\n", "\n", " # download objects\n", " s3 = boto3.client(\"s3\")\n", " bucket = public_bucket\n", " for obj in [images_file, labels_file]:\n", " key = os.path.join(\"datasets/image/MNIST\", obj)\n", " dest = os.path.join(data_dir, obj)\n", " if not os.path.exists(dest):\n", " s3.download_file(bucket, key, dest)\n", " return\n", "\n", "\n", "download_from_s3(local_data_dir, True)\n", "download_from_s3(local_data_dir, False)" ] }, { "cell_type": "markdown", "id": "05be460d", "metadata": { "papermill": { "duration": 0.012237, "end_time": "2022-04-18T00:22:55.087821", "exception": false, "start_time": "2022-04-18T00:22:55.075584", "status": "completed" }, "tags": [] }, "source": [ "## Run a TensorFlow Training Job\n", "A TensorFlow training job is defined by using the `TensorFlow` estimator class. It lets you run your training script on SageMaker infrastructure in a containerized environment. For more information on how to instantiate it, see the example [Train an MNIST model with TensorFlow](https://sagemaker-examples.readthedocs.io/en/latest/frameworks/tensorflow/get_started_mnist_train.html#TensorFlow-Estimator)." ] }, { "cell_type": "code", "execution_count": 4, "id": "7f969cbc", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T00:22:55.114069Z", "iopub.status.busy": "2022-04-18T00:22:55.113763Z", "iopub.status.idle": "2022-04-18T00:22:55.156028Z", "shell.execute_reply": "2022-04-18T00:22:55.155341Z" }, "papermill": { "duration": 0.057727, "end_time": "2022-04-18T00:22:55.157993", "exception": false, "start_time": "2022-04-18T00:22:55.100266", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "est = TensorFlow(\n", " entry_point=\"train.py\",\n", " source_dir=\"code\", # directory of your training script\n", " role=role,\n", " framework_version=\"2.3.1\",\n", " model_dir=\"/opt/ml/model\",\n", " py_version=\"py37\",\n", " instance_type=\"ml.m5.4xlarge\",\n", " instance_count=1,\n", " volume_size=250,\n", " hyperparameters={\n", " \"batch-size\": 512,\n", " \"epochs\": 4,\n", " },\n", ")" ] }, { "cell_type": "markdown", "id": "f65f0125", "metadata": { "papermill": { "duration": 0.012502, "end_time": "2022-04-18T00:22:55.183196", "exception": false, "start_time": "2022-04-18T00:22:55.170694", "status": "completed" }, "tags": [] }, "source": [ "## Set Up Channels for Training and Testing Data\n", "Upload the MNIST data to the default bucket of your AWS account and pass the S3 URI as the channels of training and testing data for the `TensorFlow` estimator class. " ] }, { "cell_type": "code", "execution_count": 5, "id": "72373e0c", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T00:22:55.209811Z", "iopub.status.busy": "2022-04-18T00:22:55.209019Z", "iopub.status.idle": "2022-04-18T00:22:56.021349Z", "shell.execute_reply": "2022-04-18T00:22:56.020578Z" }, "papermill": { "duration": 0.827712, "end_time": "2022-04-18T00:22:56.023177", "exception": false, "start_time": "2022-04-18T00:22:55.195465", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "prefix = \"mnist\"\n", "bucket = sess.default_bucket()\n", "loc = sess.upload_data(path=local_data_dir, bucket=bucket, key_prefix=prefix)\n", "\n", "channels = {\"training\": loc, \"testing\": loc}" ] }, { "cell_type": "markdown", "id": "d5444231", "metadata": { "papermill": { "duration": 0.012304, "end_time": "2022-04-18T00:22:56.048212", "exception": false, "start_time": "2022-04-18T00:22:56.035908", "status": "completed" }, "tags": [] }, "source": [ "## Run a Hyperparameter Tuning Job\n", "Now that you have set up the training job and the input data channels, you are ready to train the model with hyperparameter search.\n", "\n", "Set up the hyperparameter tuning job with the following steps:\n", "* Define the ranges of hyperparameters we plan to tune. In this example, we tune the learning rate.\n", "* Define the objective metric for the tuning job to optimize.\n", "* Create a hyperparameter tuner with the above setting, as well as tuning resource configurations.\n", "\n", "\n", "\n", "\n", "For a typical ML model, there are three kinds of hyperparamters:\n", "\n", "- Categorical parameters need to take one value from a discrete set. We define this by passing the list of possible values to `CategoricalParameter(list)`\n", "- Continuous parameters can take any real number value between the minimum and maximum value, defined by `ContinuousParameter(min, max)`\n", "- Integer parameters can take any integer value between the minimum and maximum value, defined by `IntegerParameter(min, max)`\n", "\n", "Learning rate is a continuous variable, so we define its range\n", "by `ContinuousParameter`." ] }, { "cell_type": "code", "execution_count": 6, "id": "af886a61", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T00:22:56.074525Z", "iopub.status.busy": "2022-04-18T00:22:56.074188Z", "iopub.status.idle": "2022-04-18T00:22:56.078214Z", "shell.execute_reply": "2022-04-18T00:22:56.077471Z" }, "papermill": { "duration": 0.019288, "end_time": "2022-04-18T00:22:56.079878", "exception": false, "start_time": "2022-04-18T00:22:56.060590", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "from sagemaker.tuner import ContinuousParameter, HyperparameterTuner\n", "\n", "hyperparamter_range = {\"learning-rate\": ContinuousParameter(1e-4, 1e-3)}" ] }, { "cell_type": "markdown", "id": "736ec34c", "metadata": { "papermill": { "duration": 0.012586, "end_time": "2022-04-18T00:22:56.105258", "exception": false, "start_time": "2022-04-18T00:22:56.092672", "status": "completed" }, "tags": [] }, "source": [ "Next we specify the objective metric that we'd like to tune and its definition, which includes the regular expression (regex) needed to extract that metric from the CloudWatch logs of the training job. In this particular case, our script emits average loss value and we use it as the objective metric. We set the `objective_type` to `Minimize`, so that hyperparameter tuning seeks to minimize the objective metric when searching for the best hyperparameter value." ] }, { "cell_type": "code", "execution_count": 7, "id": "ba96e95b", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T00:22:56.133872Z", "iopub.status.busy": "2022-04-18T00:22:56.133177Z", "iopub.status.idle": "2022-04-18T00:22:56.137554Z", "shell.execute_reply": "2022-04-18T00:22:56.136811Z" }, "papermill": { "duration": 0.0215, "end_time": "2022-04-18T00:22:56.139188", "exception": false, "start_time": "2022-04-18T00:22:56.117688", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "objective_metric_name = \"average test loss\"\n", "objective_type = \"Minimize\"\n", "metric_definitions = [\n", " {\n", " \"Name\": \"average test loss\",\n", " \"Regex\": \"Test Loss: ([0-9\\\\.]+)\",\n", " }\n", "]" ] }, { "cell_type": "markdown", "id": "520dd1d8", "metadata": { "papermill": { "duration": 0.013119, "end_time": "2022-04-18T00:22:56.165202", "exception": false, "start_time": "2022-04-18T00:22:56.152083", "status": "completed" }, "tags": [] }, "source": [ "Now, you'll create a `HyperparameterTuner` object. It takes the following parameters:\n", "- The `TensorFlow` estimator you previously created.\n", "- Your hyperparameter ranges.\n", "- Objective metric name and definition.\n", "- Tuning resource configurations such as the number of training jobs to run in total, and how many training jobs to run in parallel." ] }, { "cell_type": "code", "execution_count": 8, "id": "7787f38f", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T00:22:56.191842Z", "iopub.status.busy": "2022-04-18T00:22:56.191216Z", "iopub.status.idle": "2022-04-18T00:58:05.235952Z", "shell.execute_reply": "2022-04-18T00:58:05.235332Z" }, "papermill": { "duration": 2109.059965, "end_time": "2022-04-18T00:58:05.237647", "exception": false, "start_time": "2022-04-18T00:22:56.177682", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "." ] }, { "name": "stdout", "output_type": "stream", "text": [ "!" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] } ], "source": [ "tuner = HyperparameterTuner(\n", " est,\n", " objective_metric_name,\n", " hyperparamter_range,\n", " metric_definitions,\n", " max_jobs=3,\n", " max_parallel_jobs=3,\n", " objective_type=objective_type,\n", ")\n", "\n", "tuner.fit(inputs=channels)" ] }, { "cell_type": "markdown", "id": "a2e7a3ad", "metadata": { "papermill": { "duration": 0.125275, "end_time": "2022-04-18T00:58:05.492889", "exception": false, "start_time": "2022-04-18T00:58:05.367614", "status": "completed" }, "tags": [] }, "source": [ "## Deploy the Best Model\n", "After training with hyperparameter optimization, you can deploy the best-performing model (by the objective metric you defined) to a SageMaker endpoint. For more information about deploying a model to a SageMaker endpoint, see the example [Deploy a Trained TensorFlow V2 Model](https://sagemaker-examples.readthedocs.io/en/latest/frameworks/tensorflow/get_started_mnist_deploy.html)." ] }, { "cell_type": "code", "execution_count": 9, "id": "f68e54b6", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T00:58:05.746495Z", "iopub.status.busy": "2022-04-18T00:58:05.745833Z", "iopub.status.idle": "2022-04-18T01:00:06.974633Z", "shell.execute_reply": "2022-04-18T01:00:06.972920Z" }, "papermill": { "duration": 121.489524, "end_time": "2022-04-18T01:00:07.106721", "exception": false, "start_time": "2022-04-18T00:58:05.617197", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "2022-04-18 00:54:36 Starting - Preparing the instances for training\n", "2022-04-18 00:54:36 Downloading - Downloading input data\n", "2022-04-18 00:54:36 Training - Training image download completed. Training in progress.\n", "2022-04-18 00:54:36 Uploading - Uploading generated training model\n", "2022-04-18 00:54:36 Completed - Training job completed" ] }, { "name": "stderr", "output_type": "stream", "text": [ "update_endpoint is a no-op in sagemaker>=2.\n", "See: https://sagemaker.readthedocs.io/en/stable/v2.html for details.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "-" ] }, { "name": "stdout", "output_type": "stream", "text": [ "-" ] }, { "name": "stdout", "output_type": "stream", "text": [ "-" ] }, { "name": "stdout", "output_type": "stream", "text": [ "-" ] }, { "name": "stdout", "output_type": "stream", "text": [ "!" ] } ], "source": [ "predictor = tuner.deploy(initial_instance_count=1, instance_type=\"ml.m5.xlarge\")" ] }, { "cell_type": "markdown", "id": "a83a99f0", "metadata": { "papermill": { "duration": 0.141011, "end_time": "2022-04-18T01:00:07.388392", "exception": false, "start_time": "2022-04-18T01:00:07.247381", "status": "completed" }, "tags": [] }, "source": [ "## Evaluate\n", "Now, you can evaluate the best-performing model by invoking the endpoint with the MNIST test set. The test data needs to be readily consumable by the model, so we arrange them into the correct shape that is accepted by a TensorFlow model. We also normalize them so that the pixel values have mean 0 and standard deviation 1, since this is the convention used to train the model." ] }, { "cell_type": "code", "execution_count": 10, "id": "11796f4a", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T01:00:07.648835Z", "iopub.status.busy": "2022-04-18T01:00:07.648268Z", "iopub.status.idle": "2022-04-18T01:00:09.768067Z", "shell.execute_reply": "2022-04-18T01:00:09.767399Z" }, "papermill": { "duration": 2.249702, "end_time": "2022-04-18T01:00:09.769837", "exception": false, "start_time": "2022-04-18T01:00:07.520135", "status": "completed" }, "tags": [] }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6EAAABRCAYAAAAjIaCuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABt00lEQVR4nO29d3wdx3mv/8yW0zt67wDBXkU1qkuWbdmyHffe48RO7MS+N/UmcfxLT+zrm+IWJ3G3bFnNVu+iJFLsHQQLiN77wem7O78/DggSEptYgANon88HNrU4u2e+mNmZeWfeeV8hpcTGxsbGxsbGxsbGxsbGZi5Q5rsANjY2NjY2NjY2NjY2Nm8cbCPUxsbGxsbGxsbGxsbGZs6wjVAbGxsbGxsbGxsbGxubOcM2Qm1sbGxsbGxsbGxsbGzmDNsItbGxsbGxsbGxsbGxsZkzbCPUxsbGxsbGxsbGxsbGZs64JCNUCHGnEKJVCHFMCPHHl6tQucRi12jrW/gsdo2LXR8sfo22voXPYtdo61v4LHaNi10fLH6Ni13f60ZKeVE/gAocB2oBB7AXWHqxz8vFn8Wu0da38H8Wu8bFru+NoNHWt/B/FrtGW9/C/1nsGhe7vjeCxsWu72J+LmUn9CrgmJSyTUqZBn4O3H0Jz8tFFrtGW9/CZ7FrXOz6YPFrtPUtfBa7Rlvfwmexa1zs+mDxa1zs+l432iXcWwZ0nfbf3cDGc93gEE7pwnsJXzm3uPBikiEgIjLK2DDwh5xD42LXBwtL4+n6pi9FgR+e656FpA/sNnomFrvGxa4PFpZGu595LYtdHywsjXYbfS2LXR8sfo2LXR8sPI2nE2VsWEpZcK7PXIoRekEIIT4LfBbAhYeN4tYr/ZWXjQHZzQj9LBXreUre23Gmzyx2fbBwNZ6uD+Apee/wmT63UPWB3UZPstg1LnZ9sHA12v1MlsWuDxauRruNZlns+mDxa1zs+mBhazydc2k8yaW44/YAFaf9d/n0tVlIKb8rpVwvpVyv47yEr5t7nLhJkjj90ms0LnZ9sHA1nkGfg0WkD+w2epLFrnGx64OFq9HuZ7Isdn2wcDXabTTLYtcHi1/jYtcHC1vj6+VSdkK3Aw1CiBqyf8T3Ax+8LKXKEQKESTBFQsYABItM4xtJnxM3QAR4aJ6LdVl5I9Uhi1AfLH6NbyR989nPqHkR4hvrGLhKR189BkB0yEftzy2cHaOYx05c9LPfSHXIFdYndAejH1xHtFqgrR4nHnMiJxws+cbgJdXRuciVNnolsdvowieXNKbevIGBjTqpPBPnsEr1P+zCSqVAyvPffBZySV+ucNFGqJTSEEJ8AXicbMSn/5JSHrxsJbsAFI8H4dARPt/MNWtsHJnOIDPpS3++UGiSq9nNZoBlwNfmWuOV5I2kTyIBRnNKn6KiuJzZNuxyIaXEGh1HmiZY5oU94g1UhyxCfbD4Nb6R9M1bPyMEwu9jZKlO5aZOHm/+DQA/iebxja3vJX/Sd54HnJs3Uh1yhfUJVWF8CURWDfLSql+wM23y7NRSnv2fq6/E1wE50kavMHYbXfjkikahOxhr1Km76QS35Lfy6MAyxLcDiPEJZCp10c/NFX25xCXlCZVSPiKlbJRS1kkp/+ZyFepCEJrG0AdXcfirzXzm2ef53eee5mPPvEj/J1ZhbWgGIS7L9+SLEq4VdwIcmGuNc8EbRd914s0A/fNdnpMoLhdqcz0j71lF6181E74vhXKPSuaGFWhV5a/rWW+UOmSR6oPFr/GNom+++hk1GCBZV0DVXSf4aPkWTGlN/wjExS/cz+KNUodcYX1SSnwdMNAdJiUNxk0PfekgWNaV+kpg/tvoXGC30YXPfGtUvF5Y2cDkyjT/VXcvnwod5P2l2xm8qw5WNl7y8+dbX65xxQMTXTGEQrxE4KmY5K2eCXShErcm+WsfWE4Vdb7LZ2NzJoRAWbmEZLGXkWUOog0G9Y19fKBwKyOGj7+98T0UuYtwnTjvee6cQ2gawu0mdmszKb+KlpL4T8QQB49jJRKX5MZic3GooSBWfQXxMg/J8Pl7Rc+Qgas/jjjcjhWLzUEJbS4FNS8C4SDxpnxGlup8Ir+VBkc/Fgp9ZoLjqSIcUQuRyGC/ffOPcDpRAgEm6yBUFOWhWBH/3X0dR46W0jwxxIX5v9jY5CZqOAz5YRJ1eWR8CmmvgmPKwjWSwbG/HXNkdL6LeF6UgJ/BlQEKiofJU9wAhNQ46ZDA8OoL37YQAqGq2b6oMB8zz4/p0jBdKrFSB4ohUVMS10gGbSKFONaJTKYui3fpmViwRqhQFRJVaW4q7QTAlBYWFkiwR1ubXEWoKm3vCeFaPs6/rfxPqrUpitTswfOMHGfdR7/O2yJfouE381zQi0Dx+6Ewjxv+agtfytvK84kS/vcjH2TJNwugbwArmZzvIr7hkNVlHP2gj3fd9Ar/VLz7vJ//aMcNvPzSUhq/UwRH2+aghDaXgtFYwehyD+m3jnNz+TE+H25FQcHC4pl4Lc/2N+I/NoUyPMaV3WezuRCUUBCrvIA/uet+KvQRvrD9g+Q/4Kbxnq22AWqz4LHqyxnY4KfkPe18rGQHH/L38d2Jav6j5QbK/rkCsSX3jVCzOI/U3eN8vGrnzDWXkiYdAMOz8De4hMOB4nZBYT5D1xYyskoi89KUFo7x9PKfciwj2Jqo5f8dupnMsQANPyxBDI5gDo9ckfIsTCNUCFBVUCUOxQCgz4zTmgniHpI4hmJ2h34aal4EigtgcBQZjeasMaAVF2GW5HP8/QGMvAzquEbxFon3V6/Md9EuGsXlQpSX0HdHCekgpIOS6286wI2hVmq1KfxKtks7kJYMmUGOporRopfkJZ8TeITOamcvVjCDFfTCsA452u4WI2ogwOG/bSaveowPV2zmrYE9gH7e+z5Z+CLlt47zc/c1eNuLKX0+itY7itH9mgB+Cw7F60U2VjO+zM94g0KqMoVQJcqgg8IdEHqsBXNy6oLPY88rioridjG8xMPINRk+V7eD671ZAxQgI03++ol3Etmr4Os+hhWNznOBbQASa6oYXKPjVVK8FGvE97yHwPHool4316orSVfkcfx9DtRI9jydOM1HXMoLOzplGgpar5NgKxQ+34fVP4gVj1+RMttcHFOVHsbXpvlk4UGudZ8AXNzkOYK+1OQf3vd2wsuuIf9Huy7pXOUVQwjEumUMrfXzkfonuN7bCgve5Mx6qCnhMJmmMqJVLiarFdIRC0/dBLXhNm4L9FHomKRUH0MXKqVamk2eYyjLJK01xTzW0IzY3kTFb0aho+eyjyUL0ghVnE4Uvw/NaRLQshPbUUvjaKo463o0FV+crn9CIBwOgAt+iYXTCYV5TC4J4ddVlGEHVk9vbv19FBXF68EsL2Ci0ceH73yeuwO7eSbWzHdjd+J7yIE0MrlV5gtBCJRwiGRVhNimGDWFI6yOdPO5yIuUa27And29B/YkKzmSLKZ1sgjH+OU5zzyfqEJQrjnR3RlMj452mc5o25wf4XQi8sL87e2/4P3+bJTUjFSIW2lS0sA8bcqrIggoLlSRNV5ucltscu3BtynFr2uXM9ldQChtwkI0QoVAaHo28JfbBXlhxpb6Gbxacv26A/xl6SP4FcEPJlbyPeMOwi/6ELEEcgEYoYpDRynMZ6pSsKHxBO8O7KZSy7qOxWWaYdOk+GVB+MVOjKGheS6tzUmiZRqJ5iQuJcPRqUIKd06hdg9hzHfBrhRCYOYHmKh18bXbf8H7fdm2eLK/gawX2/mwkAyYCb4xdAP3hdYTbI/giMbANkJzBqFpJPIVltV3sslzhBrNBUCj7qBR7+KFqw7zkq+egnscOWmECk1nfImPiSbJu/17KVA1ThqhplQQJogF6E4i3G5kWQGjy9yMLbNoXN7Bxrx2/ihvN6oQMwuXWRSCioOgAvV6B6a/na8UvMBbtE8TOxzAPzxmG6EAqRuW07/BwZ+tvZe7vCdQcOMVBgVaFMOlIJ2O7G7pQjNazoEaCiLCIYZuLEVYEPnF7nOHixYCNRhg4o5m+m6Q/P0dP+c7nTfSdqyM5j+JY46Nza2Ac6DWVXHsU0WEVgzzyZpnudvXSr7qpkY/yPeXX0Py9lV4trVdMXeAK4HQHSgBH22/XYtYPskv1n2foJLBJSCizs77lJEm/9/zbye0T6Pk2SEqh4/aO/k2F83UXasZ3KDQ4Bggmw4QHo4HeWp8GU8cW4KROtXtC83iF9d9h3VOx8w1VSh8PrKHelc/f7bmg+gJH+49cyziEjm5+jt1XQ3DKzRW39nC+uAuyh0jVOvDVKipmffwE8F9fKv0Rqy8AGJs/IqdfblcqIEAxopaDn9S422rtvO/C5+lYFpLXKb5y4FNPHRgJY1Ho1ijudPPv+ERAtMpcLozjJseuqdCeA+1YSQWp4eI4nKhFBeS8jkuOTjWtpSgxyijwjXKumVt7P5kBXXfLkPYCyw5geL1Yi2rZXSVyc9q7qVUfe0O4u8UP4OumAw4zu+RM9eo4TAU5lH5uaP8S+mTlGtuFLIL55NWkmOpeiItJq7u6II71mAuryX5V5P8VslO3uTfT0RJ4xKgivPnH1WFIKI6+WDddr73vutx9xdB/8BlLd/CMkKFQHE6iZbrpJoTLHX2EJ4+OGwiyEgVxZRXPMrcnDKt2WiuZqzZw/B6EyWpENlVjTowgvnqTlhRs6vkBfkYZREGNkJDcw+3u/t4OjBEhz8PlBzZlRICtbGOyRV55K0aZF1+Ny4lwz3R5ejCZImzl0TUhZYwwbRAUVGDAWQ6nXXDyeFFBiUSQpbkYy6Z4i01h2nUxcxL35bJMGR52J+swJIKGani6dAIdmSwjp5AGgtzXVyEg8SrQwS1xbU6rebngZSYI6Oo4TAiFEDq2kwEbunSsdw68RI3pjN7TU1LtLiF50Av1vDInLrAZ7wKht/i0ehKtieinEgV8ETnEsZ7AvjaNDynLUJLFT4T+AirCnu5OdTCOlcXyxxugoqbkBJHSYGSzt33DKYnu0UFoChITSVdFiLj00jkqYw1g6tpnE8Vv8AKxyROoZCSFjELtiSd9Bhhnhtfgt7pRCQmkDncpwDZNldezGSNi41LjnBLsIUiNTsGZqTJsGnyUl8t/l0u1JFhzHRuG9Svm+kxQAR8ZMoiGD6dZOTUNMYRNdFiJs7DPVixeO64ISsqqs9LKg+ai/oxpUIio+NOJBeG+/frROgOlJIiBm8uJVEoSBZYlGljjFgJHotVcSRZQmciTJEzik9NUaRPoJxmqWakyoThYSTjZSAVoHW0kHhKxzBU0pNO9GENNTZxRQwCxeVCyc8j1VBEvNABZ5guCQvUtIVzJJO9ICVq0kCJp5Ht3QiPG+H1QDqDTKcXRECei0U4nSgFefRv8BOpGKZc1dHFa43QCi3OEm8/vXVXo+k6xmU2Zi6J0kImm0O8PbKFNQ4D5TTT6JVUmBdH6/D2plAmphaOESoEWlEh4xVu3lP+PJs8R1jpUDndA+98nNwlLdXHqSgYw3LmXVpKlTOwoIxQ4XCgFBUwulLyd1fdR4OW4eR2edTS6TeCaEmJSFxaQtlcQjiynfmxuz38r7sf5G3eI7ycLOWvej9M0Q4P6nOzjVDF60EJBRm9vpzRpYLn3vNP+BWVCcuiLZqPNeoAMzcGPaHpdLyrEHNNlF0rfsa/ji3jG4dvJb0nDBYoqyYIbneib9uLmUiiuF0YS6vQhqcQbZ057aJr1JcytNrDX665h3f7+jk5kplS8pPxjbw0VEvv1lKElU2hULwrjbtrEjNH6uZiiDUX0LNJo8GZQ4PLpaKopFZWo5gWygtjGEsqGV7lIeMXWNO9ZyrPQhan+Per/4db3VkD/KWkzgPja3nln9cTeVFgdffMaVsVacF/P3MTjjGF/P0mJQeGKTy6/cxl+KZgYEk9f/3pZdx43QG+X/kiAP1GkLwDEneOr/4qRQUM3VyO4QLDK8i7s4er8zt4a3AP1doUJdNGGmTdww6kBS/GmnigeyX9HXk0/DBFfW8PZk9/tk/JYYSmM7whj+H1Fs9UPTZrsheXGfakSpnalk/lN19efC6e08c2jCWVjDd5mHhzjGurWvn38qdxCg0Lyb+ONXBf92rS367Cd2QcDuWGEaq4XVBWhFwZ5ce1D/OzaCVTCSeR+S7YFUKJhBjfUMLn/tf9bHC306zrKAgeS+Tx1UffTcFOiGwb4sTyZhIRhWg1s4w9JSNwDYG/28B3aJjC6AQykwHTRKYzWIkE1hXqT5WiAkavKyPwqW4ebvwFHkWf5bJoYTFqptieKuSrh+9CSoGUgvFBP67uADU/NUhVholWOHBOWLhG0ygvjOXsXOVSUfPziC0t4pqP7+Kt4T1nNEABSlQ313iP8j933k7hLjeu3+TOPGHoqgjjtyfY4D6BU5wyiwxM/qn9TXTsLaV+x26MHHQjPhtCVYleXcXQasGHAwfxiIvfgS5QJ6kPDHFUL3iDG6GqivS6sQIG65w9eJRT28kPTa7hZ4fWU9MdxxqfmMdSXl7U/Dz67yjFs2ScTe5jBBUH6knH9Fev0AmBbKykf32AxO1Rbq06RkjReDBWxr8ev5nEk4VUtWSwYok51/Hqcor1yxld6qP0ti5Whnr4Ys/NPL19OUUvCQp7E2BK4i0+nBNpZHMNgxsDJAsgWWyQt72Agu4+pGmCzC2jTfF4EOUldNzoofDmHlY7u1E45eqYJM0vH76e8CFJRU8KYWYHJkfXCDI6taAHKtOpYHotdJGb01+1oID0sgqGV7iIl13Y31kCoiqeDZ7xlquxypLUl3QRdsXRhIUiLCKOOGXOMZY6RlDwAFCvT/KO0C4eetMqpsoqKf+34TnbDY3sGsPb7wNATZjo/RMwNHLmtjXtjTC2Jo+33LiTu8O7MKXFH/Rt5JEjy2g8MAYDw3NS7gvh5K6nGQmQCbvov9pJosSkakkvbi2DT0/xroJdVOgjVGnxmcBf21KCXYkafnhiI1MJJ6m4jtrnxDMhSBQJMoEi1Op8hle6MB2Q15LBc2IC89CReVb8KhRBvESgFSRQX3XOelcqxFeefx+lLbnVJ14KwulE8XiI3txItExlYnkGT36cinAnHyk6QJOzF6fQUIWCCtzibcEsF3x/zZsQRgjXoflWkEWUl9D59gJWlR5mSmb4p/23o+72wwWch1xwCAGRIImIYIO7nQrVQkGwJ21wz9BVVDxp4umYhMFh/PslPo+T0DH37EeYEjWeRpmIw/AoVjKV9VKw5PS4f+XGSel0kIwoVLnieBQd7VXBaRRUgoqDFY5Bfq/hOUyZnZZ3V0foXBFhc3Mtfm+UAm+MaNpJz5QH87ar8fQLfL0mrqEUaiyDOhZFTkRz6mjU60YIYqvKGFql8/bwbpr0EcB9xo+OWUkOJRsp3G3gPTqaE0eO1EAAq6mKkdWS31m5mQotw0mzaNCM02G46dpRRsFeiczk5rzmTKhFhciSfHrfk+HGukN4RHZ3WkGQkgZxmaHXVPlm/208d6wBpdeFETD551t+zlJHP/X6KdtKF+opm+MKsKCMUFQVy62je9JUa55Zv9o+WoV20Is22I2xiPLbyYCXsZUWd5adoFHPruSbUkExQVindcTTbrtTlT5G1xl8Y9UDvMM7RUbqvDTZwMTWQqqfHMU6cHh+I/FNBwsZb/QydK3BP1Y/TFLq/M4LH6H4JUHw3l0zuxEBpxOaaphYEiR5U5R1ZV0UOSf5zchGCs9w5iAXEB438YYIxqopvtP4U8rVU6tPKZlh1DQpfyaN9vTOWfctnO7t7Ji6QLpMdMxXHXbPEYI+Rpc4MW8Z53MNWy/oFkVYbHC3YaKwf3UFy5w9LHVE8YkzuRy5sabfriLVTZGa4T2rdvKgZ0U2oNgcGaHWgcPoB07991kH++lAZ7GGCGNLBP9U8jIaKlMyxcOHl+Pd7cZqPZBT7uHC5yVdnc9UmZNYscKKtx7mlshh3uU7OjPQniI7GbKwOJiq4tmRRqa25SMscAlwTIAwYKpEBVRQdJTbRgi7kwy4yygQIVyH1ZxxlxSahuLxkCywKI9MzvpdRpocSFZQ8KJG4OiVcVOccxQVJRSE/DADVym4loxx/6r/oVQzZvL3QTZoTcpKk8GkUVeJeY7yHxU3k2jTp/e+5xkhyBT4kFdNcHWojZglUfb7yTuUO+/VZUUoGCEPmYCgSVfRcGAhOZgqZe9AGaXP7Mc82RdObxicaTSXnKPvupKoCsZ0w4laaXShYEn5moBuEVXlvb7u0xaDujClJFP+JC6hoaFiIek2Eny77noeaV/KwOEg3i4PzgkLX68TR78TJZE4d3yPXEVRUVxOxut14vVprnGN4xFnNkABRkzB8WQhvv39WP2Dc1jQsyP8PsaW+ChqHOT3w4fRTjOgu0wnm+ONRA5IwntGLiiAVs4Q9JMo9fK7q57mvYF9WGikpIWJZNg0GTLd7ElW8syBJZQ8qRHa2U+qMsLLG+rxBxPU69n3UzmTL/plZkEZocKhk8x343SeOnNmIZmwkhwbyKfooJndTVokKF4vmTwvRbXDrPR2AVn3gI50PoU7EjjaT0XVUyNhRt7SxNDtKe7d9G3qdZOU1NibhscPN7PkRz1YA/N8iF8I1LwIydXVDL8lxT3XfpcfDl/PE8eWsOSbMUT/CGYmDYqKVllG+wfKMddE+dyyx0hZOtvHq3juOxupOpjAjEZzstNOrazm5r99iZv9hyhV1Vm7FX8+cD0P7l1Nc99kTqwCXhGEnLVqJsT0/+TCOWSng1RYsKKwn3f69+G9gDIpgEuomEiqtRYyEsYtQZTXnrWLKApOoc0yhEoc4wS8ydzQ/yrUhlpijXnU//kh/rbwBZxC54m4zkNj11DxMw3PjmOYOWSAAsSuqWPFX+xlubebascwTfoIfkXgVxznXPio1Ee4Oe8Iyu2StcEurvUeJSl1zFcNsrXaKBkUHi1ewXcKb6GpqxHauuf3bOH0YkHi9lUMrdb44p0Pc5u3BYXsarWFxbfHl/A/xzZSvmM0p3auLxbF70c2VnLk/QHuumU7Xw7dS6kapUrT0MUp03JKpnglGeBXo+t5sauWZUX9tE9EaPjPDHpnX04s7gmHg1i5i5+s/Q66sGjN5FG6OYlj1zHMHBzDLhWhqkzWuImXnBoHLCyeGlvKZL+fEjO3J/PyRBeVP49zbLyJq5sbkUUppCkQY6c8miyHhRZK844le2edZc3Xo9zibaFUTVGoelAQlGpOvpz/Eh8Pv8zoKhdRy03MctJvBPmv49eQeGUt1b8cxGw9Nh9yLxp59XL613rZ+OHdfCDvFTzCcc7P/2ZqBc/0NFAUHc4a3fOM0B2k64twfrifT1a9NGvH28Dko9s/SeFP3eS90o45usDcqfuH8Bom//7EHfxn5bVsqjzOwdES+keCOA+48QxI8nZPsnRsEDkRRYSDGB6VE7E8hnwBIGuEWkiQFj2ZfFrGinGmLv+7u7CMUI+HaIVOvu/UTmdKZmgzHGTGXXh64sjk/DfuS0XoDoTLibmyjpHlbq7Nb6FUH2NKpng8XszD/ctxDkwip7J/B62kGKOigJGVsLKqh9UOjTErQ2sG/uLEO3AedmP29iPnOUiF4nQiiwvou9bJ8vLjFKspnji2BH2fF3HixEzOL7WuilhjHuaaKDdWH+Naz1H+uuPtHGwro74lid4zipFLHYIQCFVFqa1iuN7BB0LbKVAE+rQPfkpmGLUMWsaLcXbr2TPLixDDLXCHknhFtp2lZAbLVMDKulHNNyKRwj0o2dtbxi8Ca6hyDONSLuwMYMxyciJVwIl4Pt2x0GufLSQfLH2Flc4elp02Fm8dr2WoN0Se2X+ZVFw6JyPHTq7MZ2iNwp/lv8J6p8mRTIIfD72ZzfubaG4bxxzIjdXq00n7FD6a/yLVWpqw4uJsrl+no6BQoEbBCXqewVJXDyv0+GvOegFkpMawlcY53S5ExkTOc6A7oekooSDj9RqsnmST5wi1+ikPi4w0eWa4iamuAGKkHSs+z8ctLhVFRcmP0LsxSGTpEJ/J20yNpuIULiwkfWaCLsPDK/F62pN5bB2oZqg7hLtTZ0eTE2kKnMUaaiwAvf3zvpMthMDUBc0OhbaMSU8mjD6exJycPP/NCxFFkMhTkOHUzPtlSklfPIg6peS8C7KVSiH7BwgfLkBNuUkUuhEmOMdPjWGWrpIKebg3uW7Wvbo7wyvlNTT6Blnmyaa18iopVjj6KVYFjTpAAos4cTnARJWHH8Q3YjzrQzmu5ZTXyVmZ9rqbqHAzvtTgjtBB1jhjwJmN0DErSa+h8cuOtUy25FGYnNv4CGdECJSaCsaqXLyr9ABLnT2c3I8/GTzLavPh39eHOTqWkyllzoU0DEQ8QeiwIDEW4InhFeijKu5RQd6hDK6BOHLPoew8Wggy62oZr9NY4x0jpMZmBS7KSMmRZAldnfk0xeOX3ZNyQRmhZlGI+J1RPliyf+bakGlwz+i1+I9qsG3/gndDEpqGmhfGKopw4vPw5sYd/FPJy4yaKVozTv748fcT2aeQ37YdaZoITWP05mpGVgj++R0/YoWjH/DwfKKEH/Vdg/J5D9V9BzHn+yUSAiUvwuCGMNs/+3XaDHhkqomqfxeor+zI7oBO6+/8rWLSq6fYde33aM0obI43MvqtKpa+1IXR05tbBijZlV8lGKDtQ0U4Vo1RpWV3ZU6+yOOWwWOxRo71FRDqBhbBQsmZmCoTfLb5Rcq1BCmpMWAaGCkVNZHIiaijRls7eSc6CHSt42dNtxOttZCOCyuXGlMItkLe/inYcYaDZrrGX37zXdy0uoV/LX8Sp9CxsNjz1BJqX0ghE7ljGCjhMFPX1hD7yAQH1v8EXagcz6T4h7472fuL5TR+8xXMHHFBfTWmQ7BuJsLfhbepbFTAOLg7OWEkeSUVYaNzDJ8yO0z9sJVmfzqf7x+5lmCLmhNnQpWAj0xDKdw0xu4NP5wVuREgIy1at1ZTvFtiDA7Pu9F1qSheD5Ori/nKF+9hrbNr5hiKhSQu0/xwfD2/OrEa9aEwoeMp8nYeJZLsAGkRfdd6xpoUqr5yiB1PNVPb4cOaiuXM5L7DCLM3VgHGQp+pnB2hqkw2mSyt7p25ZmHRNRLCOaIgc2BB8pxIiTQMlBf3EHpJEDrXZ8XsRSw1GGBiWRVPNDXwq5rstUzI4reu3cbtgQPc6s6O/QoCn3DyO+Hd3LVxL5+t+RKRQ/4FcT70ZLDM4TWCP7rpYa519Z7TDXdrsoAfD1yD8z8jNLx4DDMHcrsKh4OetxQRXZvkDyOH0YU+4277crKIrz34Hso2Gxht7fNb0EtAZjIU3XcEDANzcurU4o+Us0ZOoemceI/CppUH+GrRZlynBWYypWTCSvPgiRVU3QdK5+Bl9+JbMEaoGggQL3SzqfIAy11ds35n5uL5s4tA8fuhroKuW0Ik1sf5xPItbPQeQ0Plx5Or+NGxqyh6WRA8Mok0TdTmBiaWhRl5W4LratpY4egnomT/Fn++926U3X6qhlrnPxDRq3AKHZfIJuwWJwMNkHU/VsIh0qtivLtpD0lp8rXOu2l9uo7qllGssfH5X0E7E6uXMLrER9l13bytZN+s3ZWWtMWDk1fx09/cSH4rhA9FsSZPc+2b3kU9OZjleo7CMyE0DSUvQirPYpPnCH5FZdwyeDC6Er3fgejrwsoVw1tK3EeHKBoPEDruQqoX5iarZAycA3HU/hGMM0zyZQZO79knrCQDpoIeAy2WyY2JlxCoDbVMrswn9pEJPt3wUjZiZdzJPcOb2PPjFRTtjOW0ERPoSLFy60dmXXM5MpQFJrkp7wir3B18p+8m+mMBRqZmxw1QVQuvM82a/B6uCxwlw6m0CSmZIS5Nfji+nif6mnE/ECRyKDq/5+dfhSLka3ZufzhZxi/71lG4SxI4NI6Vw3V3IQhNY+q2pQyuV1jr7KJYzbrG/edELa9M1LD5UCPudgf+dkn4wCTq8ARmIpENCBcO0nejZMnSdt5bsI09y8sYvauZvOe6MLp75lsaAL8ZW80jLctojo0s+AXzs6KquIpjrAt3zpwpS0oTs81HpM3K+Z3QWZxvvvGqwIjWVAxH+xD5sRCBjukYHm6Vx9uu4aHI1aTzTdYsPcE1kTZ+P3wYj6JTSpqRlQIt0Yjvod05PQdQAwFkbTnH3xWkYn0PV7vb8CtnNyOmrBQPjtzI7pcaqeucRObIMSohBKkw+AOz58ZTMkVrsoS8/RJ392vTsQjdkY107cwuXsp4HJnO5FydyXR6JlWlNK1ZY7oaCiLCIWJLC5ms1IiVw5vX7ubO0D70V53OnrDS/HBiDfH2AJWt/VhXIN7OwjBChUD4fSTDKu+M7GKJYwymo1BaQNrSuILBm+YGRUUJ+BlvDOC6dYjdq++Z+ZWBySN9y2FzmMiLnRhd3dmoZLUh+q8V/OO6+3mHdxzwYGAyaSXRtvkpe3YSc3Q8pyaVQmY7JgCPksJ0azjcLqxYDCXgxyiN8JbGg3wm8jIDpsreliqW/qALs38wZ10iJut9DF4l+VHtA1zlPNXBmlKyL1XGoz1LqfvJCAwMY46Mzl6Fckx3alr2VbSiU6ciAOZQvZ0L4XAgiyJY4cz0jpNKr5XihZEG3IMi59w6jRMdcALOn6p5NpKzBJASAqFrIECZrt1RCw6mi9FioMTTOTHhFKpKvCHC0BplZgc0IyWPjK/kuYNNNH17W87sGJ0NZ/swvgfLZl3LeKG1PI++VQH6SoPs2tqAa0jB15etCylAKmA5YDIieH69i9VLO2edx4taBl2mk4d7ljF0sICGe3bPaW7Xi+XhoRUc315J446+bLte4AhNY2i1imvZGPW6EwuLCSvNPV3r6T5SSN2vMjjbuzHaO7Pv4/QinoiESFflcev6A/xlyeMUqW6eqjzCI2vWEd7nh+75VpZl90gZ7sMuSLzOtiXEzEKl0DXEabEGpJyO3CmtnJngl4QmWebO/tEtJEkp8XdA4PhUTi3sXG5kJp1d8Oju4aTDvC4EZW43SiSMlRdg/wfr6VoW5ndCB3ELB0HFhbN+ktHJIP5HtZwzaGYQAhEJMVnnZ9Ob9vGOvJ0sc5zdhMhIk34TtvVVUrzVQu0ZxsiVPlVVSYdNqgNZl/iTu6DDpsnhWDHhA5OI/hFQ1FPvm6Ig/D7we5He7AKDMhFDRqcwxyZyar4mDePMY7kQiEiYVFWEvms0fKtG+HL9i7zNe4R81c1Jl2QLC1NKhiyNX/eswNehXLFd4YVhhAI4HWS8gvXOUfynuVAdzuTz+ObVlB/L7cnTWZkOOmFevZTu1W7e9onNvDmwd+bXnUacfxu+gf4Xy6i9vxdzcAjF40E219C7SeWP7nyQa129GDjpMNL8y8BtPLFrBQ0vxxGtHchceTGkxJqMEjyeZMPm3+Ujy7bx+5EdvPD3B3jk4HKW/EEb/W+vIfPmcb4W2cLRTJjffeBTlG2RmL0DOZ2/L16gEKwaJaSkOP1cREoa/N3+NyP2+Al37MU6w8Qjc/1yeq93kiwxQJE4BjTcg4JAl4F/a0duJXQ+C0pBHl23R6ivPOWhMGE5OdBTQmRsMU85sqj1NcSW5LOiuZP35b+CLlQen2riWy2bKD2YhCPtOTFAKR4Pof/dyZ+WPTMTPGnKSvHK/1tP85aBnAtCdCbMnn7yHp29ei1UBZwOpM/Dfvcymkb6ppPEZxAeF9LnoX9ThKlqWLvpMG/J28dbvB2zxpEvd72Nnc8soeKZFJH2vgWRDy4lMwwnfDjGBaRfR85kRUUNBkDTEI7pqbJpYkWn5n9VX9fxrxvm47VbUBD8r/5reGj/Kqp/DM1tA1hDI5jTXhVqIIDIjzByTTED11t86JotfDS8lYjiYMBM8FJfDUWvgDI0nhOLQBeL4vViLavFcmtkPBq9mzQyBQa6P0Vm0ok+olH6goGnbQzz6Il572tkOk3XjjK+kbmNd678BSmZYcJSCR3PIA615cTRjDlFSqxEAtmfRoyOUfOQk6GufOKrTCDNkGkgtwcpfz4273E7zoVQVQZvLmNkjcX3ip6kWIWzLeVOWEmeT5TwRw99kIKd4H+uNRtMMgdQCwqQZQXcuvEAnyp4YWa3PiHTvPWV30Hd7afy6F4oK8ZYXkH7W3VEUQq/L4HflSLiipPnzAb57EsEOLq9iapHUjj2t2OOjJ7rq+cXIVA8Ho58roRA8wj/tOTnVOujFKsmYcWNgpg54qKg8J+T1fyscwO+/+Ml2N12xYK8LRgjVKoKUgG/4piJYpWQadrTVfg6FbRYBjUUnNlRwjCwEsmc3T0TmoYSDEA4iBXy0r/BzdTSNB8Nb6VIVQAne9IGm+PLeeDQKvLbJFZvf/a+wnz6NgRwNkxyq+cIQcXBhJXmZ+MbeOZYE+E9Ktpg1mVXKy+bDlEK1vgEMpmat0mGTKfRh6dw7yng0dBSrve18uG8l0k1axy5fhljyyw+WH0ArzDYni4mdFjg7Yrl7srgNJYOAWd6ZhdMQdBjxjmaCWIe9ZF/3MoGXZoefNXmBsygm3TQweBaB9rqcdblD+JQTFpLChgd9pModuBtz0OMjedsG55BVUkHIOg4ZRxMWi7koAtHdCFP/86DoqKGg8Qb8xhcq3FLqJNqfRwFF3uiFZgtfhyDI6fSEcw3qsqdBQe4w5Nd0Bkz47RmnHgGDRibQM3PQ7jdSLcTMTmFTCYxcyznssykMYfOHeV7ZnfM4UCWFTBV42eyDtTKGHfmHWC1q5uwcjJ9S/ad7IsH8PaCo2cCa3A46yJ/8juvcF7CiyEjTYZMg/4xP6FeiUxdWB+plRQjQ37i1SEMj4Lhzo4NSgZ83Um0gQnMYyeuZNHPiRCCEv8kdY6s98T2oUp8+524W9ox+gZQ3C7U4kKsoI94pZ9EvsbwGsmKpZ18LrKFAtWJKSW9ppPxCS91XYmcOo99wUynMlNqK8kU+xla6cZ0g+GG0KohlkQGqPGMcDxWwLHxfAanCvEVFhAs8KEPTGKd6Jw3rwYpJd5OQX9+mNSKDCYSE4GStnLayLqiTJ8zlYaBkjYRhsx6YgiYOREici+C+knUUBDyI4w3QX7d6HSU6rOnyeswdLbFaonsFwSP5lYuVKuykPFmP+8OHJ9ORXJauidTAbckccNSYsUasVJB7aouVoe7qXYNk6dOUaBNkqfEUYSk1wjytdRbGRgupnysCCWdmd9I6udAzYtAUT6exnF+q2ovb/JMoKAD+mtSsVhYxC0H0aSTwIneKzoPWDBG6JlozShsHmug6JUYKIL0mjpSoezKrms0jaNrLGcPFqv5eUytr2JotUaqKcFPrvtXVjpMnNMHvFPS4HcOfYjx/fks+bdOrLFxrFQKdWkjw+si/Nkf/oTVzt6ZfKkvJr388mc3UbErjf7UNvB5UYoL6XtTCZaebWBF26bQOgazu2vzMKmSqRTm4WOUn+hiZGANn77hE/zs5u/wN6VP8MA/nqDOMUCtNsGuVCkPD6ygcEs23cD87yGdH0VIVCFREKhC4Z7JVfz42FXU/XwMa3/rzN9b6A5afj/MhhXH+beqh3AKBZ1TqVzMquznLCxu6PoDSsYKMTq7c24SfD66Mnnk7xT42nLrXN3lRA0GiF1TT+e7LB685etUaRLPdPqI54420PRvx7FGx+e3kOfgyUQJ9w2tY6Jaw3TWATBRrRErl+TtK8TfnUJ9bveCa3uKx4PweJAleRz5WIDfu+Mx3uU/QL7imE7Y/doojppiYWmCZGUIp8eJ0jM0s6MkE0lkxsipxbBRK82T8Uacu3yEf3SBgaSEYPDNNYw1w2/dtoXrfEe40T0CwImMwmcOfoTUMyWUfH3+jFAAl5rBJbILJQMHC2n84VHMyUlUnxerroKe64NMbkjyB+ue4FrPUZp0C6fQUaaP6EzIBE9F16K1uxBbty7INCiKx4MSDtHyhTxqmvv4bt2vKFZTRBQNBeVU6q98MCslB5oFvUaYp8aX8fgza2n4x3Gsyal5abMynabk2SGElU/v7SZ+IdFRyPg1HKEg5vDInJcpZxCCobU+xlabuISCWzgoUXXSK+L0GR7Kd+ZmhNzUunoGNjj50tt/zd2+FvRzBCKysPjR6LX85vAKGu8/mDM7oCfpuiNI81uOcLv3GGHlVNwAp9D5yuonYTW85eNHcAmBLhRcQps5i3/KWMvaGcv0BBuX/Zj+JfBe6yuUPa/BzkPz7o1wJpJra+jf4OBrS3/Mmz3DcJrheaYgfw3OAarCY2RKC1Dgir23C9oILVXT3B45xN+8vxHptBBuA5c3hmkqGL0eQi0lFGz3onT1Z12N5nlHSW2sI1Ueou9aJ6l8C2f5FPUFw6wM9lCrJ2cMUABVCO4oO8z9yZWkawoxlpWQ8av03gDhmhFWO3vJn16tfy6p81KskXipSUehBnduQOoS6bQoqxygpydCcK8DJWlgTcXmd1IpJVYqRWTfJFrKzwf1z7C8qpe/rHyIAjWNRwi+3nYbAwcKaRxum0lDs1A4mVcpY2lYUiCMbGckdAfWhmaGl3vYsOII7yvcTvAMuQ0VwYxbxPgKCz1eTvie3D0PCyDdTlKVKep8wzNlV4WFkBLxqkhsiwZFRYQCDGzQqK/qolQ1cQkHfWaCv+x9M84jbmR0aiboVi6yzNEPBTv5f+8IMRrP9j1FgShrfWMcaC6hc8qF8Y6NOCYEzjFB4a4Eeu/0TlmOTuyF7mDogyuJlQqS1SluaT7AJs8RIsq5V+7fU7qTH9+l0X2dHyPlQsZrZgJN+Y9rePssgr/anVPvoSVFNijCOSY8isuFqK1kfEWE0WUC96oxri7o582BfVRpkzNRLcu1JB+q2ca/rbuZ+Ds3EtiWjUQ+10gp6Y8F6MxEwDXI8vUnOPiVeoSZ9TixClNUlfbwlvwTrHB1TefdVChSE5SoHiwkhzNe/vPZmynZJ3O2nZ4VIVDcblLXNTO02kHD0k5Whnt4dHIV+ydLOTJSQCLuxLJm71wsq+ij1jfM9YEjtK4v5MTvLaHqwQnEgaNzb4hKCUOjeIYjbE1UsdrVTbFq0vlWCNQ3UfxSFG1oEmt4FCsWz8kJ+5VA8XpRQkFG1xncsrIFp9BISYMpmUE57ibvQCb3DFBFRQ34GFjqxHHtCKtdHYTOEYjoSCbNzmQFv968nvABkT2ClGPvoFTApRqvCWeqILjKdQJFSApUJwrKa3YIz4RH0Skmw9TSNAMJP8X7daxk7rXptF8llWfhVxIzi1gW1mvmoCezOjTpg2zKO8qPb3kTBXs9qM++0Y1QKREy64akTU8mClUPb/MdhzseoNoxRLU2QbnmZNRM8e2xjfwwdDVqOkB+Mo1iWvOepiReH2Folc7vf/BBbvQcpV53ntbIZ68saah8IryFTK3K043XkCgUxCsMvnXHD7jVHZ9Z9QV4YWoJ+ydK8VdOckflYb5a+MqsZ33U+xaO72pExFO5sSolJXL3Qfx7VZTMeg6vr6G+zsQnsoGVBvYXUbBLYg6NLJgBypICU558sSUZqWKa0y+3UFB8XgbWeFHuHOaPyh5lpUPFItueT+8Isp1D9t+Nzd0cMcuJ3O/Iqcnvq7E8Dmorhmh296KK0/LA5dbYc1lRXE7M/ADOVWPcVniYoJLdAW03fLywdRlFR6yZvLc5g7SYMD1MWUl0odKoO1iij/Helfe99rOVp/75cNzFPUNXsVtdTt4hDXdPNudwLrqpCpeTqduneHfjHv6yYM/0ooiKhSAzHclSFWLWwGth8bFABx9b3oEp5czvTy6ofKz9Nra01BF+3IuZMRZMnyQ0DREMEF0Spu82k2/e9BPWOgeJTO8Inz7mBBUXnw8dZ3BpgHtuuBZvVwTmwQgFGJr0cTxVhOUf4F+qf0VXeQAAv5Kc7jctMtKkw5CMWG4OJctY7eqkRM3W5eFUBdW/zuA+PnzFzjFdCvIc81qh6SjhEEOrHVS95QQfLtlKWqr8R9tNDB8soGC3JH/YQE2c1gYVOPS2anqaAnxmxWb+rPphXims44HWWwmfcGOOz/1uqDk8gmu4iq3Reoq1CZr0OF+54VHuq1/DcKyMYJsDt5TZs8jJ1IJ5py4F4fNilkS4ZWULf1f2OE7hZsxK0GtohI6AZ3tbzp3LVxw6IhJmssnkX5c+yFI9iVOcPaTfwXQx9w2so/xpC9/2Dowc8h6ZQYCunLm9rXDor7lmYGJKSQYTS0oUIWZ2RxUEGioBRWVD4wm2Z2op1nXIlSM4p2G4BFbAwCEu7F2r1XVu9x7i+9dey1jCR/5z4oqM9wvDCJUSOTCMvzOfL3bfxnvyt3O7O3vOI6y4eJvvOE6h4BJONFQiqpPPhF/h1hsPsn9jBf/xo7dRuDuE86nd87rS1H2zxmfufJK7fK1EFMcZV1kMzJnGXa45+XzeizR+pR+/miCkxNngnEDBNeuez4RfIRpUiFdoFKhpdOGmz0ywP53P72/9AL7tbsof6sQaGp4rqReMlrRQ0qd2KBQUIkuHGU3nE7xXzZ3ASq+Tp/ubULYHEBPtaFXlnPhQGZHr+/l60z3U6yYpadFhSJ6MNfPM8BIqPWNUuYf5ndBBnNNREEPOBNKd22cq1UCAWJGbT5Q/zkZXO6Z0MWklGTL86HGJSOfWoHqpCN2B4nXT+9FlTCzL8IOV36NWmwLcPBwP8uP+q6l8zMR9fCTn3MjNySme/tx1/KrhNsbfFKexZJCrIyf4bHgnhar3rPdd7xpjSekjbP/Mfg4kyrnn0Dr0gx5KX07i2Jd7wRhefbTKwuL/ji5lf7SM3X1l1OWPcFW4nY+HduAVCo/Hyxg3PUQtF/ujZfi0NP9c8gIexYEKfK3812zPr+BP/u7d5G/RiPz3lnnRdaEITUO43fR8dgXReoNNq1v4TPgQ17qG8CvO16x8n45PTWH5TCxdvYA9gMuPTKdxP+PjhyPX8f47txNSYI0z6w2TkhYtGYs/OPZeureWkXdAYuqC4TuSvGPpXjYU72DITHE0UYT7cD9mDo53dcFhNlfkg+u1E3nF78dqrubQZ53ctHw/Xyx6ij9tfyctx8po+m6SyPgATE5BKgWvSvnU2JtPujzEXR/9Atc0tvH3Fb/m2zfejOFeSv7P987Lgpije4zn719L5x1h1tbdy92+Fm6qP8L2P6xif6yc1mgR7c+uJtAmyXvsGDIWy72Fu8uIUVdC/9Ve3hboIDy9aPmzyaV8q2UTZScSWDl2/h5ANtdx+A8dvGf5K1zrHMWjvNZIg2wfO2qm+GbbrSQeLKLkYF/OuV0L3YESCpIssLgx1IpXnDu145iVYMBU+HLbuznSWUzxozqKkT3Hm/jYGO+t3s2XIodmYtRUeUbZGyqbFbk6l4hsH8I1FuG/ll0PhS9yjev8Gxs1usW31/2Ez5ofwd+1Hs/Ojsue7WBhGKGATKZwTGbYPVjGhsAJcGcjcSoI8pTX7iKWqB5KVIuNzjb+pTnJSMpFxf5CrPGJK5Lr5kIw89O8O7CbItU9Y4BOWknGLYtDmXxGDB8nUoXUOAep1EdZ6kgQUTQ+Hjh9Rdr1mueqgENYTFgq+9P5PJaJsG+qgoPjxXh3u4kcng4bnmu7FopgqkQjVWAQtUx0xUAXKksiA2wuCCFUJZt/caEiQPo8GBEv6aVxbio+yioHmFKlz0zzk7Freaq3ieH2CKN1HsgDM3SqjjRhgZJbdTYLIaC8mFiRylJnD3mqxEIyZEn6U0EcEwYinnsrgpeC4nYhwiEmmg1WLelkvSO76GNh8cT4cvZ0VtB4YgyGcycQwwyWiXhpDwV91SQKSzk0WMWRogK6asMUOydnPqYgWeNpp1CNUqQmiKgqdbqPOn2MYU83+jKTexxrGcgEKB8rRs0YmJOT5/jiOcQ0SXd7ecS5FF2YqMIiI1XuPbaa+IgHd6fOgSIfR4oK6K0J4VbSvDxYQyqjkcpoxEc8CIdFSIuz0tPFWlc35aqDG91dbFrRytbRZRRWVWANDM1rChcVcCkZLEd2Icg8mX9PUVHLS0lX5DG1Msm62k5+u+g5KrQ4QcVNXKaJWyYdhpsCNUGlNnvs1BUDNIt5sUABTJPQ8QyG28HX19xOuWuMfD3rvRO3HByeKqF9fyllO0z8O3sx84MM3+FAEdl+8jdTTTzfV0/eRH/OeI9IKVFMyYCZosQ1ibcwhtRfO/USHjepAheblrWyMdBGTOq0tJfgb9Fh9+5z75JNTuIYL8B3sIGD4WIiVRru/DjxogCoZ3dDv6JEY0QOmxysKefbkXW8M7CbItXiQ/4++jxttId9fH71BxkOBnCN1+AaSqL1j2P2DeRM3V0Wpl2sJ8tcRJsyVDuGZrws9k2VYxz1o4+N5twuqBoKEi/1cFvTAW72t+BTzr4DmpQGrZkAvX1h6lpSyLGJHHQtFgiXE+m0KNYm0MWr3VAlRzJpRiw37el8TqQK6UxEOLK/gsBxhdBzxxCKgvS66XtHiFFj9sKtR02j62b2TNVcMR3xVrhdyNICpEPDcmroncPZtIan70QPjeIBXjpWR9rSGC/chjLtelupjRFUMpRqsxcpXULjKmeSpaX9tDXX4TnihcucsGHhGKGZNNpojOihfLYX1vCZYNf5byJrkD52479y7+q1PDByC5GDUdhx4AqX9szoLoM63Qecykv0eLyMx8eW8/KTywmcgIIXB/n1jTcystbk9zY9xY3ew6x+bRwN4KTbp8kPJ9awe7KC7Seq0NtcRA5Kwtv6cPUNUGr0IK3cPBsjnE4qP3qMr5Zu5sVEBSucvTTqKp8sfJHRRi+Wy5lNPbAAd0PfUbaXX9ygMjRQQDog+PO197LW1QmoDFtp7p1cw5P/eh2hIwnyDx+j5f+rozaYeyv350JoOt1vzie2JkGTbuARLiwsNsfreKGvnvxXDmPM04LPFaOkkGhzhM9d/yy/Fz44c87QlJInn15DyTYLq60zp4LYvBqjrZ2yr3eDqiKEoFNR6BShmd8LVeUXX7iJeH2aTc1HuCtvL+/1ZVfp81Uvf55/gD/O30t8Y4ZrPV+h5EU3juf25sSkw4rHafrrVnDobNNrZq5XJfqy6UeSKYRDRzgcdLhCoCiEU+PZ/tHKugeiCPb6Knj0HddT8lvt/FX1g6xxuPl2xVN88WaTF+OrqL7PCwcOz5tOl1BocPQTLzdIr61H33kUmUighIJ0vrucmre1cV/lgzTqYpbr7e6Ul+2JGr6z9wZurj/Cf5S/MG8azoQ0DPSndlO+zUfX89Wc8DdheLLvmJK2cA4nWDLUidE3gOVykmko4m+uuo8Nri4MnPzfX95N8dYMMjG/wZVmYZroMYt7Jldxje8Y65af4L98d83+jBDI4jwmqnS+Xv4o3x5bx2d3f4T675voBw5ekIFijoxS+ZM2Trhq6Vpt4XJkmPLLeduZMQcG8TwwQvPBGp6q28SLX6nj/aXbuct7gnzFQZnT4pUN/8PoujQvvrWCb7bdwtj2Mmp/rGMeOT4vZb4SKE4n1FfStwkev/P/Uq7qWKgMmwleaK+j5sEY9PTPdzFnITSNxNWNDKzT+W7RUxQognNl1u41JP/c9SYC+5yoz2/JyWBgQlVB18BpUqBGZxlbBiZRK81nD3+E3vZ88l9R8fUZuLqjLOk8hBWPYxoGakMtsSX5VBYNsMbTcU6vkiuOoqI4dKxltUzWeTE+NMqGomPcFGzhr37wIWp+IDH7B2bGZXNsDMbHafpqNUOVNfzRzU2gZM/IBleMsLawi2+UPj/LhtZQ0QT8btkz/Ow9cTr3NKJd5ujpC8YIBRDROKFWOLqmACrO/JlBM06GbOwqj6LiE06KVIXl7i7+uxmck148O+ay1KeQHR6+3LeWjFTpjofY110GvS5cQwol+zO4BhMwOIwjWoCSVFCEhS5OuWNaSL46tJp9E2W09BQjpUBaoLe5cUxA/rDEPWTg7olmD/3noF/6qyl2RylWJzmULKMjnQ/Ajd7DNPkHOFjTjNo7dNm3/+eCjZ5jUAn/c/dGvM40q51dFCgGGQkfa/0Q7ceLqBg1iZW5SDc3smxJO2+KHERnnlatLwZFMFVjsryib1a5TZSst1gOB+W5WIx8HxNVGiX6GE6R7T4PZtK8HK8jeBT8reOYOZzT9iTSMMAwznxsVwiKtqWY6nXw0uRSOpdGcNU8wSbXMLpQeD4ZolobY5nDQ91NJzhYXEHTSBNa/whG3/xPpqxodMbAPok0jOxinGUijQwilUJMp+6QpjVzjllOuzkqyRRavJKUqWHJ7ERDFypuNY3lkKflVZgfnEKjQouzYeVxtlNHjdaImrLoX+0mc1WU9xVvp1Qz0KejNU9YSQZMhd/b9xFiXX48vSp7gmVQfuqZKZnhpZF6IlsdaIOj8+dObplYsQRK9xBOlxPH9DktYZjIeAIrOgXSwlzTyMhyJ0udfYxbDvbHw/g7JJ7jo5g51PdI08Q1lOI7+zbx/qU7eWdwJ303BikIrUV9dlf2Q0IhVeQlFQFdKOwar8BoCaAPDWHFLizFjFBVpNeN5chGahdCzvmOtpqfh/D7kCNjWKlUdkdzeBSPZdH7QBV/W1zJ1yqT+HxJCv1TfLz8Zer0QVY5e3hf5U4edSxntLWcsEPHOtiak4vnrwfF40GUFdP1pghFDQMUqQq6UOk2Eny5450oh3xo3R2YZ8ghPl9oNVWkyyP03KDhWDJBRGFmrDsTxzIpfjO1granaig6lM7ZOpPpNHJiEmWsmK2JOmq1VjxKdofnP8ZquKdrHdEniinttvC3RVHGYzAxhcgLI2rKmKoPMNakklqW4Iule1jq7JsVaT1p6RiG8hpX+SuFGg5iNpTTeYcPsXKST1fvoMnVyxJ9GG3DGJ3pKip+mJk9f5YShkdxGSYlrqKst54QTHbn81xJHp+82c3b8vfyXl/2HouTGR8sHIpxzvPsF8uCMkKtiUny9kzSdkfgrJ/pNR3EpAOXyFBKCp8KPuGkQR/Gv3SUqa6800L6zC3BY3BfaD3CFLh6Veofn0Tt68QcHkGmUkjABNSURBgKHiWNVxgw3dBTMsNP927A0+Ki9pkoIpONQMrRI7NcjHP7FOFpKAoFjnH8SoaORD4Hx4sZmPBz27qDrPB2s61mA/5UBnLcCBVydgweBcFVTslVzqP83oaj01c1QGPKSjH4TBkVLQb6pMnIRg3PdcP8SeXDXOWUMG3MWVjT0S9z83wBZCc9edVj3F2451TKgGnkleit5hshSBQ6maq2KNROBfjalazk590biByKYx5snccCXiakRH9qJ/nhMO6RBnpSJfzcu5HK0sfwKxnuG17PjaHDLHMM85vGR3miXOf/vPRpwgqQA0boOQ1smJWz72xYSRMlA6alYCI4+YbrwkRqzEtOP0tmgyvpQkUXKiWqm3+ufJDDxWH+oOszqCkoemsX7y/dPj2JOHV0Y8gU7ExWoD8RpO5AEsOr0d5wahy1sEhKk5b+Iuof6Zr3+AEykz7n4qPQNAZXe5hYkaFWgxeTfh4cWUPoeBLzaFtuTYSlRO8bJ/BCKbtKKvj9vK1krp+kzxOg/Hk16yavCGJFOqn87Oh9dKSAvAMSRicu2KtCqApmvh/DDS4hkVIg5noyUJRPosSH2zBRolHMVCp7ZnxklKKjbSh+P+byWmLlAYZKwvzoHVfz1qIDfCLYyqeDh/lw4CBXr/0ywgoRPKzmhHfFpaAE/CQrwgRu7eeDldvxTQf16TACHHixnuI9RvaoVA6RqMtneJWT5dcf5c6CgzNB986EhcXeVBmP9C2n+peDMJh7sRBOIg0Dc2QU54jCKxM13O1rITz9ux+1X0Xm0QIqftWG0T8wMxcXqoq5tJzJSheDNxjcsuIgXy9/Eo9wzDJALSQxw4mRmTuTSgQDjCz30nzbUX5W9wjA9M6smz9tfoz78tcy9Uj+a+bP5vgEjE/gbO+cuebxeBDV5exSmohf5+Dd9b+eeZ6FxJRXbsd3QRmhF0KpmiZDGh3wnxZK2qtYrCrsZWteXjYAwDyEBi96rJOCrf5s6opECmtoBDOZQk7vnCguF6KilOFVKmuvaeU693FK1GxDfyml8MTkBgqedhLZO4Y40YOUEiwrGwZ7gaFVV5KsK6BEf4y2TIRHXlmNt1PF3yfZs6wSv5ognq/gCZy9A8wVvL0W/YcLGW10ocBMhNgz5V7yKDp/9LFf0JcJE7cc1DiHWOLoo0lPcdLdpSWT4eV4HQd/2UzDttiCSrZuSskjgysY7wpRKNvnuziXDa28jJ53VpG4for/s+ox1jpHSUiVA2mdfzp4O96HAng7O3IyGufFYk5M4t1ynIYjYYYfquJLgS+QjKgM3GBirRJ8NLAZgDw1xugy0BM+vDvnudCLEBmL4+gZQ3uklGV9X+A7t/83G5wT+BQn+YqDtc5x/vCj92GisM7VTqma5tXR1n81uZYfHLiavKhkqtyJ51O9fKV096zPWICRVjEHh5Dp3HUnV/x+lEgI5c5h/rz+BZxC5+/b3szUL0soauvCyCUDdBprYIjip1VaVpXzSEkNX1r2DP/luRZlaQOifwgrOsXoMgjUjHP/VBWp1iCVW7ovOEm8Vl1Jor4A63+PcGe4lUemmjCez6N6c3ROg/20vyOPils6if1LCd7DGryq/FYsjnbwBME2FyGXk8z2fH5efidfv/N2rms+xp+UPsrdN23jN+XLCT3kXNhGqKLS985axlaY/LjxXmq1OOBhT9rgZyPXUfvLSZTuwZwx2oSmofi8tN3m4H13buadwZ0UnaEvOUlKZohaBn+x6+3o+71U9u5fEHMVPQZHxwuIl55aSCz1TXKwPJ/BN9egx6pRDMlEjUqs2mTp8k5uCvZxs7+FWn0Uj3DNCirabsTZmSrj2fvXUbkjPWfvW6YkxOgNKT4Yea3reoE2Sb13iOeWNxCi6cIXx0U2JsRccl4jVAhRAfwQKCK7HPxdKeU3hRAR4B6gGmgH3iulvLLROCwLJW2QiXrYmTZZqpu4xewDkx5FnQmjfHpeOAVwqxmkyiw3raSMc5DtpEkCgjJqqBQNZGSa/WwlQRx3du/0kv0kje4e6D7LL4VACQWZWFlAqjrFLZHDFKkWqtAYNOM8NnEtDxxfSfnxBKKj93UFArkQjUCDECJ8xetwGivkI1as41FSDBp+fG0q/m4L91CaCdNDgTZJKiwwPPp5G+lc1uGZcA9l8HY6eSq6HJfYwzrnWQ7xkl1Z+mhgGBieSRkBMGZJ+swEMUvh8akV3Ne1mkhLGu3ACWKZyXnVd0YdHg9KJEzAlSSkZjvdjDSJywwHe0rwdKmvyx03F9vo6cigj4lVad5a28rd3nY8ioNRM8Xm+DKSXX4qDkaxJs+e/mi+2+hFYZnZCIfDI4jW7BKJt6KckZUVjKVO+ZPowsIImsSsSQ7L53O2Dl8XiopUQVdN1OlBuavH4Befe57hzufpnzQpk2VzUofSMJDRKcJHUwjLyQPr12GFd3OrO44uVIJC5aOBk7sps1OvnMSU2anFVJmC5YA/qHyeFY4+Ti58mVKyo1PQ97XvM5QcIpfbqBIMkCmLcF3JIW7xHGPCEnQPhanbH0NOnHtcnK9+xkokULp6cfcW89jIcj5S9DKr8nvYs3Y1/i4P+mgCWZmgIW+I7nQEfUJkA4ucx7VfDQQQeWGiK4sYr9f4vPMx/vl3evjBgECOHca0aik3y+esDpNFJh8t38LXy9+LYzyIaHv1H8LMzl2m5y+io4tQSTHxolq2OGp4PLCMjf7jjFZ5GaguR+0dzJ5lO/07cnysgOnxMRhgolFS39THeqeJRjYF3W8mV7O5s47q451nnMfNlz7hdEJpEbIiySfCWyhRHejizAYoQIch2ZmsRjnqIXTMuuCNnfkeC12jkv7uCENNbqq0DBoqdb4hDlSWMOpwoaYFIiMwaxOsr+rkt0ueo0mfoET1cHrfmpIGfWaaJ2NNPDSwishhE/fBXgzTnJM6lIpA0Sz0M6RcCSkJKp0jjDUpmI4IYdcylGgS8WrXfl1DOnSMPB/xUheZggxF7te2Sa+SIk+Pcdit4vR4LquhfSE7oQbwZSnlLiGEH9gphHgS+DjwtJTy74UQfwz8MfBHl61kZ8CKxxHH2il+bg3vMz7PD+78Ltc5Z/ub+ITzjOcg4lJwaKwYx0T2OSdfFoGggZUERBhDZtjG00RkEX20E6GQarGEdnmYUQaLr6Q2xekktraSd331Sa73trLKARpuOo043x69nofuv5ba/+7EHBx63flOL0TjU/LeKHNQhydJlHoZb1Qo1iZ4LtpM+a/7kU4HmYLs5LZYmyC+LMlUn5PQeZ4133Wob95P+W4v91k38YM1V9Ny23fOeWD9dOMTsi4tP55YwcGpUg6PFzL+QjHVP+3G6j+AmUzOu74zYS2vY2iVj98v/yV3eUdQUBiwEhxK51HxPxqO53e+rlXsXGyjp5Mo9/P8Hf+XfMWBPu1SdSgT5FvP30bxFpC7Ws6ZTigX6/CyI9ScrsMLRlFRw0HSAUGtfxi/kgYcoFo0/e715O+6joLHO9hy7PtzUofSMDCHR1CfH6dgm4ttYg2PrlvJ4bf9+wUHxvhfeXv4/U07yGzKGtRBxYFyWqCRYSvNfwzeTvENBTQc6czpNhpfXkrvdRq/G2zBqwjum2pA6XCh7NuLeZ6dmHnrZ6TESiYp3ZykdWwJP/mA5IbwET711Rf46eg1bB2o5j+b7sGvJLl/Yh2KwQVFiI1vaqLjLsFnrnueuwJ78Y1I/uUDv8X1/12MOd7Bltj9hPDPaR16lRTx26dIFPqp2HL+3IJGXz9F3xklf89SfrDhTj796Yf5ROFmPvaFz1K8OUzgp1tnfT7XxwoAa2U9A2u8/OGbfsPHAkfRcJCQaQZMg3t/dBPlL8Uwp84ctG++9InKUtrfmc+1tQdeEzX7TPxJxzs5+kwttT/txzrRecHp9OZ7LMx74CB5Wwu4b+16Inkv0qir/G3xK/xF0UtY023VRKILBZ3s0QflVYf4UtLgmGHxxaMfou/lMmruH8ff1oJxcnFlDurQ0TlMwW/KeLRkGZ8NHZv1u+UOQZN+nE2f/BdGLDeHU6V848CtiH1Fs86OJYtMnMVx3t+4nTWedq52DeERKq+2/5c7UlTkvcyvm6/HNVAHOw5dNk/S8xqhUso+oG/631EhRAtQBtwN3DT9sR8Az3GlX3gpkRkDb2+awFEXo7f7gAvbEYxbGp19EfLGZDbJ+jRO4cY5vbqhCR2P9JMiwRC9rONGAEqo4hgHwmd88CWiVZRjFocZWu1jbKnkem8r1Vp20vO57k3sHChn6lCE4r0G1sjoRblJXYhGYAR4B3PRaSsq0QoNfeU4/UaQQ5MliGgM6XRguFV0YTJuetA7nbhHzm/IzHcdykwaawoK9ibREk6uDn2E99Ts5qOhnUQUx6zzkq+eNPaZCdoNH/+242YcnQ6co4LCwxmsoRGsdCYn9L0aoWlM1HkZWWtSqY/OaNqfzueBkXXok+nXHWI/59roSYRAq6litEwnomg4hYaF5KFYmHsGN1C4RRA4Ej3vAJwTdSgEWlUFmBZG19lcMs6OVlFOormYynU93FlwKsJ43NJx92oEkx6cIlv8nKrD14FaVIgsitB7U4SpjQluDh4mophkpMlwRKM3tILSQ0m0yTQe5rgOLRMrkSR0LIWlOrmp6v28q2IP7wvspUg9d/7Pk+dIz0Rcpmk3fBwcXE6BoQGdOdHPnI1YkYZomqJYneBoxs3fb7uT/NZs4JHzGTzz3c84O0fJl2F25zezpbyOO1YexLBUlkX6KVKn8CsWV3mP85OlVxF790b0qJnNTQjEC3XihQqmOxvREiBel+a65qMU6RNsT1TzT8dvx9XjRwwcRUmac95Gfe0q/3DsTawr62JrugatqgJrZCwbLOwcyEwafWiKQIeT48kCVrk7qKobZOREKeFAIGuwTfex812H5+LkkareDT5SN0RZ52qf8db7abSW/26/hkirgd49giHPfGB3vvRZPheJujSN3nPH4IjLNMcyKod6iilqsWA8+roWnOd7LLTicZTRcR44tIrxBg9/V/Y4PqGfdQPrdEasBO2Gg7848U6O9hbi3e2mqNVA9A3POhY3F3UoJyYJtgZo7Sni8fIgN7pHps+qCkDBKRTKtTQRa4o85TidDXm8FKqdFa+j1DtBrXeY2/wHqNDihJXXLj4oCKKWydFMEC0GSjxzWePOvK4zoUKIamAN8ApQNG2gAvSTddc90z2fBT4L4LocIYEsE2f7MPlKAT2ZMBdqhI5YHpwnXHgHM2cdqBIyRpRxgkRIk8I57YrgyAZ4OOPf6lL1peoLGV7uovrdx/l0wQHWObLuVJNWkheeWEn+fknJSx3Z/KaXYQv8bBqBDFB1pnsuax0KgdA1Jmvga0sfpSOdz7GRfCrjXaDlkfGp6MKg3wgSOShxd0+9rgY/H3UI2UFUfXYXRfsiJNpr+f6Hr+X661rx6nE84swJngG6DA9PR5dR9qCG/8VjmENDwNmDS82XvhkUFeF2M96g8K6rtlGlTWJN76hsmWrg2eMNNEwlL6mTmvc2evpzVZWppYVEq8TMJD8jTX7Udw37DlTT+Mvdr9vgnq86FKpKvKkQNW2h9vTNRIQ99YHTjJhX/U6o6nRf5eCZpp+Rr2ZzpJnSYtzyEGizcPXHZxZZc6kOLxghsMoLGFsaYOOHd/Pm8H7e6pkA3ExYSV6OLyPd7kN9bitT8/UeWiaO/R0U94aZGMnne++6nrVXteMXUTyKflEpAyYsk0PJMvJf0Qgfzu7OzHs/czaEIFYq+EDTTorUBI/Emqn4pYb36NDrzrM4H23UaGtHaWun5lABmSXlPPnelZQ3DvK2sn34FYuI4uAW9yjvXb2T+12rMPvcqOnspNHbPMbnG19gnaudkJJdjPaIbOyLe6cq+XnvBmr+OoPob8UcHpmXOsw/kGbQUcQHPv0wJa4JdtavxS0E1tTU+YNFjU7gO65zLFpAMqjzycoX+avK30LkhVHSaazkaxf6Xm8dXuk2KnxeppblE782xuNXfYtSzcnJXaUfdlyN8ctCIrs7LzgY0Vy20UzAwYbGE6zxtJ/zc1HLZHO8GbXNTWhbzyXliJ6PNioNA2siSmizi2eiS2kvepZqLX3OCMAnacu4eGpqGYO/rKR+5xTseCXrYn6Oe65UHZrjE7BzAsfRa/lpzdWsrXgI33TU9pN5aD3CgUeFfBW+WrAXCvYCzIpXcup8q+esv+s1nbw41YRz3EKZmMI6ywLKxXDBRqgQwgf8CviSlHJyVvh7KaUQ4ow9jJTyu8B3AQIicllOvFqDw7gMk+8euZ5Uvc7vhY/OOig867NI/rh/A4+eWErZc0mcHSNnDB5iSIN9bKGJ1WhCn7VlLYTgbGd1L1XfWKOT+HVTfKbseVY5hgEP358s556e9ZS8bOA91J+Nnpu59AP659J4Us6Z7rusdSgUhKZhBE1u8XSTdHdxsKSEEb+PwfVBxm9JsNTVw/5kBY6ohUhc+M7vfNXh6VgTk7h3nqAo1MCne36H227dzU3BFu72njnaZI8RZv9EKc6xDNZ5glDkgj7F64G6CpLVaT6Z9xIFqkbUSnMg7eenezdQ9oCO6Os8/4POQk600WmU5UuILglifmqYT5TvQxcqO9MmL8aW0PmzWhr2xF63Z8J81qFwu0l+aYyNhR30JkIcGiwmNujFe0LD8EqW3NiGpphYUrC3oxwrkR0efAUxmgsGqHDvodo1MhMt0ZQWS57/JK49Hipf6kaOjmOeR+NJORerUegOlKAf4fMi3bPz1olUBtKZWbnRLhQ1HIb8MO1vCZJpjvPZguep0DKAi20pwaOTV/Pw9zZRfTA57++hNT6BSCQJTU6hZir54t7fJl5u4iqO8RcrHp7Ot3z2c+kn2ZJS+fX4Gu7dtQ7PcQfVz/UiR8dI5UA/cybUUJD06joSTSk+ENqePaZybAW1e3qwRl/fsar57mes8Qn0AwZN0RIyoTC/8d3KPUV3kCgUsH6CqsgYdzfto3rVMC5x6lyoicIfHHkf/aMBjEE37n6F4AkLLWGhR00cHUewEol5a6PufV2Ujxbw/26/jeZQP+mvjNHeWkLwSBmF26NogxMYHV2zDFLF5ULUVjJ4XR4jV2f4h9JHWOscJ+MY4zv1QwxvKiX/WRPrVd4bF1OHV7KNKn4/qVXVVP/RYb6Yt4ci1YGGypFMki8dfy9jzxdT/Vwv1vDIBT1vrtuoYlj0TAUZMXzA2fN7T1gqL47W4xoSWP2DFx3EbD77UWlkKH6ih0hrHh/v+yKuDSN8rmEz7/IdPeNu4OFMik8d+gijewso2m5RvK8fhkcxz+MBNRd1WPn4FMc7m/jGF27gbaE9bHIZZwyKeTLdyqt/d/L6yX+fTp8Z53tjG/nR3o0UPOkkf0sf5uDQZY0+fkFGqBBCJ2uA/kRKed/05QEhRImUsk8IUQLMWR6NkzuCsaOV/NK5hms9R6nS4hSqpyz5uEwzYZkMmQ4ea2/GPOzHeaIba3T8tc+TFvvYQjGVFIoyABw4SckETuEmJRPAZQ58qagoLieJQsHaim5WOIYpUJ2MWAmeGmnmxKESlrSPYvYOXJbE9+fTSDa16tzlQhHgESqFqos6zzCDhRVMVQg21WYjffWkwjiiGUTqwrTPSx2egZPnt4ItBQgryJOVSxiu9FJZ+gSlWoIi1cmElSZmSXpNDy9MLOFAbwm10dQ56zlX9AmXi1i5D38kSr2e7T5GLYPtiRr0bif+/f1YZznrcj5yqo0KQbLMx2iTyperX+JtvuMouOk3guyfKiOvJYmy58jMGZILYb7rUAhBgSfGLYFDXF04xHOFpbwcbeCxSDNhf5xf1j+CLlRMafGvebX0pLLeUDcFWnir5/QI3CqdxhT70/m49ngo3pLA7O7NrjBf4TpU3C5kcQGJMh+pkMbpkeO1pERNWni87pl+QyaSkEhipVKzA2UJBTU/Ag4dhCBTEiZe6iLdHGdT7XHqdZO4JTmYNrhv7Bqe6W6kaOsEonuAHfP8Hp5ML2PFYvj3uXCOhZmscjFVHuD74etZFupjqaf3vM95YayRbZ2VhHfqBNvSmJ3dmJl0TvQzZ0J4vYw3OCkqGqRc1dkyWINxwoc50Pq6xshc6GdkJo05loaxMVQh0DQdX2E+RkU+XWqQw41OPFqaQsckupptt8eThbROFtFzqAjXoEK4WxI8FkPZ0QLSQloSyzLntZ8xh0dQMhlajtdBHXy6+kV+pl/FEV8xiuHDO+DG53WDdVr+c4+D8SY/o8slty1vYbWzd8YQyHPH6QoI0Ga7kudCHc5CCJT8CFNlDv689FFKVRWncDAlU7RmCjl6oJyS4xZmV+95g03B/OgTGYvRqJcJ03vWz0xZKdqNCPt6SomMWBedi36+x0KkxGjvRI9OUazX0OOM8APH1QyX+inSX7sZsDdWweieAgp3WvifapnlHn425qoO1WM95McLeKy9mVSljj9/M6Vqmog6e5H2ZLoVyMYhOdN1yNZxVFq0ZQIcTFVz77HVePe5yN/cnV10eJ2eX+fjQqLjCuD7QIuU8uun/eoh4GPA30///4OXtWTnwYrHafybQ1j1FXzog5/npmsP8N2KF4DsH/HxeBk/7L2GlpZyqh+SeI70YXT1vqbhSCk5xA68+KkSjTPXCyiljw6qWUIfHQDjl7P8ajhIprkSuTLKP1Y8RInqptNI8O2RTRx4tInmn/Vi9vRdFgP0QjQCecBPL/nLzlsYC2kYaOMqj8cLuc0zwHJ3Nz97x01UXd/J35Y+yj8P3cCDh1fRsOMIxgW4IM9XHZ4L68BR/K0awX0VDNbW8sG3/i63X7WPb5Q+z7dGN7BlpIbjOyoJtUDd9jE40XXWwNi5pE8W5dF1p+Cu8lMH4Q+nw/zHtpsp2S8xj7df1CpZTrVRRUX1eRm4ysEtb9/JLZ5jhBU3qlDoN0IcHS/AN5V+XQNwLtShOTkJ71D5+rUfou7/tHB33i7+oXgLXy3ajIpAF9kdTlUofD50fGagyrp4zp4Efqz1QyR+UELli90zBuic1GFFCV13RSi5vYs/qXqMCm0Cx3QyxOOZMEfTxXy7dROJhA8AxyEPoaMWwZZxlKlT9SU1laOfLMQoTqPoFmurOvlK8QsscYzhFwqWhK8Pb+LeHespeVql6FgUufcI+zMv58R7eBLjRAdqZzfhrSoRVUV8y8kRJcARETz/zVJSn2nPJnE3DKxMZt7b6LkwS/PIe38X7y3dwYSVZuzpEqq2JS9oUn+SnOpnThUKmUlj9PZB3wBV+5wITSOuqjyrVs/6HFaaxvT+7D2mCaY5a9d/vvsZaRiYo2Ms/fshhq+vgD/dxt/W3kdVY4b47ZJ2w8dPhq/Bkko2DzYQccR4R2gnxWqcIlXDKU6lZeuf8hM8kUHGTwWcyrk6FALhcNB/Rymja0zKVX0mdsC90Rq+07aJ5n/sxBobx7qA+dx86dMmksiWCPsqyyH06rDGWcPlwVg13zmxibq/NxB9bReVXma+2+jpmCOjOJ6bpHaLE6FrvKiVg6h87Qctk7rkPmQ6g5ljdWiOjCLGJ6j6P7XsbVrNu29fxQeu2cpfFl5cvrRnEsX8cmg9+x9eQviISdVzx5BTxzFSqSuSf/lCdkKvAz4C7BdC7Jm+9qdkjc9fCCE+BXQA773spTsP5sQkWvcQRa/4eHliJY0NDQBIQ4EJHXe/QlGXhbttCDk8esaViwlG6KcTH0G2yicBqGc5VTSxn630yPaT4aL7XnPzJSB8Xibq3UT8A8QshW9Ha9k8Vs/ulxspOWAgh0cviwsuXJhGIEC2Tq8s08GlXKOCXw2tY0PFr2lwDFJ4dR93Fe/Ho6g8emIpjhY3Mn3287unM191eE4sE5kyYXAEj5QUbing+ZE1LK9pRut14hgXFB0z8XYnsnnizrG6lEv6hGWhpAQpK9t1HMsYPBdtJrjHga8zdtGdVC61Ua2ogJFbqjGWxrgtdJCQosy4qzw10szg7iKCY32va5k2V+rQHBvDc3SUlx9bybP5y9DyE/xi4/dY7cyump7ITNFhBFjvTOJ7VZLy706UsjNaxTPHmnDt8VB2cAI5Oj4zCZ6LOhTjUcKtIdrD5Xx58j2sKOxjqb+PDwR3UK2PU6xFGawNMGFkd1K2RyrpawgxuCGEkpk+siFACqhd30VdYBiHYrDa20mdPsaQ6WCXEeG/eq9n/6FKCl5RCbZOoPSPMGz000/HvNfhLKTM/v0NI7uIdQlxA3KljZ4JNS9CrMjN7QXbiVou/nt8Hf4OC2fn6GV/D5mrsfDVSAnSvKTYDzlRh1JiDY0QPBbkqy/cjTc/Tllwgt+veopidZK3R3ZjnubCEFCSLNWTeBQn2vRiV0KmeSXlZag/yJKu6CwjNNfqUI2EoTCP0Q0ZNja3zQQAs7D4cfdGRlvyyBvrwbrAHJrzpU/Ek3j6oCceJCUzOE+LZfFC0sELU0v40e6rcbc6CfcdQ0anLup7cqKNnsZJz5LLyZzW4ckxoG8IvxAU+iP83LiGFxrruabwBD41O7dscvVR6xikSTdwnXb+1cJid0rhcLqEH3VfzYnuApwnnBTtzuDuimKOjF4R4/MkFxId90XOHjPq1stbnNeJlBj9A/jvGcB/jo+da7UmJPK5jXef8XenRbHiKXnvZc0pbAW9jDXDKu8kXUaQf91/I9pBHw3fPoqMRjEv0s3hTFyIxqfkvUeklKOX7UvPhWXi7bHY0lJHb4mbq10qL6y4H1NaTEkQOwMUv5KaFcX4XMxXHV4I5tgYjI0RPHKcs+1NnK9QOaUvY+AYVRhK+shIk+3JKp7sWULpI30wOHLRibdzqY0aVYVk3j/KF+u2TAemOZlL0WLH8SrqH4pjDV3YuZ6T5FIdmkeOU/lXWbd3xe/nNy+tYrXzMADbU2U8OrqSipJH8Z3m6pqRJt84cCvKbj91/7ITmUohmd1256IOjZ5ePPf30nCglnRZiAPrmtnSWE/jzX2sdvbSrOv8ef6+UzcUvwIrZj/j5CTRlNaMG9KUlWLUgpfjDTw53EzXPbXU70+gbN6GRTZQWIg8bhO5UYdXglxqo7MQAllWSLRc5Z2B3fzXyHX8+uhy6g6OY7S1v65H5VI/cyXIlTq0olHUXa00D5WSKQ4SLS3ne797I+8v3sY7fYMzxuYpZi94jVoGPxi8Ds9xB9b+1lmT4Jyrw6J8okvC/OX19/MRfz8np8sZadKzs5TiHRZW8sJ3keZLn5yKEzqapnsiyLhlEFGUmb7yp0PX8MyepTT8MIXe1o0xcPFepLnSRq8k81GH5tgYjI8TblXJ21lHvDKP+28txPRn/4xlVSPcVHyUz0a24FJPmX4ZafLQxDoe62rG/aMQjcem4NABZDr9uo4bXSyvKzquzeVDGRyj9MUAnYca+GN3A5WHUziGRrDGJ16Xe9FCJbJnDD0R4q/r3sZvFe/iJs8x/qD93RzaVkPds1Oox/vOe+jbZu6RvQNUP6AztqOK2/K/hJ6QBMcyyN7DFx2gINeQqkK+J05IPbUj8et4gK88+GFKXpFoRzsueFU717FicZ78sxv4dehmAPS4hRa3+HBoGZZ2au1RSEl5Rwp9ZBgzB+pZ9g7gGJugvC9MuiTA3x74EJMNJu6y7Op8YWCKL1Q9ywpnHzWa66zPaclk+O2WDzHYWkDokMA9YuGImpS09cH45EUvqthcPoSm0/6uCMqKCUwEDx5eRekvdETf608zZDN3WOkMSu8A+tgE4RMuJsYr+LvmOg58fDPN7l6q9SFWOdIzKUxeSik8HV3Gi0N1dAxGiDzuprwlekV3YS4JRUXxeuh8ez5L72plk7sNVWSPAWxNmjwztYrCHRbB7b0YC2AuY41P4NrfRdE3ynhP6VeQp60TuEZMGkcSqMd7zprf1CYHmHbTl529eEfGqe/PR+rZ1WTDF2az9xqe9l03q26R4JywiEwZOI91I6emsmP8HL13thE6T1jRKbzHx/G2moiMgdndd0G+5ouG3kH8GYOWo2X82NDpLoyw/2AlFS+ZqEe7sy4ANjmHFYvBwVYcB+H0+JuXM2/UfCMFBBzJWVEp21KFFG+VBFrGMUfHL1ui5nnHMnH9ehuvNtOcZ/zw+Xft5worFoNYDIZH0Hv9lAyX4RoNEa3M+ht05fn5iXMjK4K9rHR3zbpXnT5DakqFbbFaRvYUUrxbEnz+ONboeDZozJwrsjkbQtfINMa5ubydXclKlE4X/p2dmJMX5w5oM0dY5qn3FHB091A6sJQHbljJjlAltf4RjvrbZhb7Hh1bwUvdtSRP+HH3KeS/1Acj4zn7LgpdQ4mEiNVk+P8qHqJcc5KRJlNWipfjy3moewWh9lg2FskCQGbSmAODKAODBM7ymVytC5vTkBIrGs3m5u0fmLmsTf+cfUl2fiLO2UboPGHFYnD4+GkX3livtzk+DhOTNP/vYdA0tmkVNCdbsRLJnNhpsXnjInWFGu8IBdqp/GcTpjt7fnd04g33ruY6VjQKh44SPqYTVrNLvEJVSTqd7FDL2SEqXnuTomQjdEpJffwgVio1p6u/NheIEAinkzsbWqhwjfKNb72b6h1xjJ5eu64WIPLAESo/EwBFpV0J0KGsPfU7y6LS6INMF9I0MRPJ1+YyziHU/DyGbyinuHKQGs2FgqDTiPNvwzfw68c3UvurScThE0h7vLCxOSu2ETqfvJE7p+kgDOZ5cmPa2Mw1aizD451LsKTACu3hqegy7ju2iprBSWTs4oOG2FxBLPOMyextFjZqfQ3x+jx6E+0cnSwgryWN3jeOYRugCxJpGIvGy0kaBo6YxXjSybCZoMt08nxsFfdv3UDhQYnSfe6AgzY2NrYRamNjYzMLbWgSXijj/tWr6asLsmPzEoJHwGhveWMvHNnYzDHj6wrpv17S11eC2e2h4YU92VQBNjbzjIwn8HZM0T3i4VAmyCPjq/jNseUs+fcxGBheNMa2jc2VxDZCbWxsbE7DGhym/HEX6V1euoMN1PZOoYxN2YGybGzmmPDWXnydYUyXjjYVw7rAtF02NlcamUigtPVS/4Nq/vT5z6LHLcrHDehux0pcvuwGNjaLGdsItbGxsTmNk8GXVMANr0lFYmNjMzcY7Z2I9k57omKTc0jDwBwbQ9k8Rmjzqev2WGFjc+EIOYerikKIISAGDM/Zl148+cwuZ5WUsuBcNyx2fQBCiCjQesVKdfl4tT6w63DR64PFr3Gx6wO7n8kx7DZ6BhaQRruNnoHFrg8Wv8bFrg/eAGPhXBqhAEKIHVLK9XP6pRfBxZbT1pcbXEo5F7vGxa7vUu+dS+w6vLz3zTV2G7389801i70OF7s+sNvolbp3LrHr8PLeN9dcbDmVK1EYGxsbGxsbGxsbGxsbG5szYRuhNjY2NjY2NjY2NjY2NnPGfBih352H77wYLractr7c4FLKudg1LnZ9l3rvXGLX4eW9b66x2+jlv2+uWex1uNj1gd1Gr9S9c4ldh5f3vrnmoso552dCbWxsbGxsbGxsbGxsbN642O64NjY2NjY2NjY2NjY2NnPGnBmhQog7hRCtQohjQog/nqvvPR9CiAohxLNCiENCiINCiC9OX/8rIUSPEGLP9M9bLuBZOadxseuDy6dxseubvmdRa7T1zQ92G7Xr8FXPyjmNi10f2G3UrsNZz1nU+qbvyTmNi10fXF6NSCmv+A+gAseBWsAB7AWWzsV3X0DZSoC10//2A0eApcBfAV9Z6BoXu77LpXGx63sjaLT1LWx9bwSNi11fLmtc7Poul8bFru+NoHGx68tljYtd3+XUKKWcs53Qq4BjUso2KWUa+Dlw9xx99zmRUvZJKXdN/zsKtABlF/GonNS42PXBZdO42PXB4tdo65sn7DZ6wSx2fZCjGhe7PrDb6OtgsWtc7PogRzUudn1wWTXOmRFaBnSd9t/dXGSBryRCiGpgDfDK9KUvCCH2CSH+SwgRPs/tOa9xseuDS9K42PXB4tdo68sB7DZq1yE5rnGx6wO7jZ7n9sWucbHrgwWgcbHrg0vWaAcmOokQwgf8CviSlHIS+BZQB6wG+oB/mb/SXTqLXR8sfo2LXR8sfo22voWtDxa/RlvfwtYHi1/jYtcHi1+jrW9h64PLo3GujNAeoOK0/y6fvpYTCCF0sn/In0gp7wOQUg5IKU0ppQV8j+zW+LnIWY2LXR9cFo2LXR8sfo22vnnEbqN2HU6TsxoXuz6w2yh2HcLi1wc5rHGx64PLpnHOjNDtQIMQokYI4QDeDzw0R999ToQQAvg+0CKl/Ppp10tO+9g7gQPneVROalzs+uCyaVzs+mDxa7T1zRN2G53BrsMc1bjY9YHdRqex63Dx64Mc1bjY9cFl1Tg30XFlNoLSW8hGUDoO/Nlcfe8FlOt6QAL7gD3TP28BfgTsn77+EFCyEDUudn2XU+Ni1/dG0GjrW9j63ggaF7u+XNW42PXZbdSuwzeSvlzVuNj1XW6NYvqBNjY2NjY2NjY2NjY2NjZXHDswkY2NjY2NjY2NjY2Njc2cYRuhNjY2NjY2NjY2NjY2NnOGbYTa2NjY2NjY2NjY2NjYzBm2EWpjY2NjY2NjY2NjY2MzZ9hGqI2NjY2NjY2NjY2Njc2cYRuhNjY2NjY2NjY2NjY2NnOGbYTa2NjY2NjY2NjY2NjYzBm2EWpjY2NjY2NjY2NjY2MzZ/z/uTI3oXBvhegAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import random\n", "import gzip\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "%matplotlib inline\n", "\n", "\n", "images_file = \"t10k-images-idx3-ubyte.gz\"\n", "\n", "\n", "def read_mnist(data_dir, images_file):\n", " \"\"\"Byte string to numpy arrays\"\"\"\n", " with gzip.open(os.path.join(data_dir, images_file), \"rb\") as f:\n", " images = np.frombuffer(f.read(), np.uint8, offset=16).reshape(-1, 28, 28)\n", " return images\n", "\n", "\n", "X = read_mnist(local_data_dir, images_file)\n", "\n", "# randomly sample 16 images to inspect\n", "mask = random.sample(range(X.shape[0]), 16)\n", "samples = X[mask]\n", "\n", "# plot the images\n", "fig, axs = plt.subplots(nrows=1, ncols=16, figsize=(16, 1))\n", "\n", "for i, splt in enumerate(axs):\n", " splt.imshow(samples[i])\n", "\n", "# preprocess the data to be consumed by the model\n", "\n", "\n", "def normalize(x, axis):\n", " eps = np.finfo(float).eps\n", "\n", " mean = np.mean(x, axis=axis, keepdims=True)\n", " # avoid division by zero\n", " std = np.std(x, axis=axis, keepdims=True) + eps\n", " return (x - mean) / std\n", "\n", "\n", "samples = normalize(samples, axis=(1, 2))\n", "samples = np.expand_dims(samples, axis=3)" ] }, { "cell_type": "code", "execution_count": 11, "id": "1269e7ef", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T01:00:10.025536Z", "iopub.status.busy": "2022-04-18T01:00:10.024926Z", "iopub.status.idle": "2022-04-18T01:00:10.252099Z", "shell.execute_reply": "2022-04-18T01:00:10.251364Z" }, "papermill": { "duration": 0.356148, "end_time": "2022-04-18T01:00:10.253856", "exception": false, "start_time": "2022-04-18T01:00:09.897708", "status": "completed" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Predictions: 6 4 8 6 1 7 3 3 6 9 6 5 7 1 6 3\n" ] } ], "source": [ "predictions = predictor.predict(samples)[\"predictions\"]\n", "\n", "# softmax to logit\n", "predictions = np.array(predictions, dtype=np.float32)\n", "predictions = np.argmax(predictions, axis=1)\n", "\n", "print(\"Predictions: \", *predictions)" ] }, { "cell_type": "markdown", "id": "57b6bbd9", "metadata": { "papermill": { "duration": 0.130717, "end_time": "2022-04-18T01:00:10.516390", "exception": false, "start_time": "2022-04-18T01:00:10.385673", "status": "completed" }, "tags": [] }, "source": [ "## Cleanup\n", "If you do not plan to continue using the endpoint, delete it to free up resources." ] }, { "cell_type": "code", "execution_count": 12, "id": "643673c7", "metadata": { "execution": { "iopub.execute_input": "2022-04-18T01:00:10.773684Z", "iopub.status.busy": "2022-04-18T01:00:10.773043Z", "iopub.status.idle": "2022-04-18T01:00:11.024500Z", "shell.execute_reply": "2022-04-18T01:00:11.023814Z" }, "papermill": { "duration": 0.383405, "end_time": "2022-04-18T01:00:11.026510", "exception": false, "start_time": "2022-04-18T01:00:10.643105", "status": "completed" }, "tags": [] }, "outputs": [], "source": [ "predictor.delete_endpoint()" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Notebook CI Test Results\n", "\n", "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n", "\n", "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-2/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ca-central-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/sa-east-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-2/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-3/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-central-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-north-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-2/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-2/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n", "\n", "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-south-1/hyperparameter_tuning|tensorflow2_mnist|hpo_tensorflow2_mnist_outputs.ipynb)\n" ] } ], "metadata": { "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (TensorFlow 2.3 Python 3.7 CPU Optimized)", "language": "python", "name": "conda_tensorflow_p36" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" }, "papermill": { "default_parameters": {}, "duration": 2242.184008, "end_time": "2022-04-18T01:00:11.675935", "environment_variables": {}, "exception": null, "input_path": "hpo_tensorflow2_mnist.ipynb", "output_path": "/opt/ml/processing/output/hpo_tensorflow2_mnist-2022-04-18-00-11-21.ipynb", "parameters": { "kms_key": "arn:aws:kms:us-west-2:000000000000:1234abcd-12ab-34cd-56ef-1234567890ab" }, "start_time": "2022-04-18T00:22:49.491927", "version": "2.3.4" } }, "nbformat": 4, "nbformat_minor": 5 }