{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Autopilot Blueprint and Trusted AI Guide\n", "\n", "This notebook guides you through the process of using a no-code ML pipeline (blueprint) to deliver an optimized ML model. The workflow uses:\n", "\n", " * [Amazon SageMaker DataWrangler](https://aws.amazon.com/sagemaker/data-wrangler/) for data prep. It provdies a graphical interface for creating data prep flows. \n", " * [Amazon SageMaker Autopilot](https://aws.amazon.com/sagemaker/autopilot/) for mode creation. It provides AutoML on tabular data through a fully managed experience.\n", " * [Amazon SageMaker Clarify](https://aws.amazon.com/sagemaker/clarify/) and [Inference](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-batch.html) capabilities are used to facilitate Trusted AI.\n", "\n", "#### Leaning Objectives\n", "\n", " * Learn how to use the AutoML blueprint.\n", " * Learn the facilities available to you to facilitated Trusted AI.\n", " * Deploy a real-time hosted model endpoint that is ready for integration into your analytics systems.\n", " \n", "#### The Data\n", "\n", "This notebook uses the [UCI Bank Marketing Dataset](https://archive.ics.uci.edu/ml/datasets/Bank+Marketing) as an example. This tabular dataset contains the results of a marketing campaign to acquire customers for term deposits. Each record represents information about a prospect. The last column is the target variable. It contains information of whether the prospect subscribed to the term deposit.

_Citation: Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014_\n", "\n", "#### Compatibility\n", "\n", "* __Amazon SageMaker DataWrangler:__ design and tested for \n", " * Flow definition schema: 1.0\n", " * DataWrangler Processing Container: 1.3.0\n", " \n", "\n", "* __Amazon SageMaker Clarify:__ designed and tested for\n", " * Bias processing container: 1.0\n", " * XAI processing container: 1.0\n", " \n", "* __Amazon SageMaker Studio:__ this notebook was designed and tested on\n", " * Python3 (Data Science) Kernel\n", " * Amazon SageMaker SDK version 2.x\n", "\n", "You can contact me regarding issues with the blueprint and notebook: Dylan Tong, AWS\n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequsites\n", "\n", "Follow this [guide](https://github.com/aws-samples/automl-blueprint/blob/main/automl_blueprint_quickstart_guide.pdf) to prepare your environment: https://github.com/aws-samples/automl-blueprint/blob/main/automl_blueprint_quickstart_guide.pdf\n", "\n", "The guide provides you step-by-step instructions for satisfying the prerequsites:\n", "\n", "1. This notebook requires access to Amazon S3, StepFunctions, and SageMaker. Attach the following managed policies to your Amazon SageMaker Studio execution role:\n", "\n", " * AmazonSageMakerFullAccess\n", " * AWSStepFunctionsFullAccess\n", " * AmazonS3FullAccess\n", " \n", " \n", "2. Deploy the [Autopilot Blueprint CloudFormation template]( https://dtong-public-fileshares3-us-west-2.amazonaws.com/automl-blueprint/code/cf/automl-blueprint.yml) before you continue further. You can deploy the template from the console by providing the following TemplateURL: https://dtong-public-fileshares3-us-west-2.amazonaws.com/automl-blueprint/code/cf/automl-blueprint.yml.

Use the __default settings__ or you will be required to modify the code in this notebook.\n", "\n", " ![Deploy Blueprint Video](img/cf-deploy-bp.gif)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "3. Optionally, [clone the Github repository](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-tasks-git.html) over to your Amazon SageMaker Studio instance.\n", "\n", " ![Setup Git Video](img/sm-setup-git.gif)\n", "\n", "---\n", "\n", " The blueprint consists of the following assets:\n", "\n", " * __code/deploy:__ the CloudFormation template to deploy the solution.\n", " * __code/workflow/implementations/autopilot:__ contains the Lambda function code executed as part of a StepFunction workflow. This workflow implemention orchestrates SageMaker DataWrangler, Autopilot, Clarify and other core SageMaker processes. \n", " * __code/workflow/layers:__ contains Amazon SageMaker dependencies required by the workflow. These dependencies are deployed as a Lambda Layer.\n", " * __config/blueprint-config.json:__ this file contains parameters you can configure to change the behavior of the blueprint. The blueprint looks for this configuration file in the S3 bucket that you configured as your workspace. The file should be stored under the prefix /config.\n", " * __meta/uci-bank-marketing-dataset.flow:__ this is a sample flow file produced by DataWrangler. You can create and save your own flow file in your workspace under the /meta prefix. You will need to reconfigure blueprint-config.json to use your flow file.\n", " * __notebook:__ this directory contains the assets for this notebook.\n", "\n", "___" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Install Dependencies\n", "Run the following to install dependencies that aren't included with the Python 3 (Data Science) Kernel by default." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting package metadata (current_repodata.json): done\n", "Solving environment: done\n", "\n", "## Package Plan ##\n", "\n", " environment location: /opt/conda\n", "\n", " added / updated specs:\n", " - shap\n", " - tqdm\n", "\n", "\n", "The following packages will be downloaded:\n", "\n", " package | build\n", " ---------------------------|-----------------\n", " conda-4.10.0 | py37h89c1867_1 3.1 MB conda-forge\n", " openssl-1.1.1k | h27cfd23_0 2.5 MB\n", " shap-0.37.0 | py37h10a2094_0 510 KB conda-forge\n", " slicer-0.0.7 | pyhd8ed1ab_0 16 KB conda-forge\n", " tqdm-4.60.0 | pyhd8ed1ab_0 79 KB conda-forge\n", " ------------------------------------------------------------\n", " Total: 6.2 MB\n", "\n", "The following NEW packages will be INSTALLED:\n", "\n", " shap conda-forge/linux-64::shap-0.37.0-py37h10a2094_0\n", " slicer conda-forge/noarch::slicer-0.0.7-pyhd8ed1ab_0\n", "\n", "The following packages will be UPDATED:\n", "\n", " conda 4.9.2-py37h89c1867_0 --> 4.10.0-py37h89c1867_1\n", " openssl 1.1.1j-h27cfd23_0 --> 1.1.1k-h27cfd23_0\n", " tqdm pkgs/main::tqdm-4.42.1-py_0 --> conda-forge::tqdm-4.60.0-pyhd8ed1ab_0\n", "\n", "\n", "\n", "Downloading and Extracting Packages\n", "conda-4.10.0 | 3.1 MB | ##################################### | 100% \n", "tqdm-4.60.0 | 79 KB | ##################################### | 100% \n", "openssl-1.1.1k | 2.5 MB | ##################################### | 100% \n", "slicer-0.0.7 | 16 KB | ##################################### | 100% \n", "shap-0.37.0 | 510 KB | ##################################### | 100% \n", "Preparing transaction: done\n", "Verifying transaction: done\n", "Executing transaction: done\n" ] } ], "source": [ "!conda install -y -c conda-forge tqdm shap" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Initialize Notebook\n", "\n", "Run the following to assign global variables and import the required libraries." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import json\n", "import os\n", "from time import strftime, gmtime\n", "\n", "import numpy as np\n", "import pandas as pd\n", "\n", "import boto3\n", "\n", "from utils.bpconfig import BPConfig\n", "from utils.trust import ModelInspector\n", "from utils.wf import SFNMonitor, BPRunner\n", "import utils.wf, utils.bpconfig, utils.prep\n", "\n", "sfn = boto3.client(\"stepfunctions\")\n", "sm = boto3.client(\"sagemaker\")\n", "s3 = boto3.client(\"s3\")\n", "\n", "account_id = boto3.client('sts').get_caller_identity().get('Account')\n", "region = boto3.session.Session().region_name\n", "workspace = f\"bp-workspace-{region}-{account_id}\"\n", "bpconfig = BPConfig.get_config(workspace, os.getcwd())\n", "local_dir = os.getcwd()\n", "\n", "bprunner = BPRunner(workspace, db_driver=s3, wf_driver=sfn)\n", "\n", "inspector_params = {\n", " \"workspace\": workspace,\n", " \"drivers\":{\n", " \"db\": s3,\n", " \"dsmlp\": sm\n", " },\n", " \"prefixes\": {\n", " \"results_path\": \"automl-blueprint/eval/error\",\n", " \"bias_path\": \"automl-blueprint/eval/bias\",\n", " \"xai_path\": \"automl-blueprint/eval/xai\"\n", " },\n", " \"results-config\":{\n", " \"gt_index\": 0,\n", " \"pred_index\": 1,\n", " }\n", "}\n", "\n", "## If you modified the Cloudformation default parameters, you will need to update wf_name accordingly.\n", "WF_NAME = \"bp-autopilot-blueprint\"\n", "WORKFLOW_STAGES = 9" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "### [Optional] View Pre-training Analysis Reports and Data Flow\n", "\n", "This notebook includes a sample flow file for prepping the UCI bank marketing dataset. It includes examples of analysis reports created with Amazon SageMaker DataWrangler for target leakage and data bias analysis.\n", "\n", "Run the following cell to download the sample flow.\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The flow definition file was copied to /root/automl-blueprint/notebook/uci-bank-marketing-dataset.flow\n" ] } ], "source": [ "fname = utils.prep.copy_sample_flow_to_local(workspace, local_dir)\n", "print(f\"The flow definition file was copied to {fname}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "Launch the DataWrangler GUI from the flow definition file and navigate to the analysis reports as shown below.\n", "\n", "![DataWrangler Analysis Reports](img/dw-analysis-reports.gif)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "The target leakage report should look like the following:\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Target Leakage Report](img/target-leakage-report.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "The data bias analysis report should look like the following:\n", "\n", "![Data Bias Report](img/data-bias-report.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "___\n", "\n", "### Run the AutoML Blueprint\n", "\n", "By now, the CloudFormation template should be in the COMPLETED status. The template will create a StepFunction workflow which will orchestrate the process of data preparation, model creation using AutoML and model evaluation.\n", "\n", "\n", "Run the following cell to find the ARN of the blueprint workflow that was created in your account." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Found the resource id of your workflow: arn:aws:states:eu-west-1:803235869972:stateMachine:bp-autopilot-blueprint\n" ] } ], "source": [ "wf_arn = bprunner.find_sfn_arn(WF_NAME)\n", "print(f\"Found the resource id of your workflow: {wf_arn}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "This blueprint is designed to provide a serverless and no-code experience. As you a user, you provide:\n", "\n", "1. Raw data\n", "2. An Amazon SageMaker DataWrangler flow created through the GUI\n", "3. Blueprint configurations\n", "\n", "Run the cell below to view the default configurations. You don't need to change any of the settings for this tutorial." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " The blueprint configurations were downloaded to /root/automl-blueprint/notebook/blueprint-config.json \n", "\n", "{\n", " \"automl-config\": {\n", " \"engine\": \"sagemaker-autopilot\",\n", " \"job_base_name\": \"automl-bp\",\n", " \"max_candidates\": \"1\",\n", " \"metric_name\": \"AUC\",\n", " \"minimum_performance\": 0.9,\n", " \"problem_type\": \"BinaryClassification\",\n", " \"target_name\": \"target\"\n", " },\n", " \"bias-analysis-config\": {\n", " \"bias-config\": {\n", " \"facet_name\": \"age\",\n", " \"facet_values_or_threshold\": [\n", " 30\n", " ],\n", " \"group_name\": \"job\",\n", " \"label_values_or_threshold\": [\n", " 1\n", " ]\n", " },\n", " \"engine\": \"sagemaker-clarify\",\n", " \"instance_count\": 1,\n", " \"instance_type\": \"ml.c5.xlarge\",\n", " \"job_base_name\": \"bp-clarify-bias\",\n", " \"output_prefix\": \"eval/bias\",\n", " \"prediction-config\": {\n", " \"label\": null,\n", " \"label_headers\": null,\n", " \"probability\": 0,\n", " \"probability_threshold\": 0.8\n", " }\n", " },\n", " \"data-config\": {\n", " \"prepped_out_prefix\": \"automl-blueprint/data/prepped\",\n", " \"raw_in_prefix\": \"automl-blueprint/sample-data/bank-marketing\"\n", " },\n", " \"dataprep-config\": {\n", " \"data_version\": 1,\n", " \"definition_file\": \"uci-bank-marketing-dataset.flow\",\n", " \"engine\": \"sagemaker-datawrangler\",\n", " \"instance_count\": 1,\n", " \"instance_type\": \"ml.m5.4xlarge\",\n", " \"output_node_id\": \"82971d23-e4f7-49cd-b4a9-f065d36e01ce.default\"\n", " },\n", " \"deployment-config\": {\n", " \"engine\": \"sagemaker-hosting\"\n", " },\n", " \"error-analysis-config\": {\n", " \"engine\": \"sagemaker-batch-transform\",\n", " \"job_base_name\": \"bp-error-analysis\",\n", " \"output_prefix\": \"eval/error\",\n", " \"test_data_uri\": null,\n", " \"transform-config\": {\n", " \"assemble_with\": \"Line\",\n", " \"input_filter\": \"$[:-2]\",\n", " \"instance_count\": 1,\n", " \"instance_type\": \"ml.m5.xlarge\",\n", " \"join_source\": \"Input\",\n", " \"output_filter\": \"$[-2,-1]\",\n", " \"split_type\": \"Line\",\n", " \"strategy\": \"SingleRecord\"\n", " }\n", " },\n", " \"model-config\": {\n", " \"engine\": \"sagemaker\",\n", " \"inference_response_keys\": [\n", " \"probability\"\n", " ],\n", " \"instance_count\": 1,\n", " \"instance_type\": \"ml.m5.xlarge\",\n", " \"model_base_name\": \"automl-bp-model\"\n", " },\n", " \"pipeline-config\": {\n", " \"engine\": \"aws-stepfunctions\"\n", " },\n", " \"security-config\": {},\n", " \"workspace-config\": {\n", " \"s3_prefix\": \"automl-blueprint\"\n", " },\n", " \"xai-config\": {\n", " \"engine\": \"sagemaker-clarify\",\n", " \"instance_count\": 1,\n", " \"instance_type\": \"ml.c5.xlarge\",\n", " \"job_base_name\": \"bp-clarify-shap\",\n", " \"output_prefix\": \"eval/xai\",\n", " \"shap-config\": {\n", " \"agg_method\": \"mean_abs\",\n", " \"num_samples\": 1\n", " }\n", " }\n", "}\n" ] } ], "source": [ "config = BPConfig.get_config(workspace, local_dir)\n", "print(f\"\\n The blueprint configurations were downloaded to {local_dir}/blueprint-config.json \\n\")\n", "config.print()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "___\n", "Run the following cell to execute the blueprint workflow." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "execution_arn = bprunner.run_blueprint(WF_NAME, WORKFLOW_STAGES, wait=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "You have the option to monitor the workflow progress from the StepFunctions console, or from this notebook. Run the next cell to obtain the link to the StepFunction workflow console.\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "
You can monitor the progress of your workflow from the StepFunctions console:

https://eu-west-1.console.aws.amazon.com/states/home?region=eu-west-1#/executions/details/arn:aws:states:eu-west-1:803235869972:execution:bp-autopilot-blueprint:0700a293-b709-4545-9e0d-8590292bae21

Alternatively, you can run the cell below to monitor the progress from your notebook." ], "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from IPython.display import Markdown as md\n", "\n", "sfn_console = f\"https://{region}.console.aws.amazon.com/states/home?region={region}#/executions/details/{execution_arn}\"\n", "\n", "md(f\"
\\\n", " You can monitor the progress of your workflow from the StepFunctions console:

\\\n", " {sfn_console}

\\\n", " Alternatively, you can run the cell below to monitor the progress from your notebook.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "Run the following cell if you like to render a progress bar to monitor the progress of the workflow.\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "445f0f1ea6304051a4adc4dc6844b49a", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Workflow Initiated: 0%| | 0/10 [00:00> Parallel Stage: Error Analysis: 0%| | 0/1 [00:00> Parallel Stage: Bias Analysis: 0%| | 0/1 [00:00> Parallel Stage: XAI Analysis: 0%| | 0/1 [00:00
\n", "\n", "\"error-analysis-config\":{
\n", "  ...
\n", "  \"test_data_uri\": _\"the s3 uri for your test dataset\"_
\n", "  ...
\n", "}\n", "

\n", "For convenience, _test_data_uri_ is set to null in this tutorial. The blueprint will by default use the data used by the automl job. Do not do this in practice.\n", "___\n", "\n", "Run the following command to visualize the ROC curve of the best model produced by your blueprint. This gives you a sense of the trade-off between true positive and false positive rates as well as an idea of the overall performance." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxV1bn/8c9zkkCCQEAGJ0DmVlREjajYOvy0FmnFaq2IU7lWUSu192pb66/3V63X3mtLe71QtZYqUocC1pHbi7Wt17lMQQEFRBAQAipDIIwJSc7z+2PvHE5ChhOSnUNyvu/XK6+cvfc6ez/7EPZz1lp7r2XujoiIZK5YugMQEZH0UiIQEclwSgQiIhlOiUBEJMMpEYiIZLjsdAfQWN27d/e+ffumOwwRkVZl4cKFW9y9R23bWl0i6Nu3L4WFhekOQ0SkVTGzT+rapqYhEZEMp0QgIpLhlAhERDKcEoGISIZTIhARyXCRJQIzm2pmm8zsgzq2m5lNNrNVZrbEzE6JKhYREalblDWCacDIerZfBAwKf8YDv40wFhERqUNkzxG4+5tm1reeIpcAT3gwDvZcM+tiZke5+6dRxSSHtnjcqXSnMu7Eq37HSawrr4xTGXfcwQl+x91xIBhN3Yk7B24PR1qvXn7/7/JKp7S8kpysWOK4VWXjVb/j+19Xxp0tu8ronJuTiD3Y2341R3f3erel/t6aBWpuO+C9NVY0WL6e99YTRvje6D6DA4+d9BnUKLuxZC89Oravfwet1PnHHcFJvbs0+37T+UDZMcD6pOWicN0BicDMxhPUGujTp0+LBJdp9u6rpHjPPjbtKGXj9lI+Kd4NQFl5nLVbd9M5N4eKuFMRXozL405ZeSV7yyupjDsVca92Ia+MOxWVzp7yCiorg/VxJ7yg7r/YVpUvq4in+ROQtsYs3RE0v56dc9tcIqjtn6nW7wHuPgWYAlBQUKCZdGqo+la7ZVcZO0sr2LyzjPLKOOuK9xCLGbvLKti2ex87SsvZuL2U3WUVfFpSSvucGBu376W0vP6LcFbMqIw7PTq1JztmZGcZ2bEYWTEjNydGXk4WMTPaZQfrsmJGlgW/22XHyM3JIsuMWAxi4fqYVf3s339ZRZyendoTS9pHLGZkhWXMjJ2lFfTo1B4j+I8eM0v8h696bVi4DaBqXdL2pDJmhgGl5ZV06dCOWLguiHH/e2JJ6yw8YG5OVrXPqeYfdM0LkSWVOHBbzTfX/d6a7z/wuDXKNiKumm9oKM6mHKuhC3VDcdb3GWRn6T6YxkhnIigCeict9wI2pimWQ5K7U7K3nM92lPLR57tYs3k3n+0o5a2Vm2mfHaO0PM6G7XtT2ld2zMjPyyEWM47snEvPzu3p2D6bU/p0Ze++Snp1zaNn51w65WbTt9thdO/YjqPy82ifHSMWa4NfrUQkIZ2JYBYwwcxmAKcDJZncPxCPO8s+3cF767dTuLaYwrXb2FVWQcne8gPK9u3Wgd1llYwY0I32OVlUVMY5tlsHOufl0O2w9uTmxOjQLptuHduRn5dDx/bZwbd2XdBFpBaRJQIzmw6cC3Q3syLgbiAHwN0fAWYDo4BVwB7gn6KK5VBUXhln3upiFhdtZ8HaYhasKWb3vkoA8vNy6Nv9ML54ZCdOObYrnXKz6XN4Bwb06MgxXfJ0QReRZhXlXUNjG9juwK1RHf9QVLx7H9P+sZY3Vmxi5aZd7Akv/AN7duSrJxzJaX0PZ3i/w+nf/bAD2l5FRKLS6oahbm1Kyyt5+YNPmfbOWhYXlSTWn//Fnlx80tGcemxXeh/eIY0RikimUyKIyIK1xbz8/mfMWLCOPfsqycvJ4mtDj+Jbp/binME99I1fRA4ZSgTNbNWmnfziLyv427LPMYOzB/XgytN6c94Xex5wu6GIyKFAiaCZ7Cqr4Of/s5zp89eRFTOuPr0PP77oi3RKevpURORQpETQRO7O4++sZfL/rmT7nnIuGXY0/3fUcRzROTfdoYmIpESJoAkqKuP86NklPP/eBob17sJd13yR0/t3S3dYIiKNokRwkNyd//fSBzz/3gbGjejLT78+RPf3i0irpERwkP7j5Q+ZPn89V53eh3tGH5/ucEREDppGZjoIzy0sYsqbq7n81F7cqyQgIq2cEkEjLV6/nbteeJ+T+3Th3y45QaMcikirp6tYIxTv3sfNTy2k+2Ht+P11BeS103MBItL6qY+gEf7r7x+xaWcZz9x0Jt3b6AxIIpJ5VCNIUdG2PcxcsJ5LhgXjA4mItBVKBCnYVxHnpicXkh0zJpw3MN3hiIg0KyWCFDz29hqWbtzBXaOOo3+PjukOR0SkWSkRNKCsopLfv7WaswZ245ozjk13OCIizU6JoAG/emUFxbv3ccOX+6c7FBGRSCgR1GN98R4ee3sN3zylF+d9oWe6wxERiYQSQT0eeeNjYmb84KuD0x2KiEhklAjqsLO0nOfeLeKSYcdwVH5eusMREYmMEkEdXluxmdLyOJef2ivdoYiIREqJoA7PLSyie8f2DO93eLpDERGJlBJBLZZuLOGNjzZzzRl9yNIcAyLSxikR1OLtlVsAGDu8T5ojERGJnhJBLeas3sqx3Tpo3mERyQhKBDXsq4gzb3Ux5wzuke5QRERahBJBDYuLtrO3vJIRA7qnOxQRkRahRFDDWx9tJmbobiERyRhKBDW8v6GEgT07cvhh7dIdiohIi1AiSFJRGWf+mmJO7q2JZ0QkcygRJHl33XZ276vkrEHqHxCRzBFpIjCzkWa2wsxWmdmPa9nex8xeM7P3zGyJmY2KMp6GLPxkGwAjBnRLZxgiIi0qskRgZlnAQ8BFwBBgrJkNqVHsX4Fn3P1k4Erg4ajiScU/Pt5C/+6HaWJ6EckoUdYIhgOr3H21u+8DZgCX1CjjQOfwdT6wMcJ46lVWUcmCtcWcNVDNQiKSWaJMBMcA65OWi8J1ye4BrjGzImA28L3admRm482s0MwKN2/eHEWsLCkqobQ8zlkD1SwkIpklykRQ22htXmN5LDDN3XsBo4AnzeyAmNx9irsXuHtBjx7RPPH792Wfkx0zztSDZCKSYaJMBEVA76TlXhzY9PMd4BkAd58D5AJpuRIvWr+d44/JJz8vJx2HFxFJmygTwQJgkJn1M7N2BJ3Bs2qUWQecD2BmxxEkgmjafuoRjzvLNu5g6DH5LX1oEZG0iywRuHsFMAF4BVhOcHfQUjO718xGh8XuAG40s8XAdGCcu9dsPorcR5t2srOsgmG9u7T0oUVE0i47yp27+2yCTuDkdT9Ner0MOCvKGFKxevNuAL5wZKc0RyIi0vL0ZDH7HyTr3+OwNEciItLylAiAVZt20aNTezq0i7SCJCJySFIiADbvLKN/d9UGRCQzZXwiqKiMs3rLLo47qnPDhUVE2qCMTwSfFO+htDzOECUCEclQGZ8IVny2E4DBumNIRDJUxieC1Zt3ATD4iI5pjkREJD0yPhGs2bKH7h11x5CIZK6MTwSrt+xiYE/dMSQimSulRGBm7cxsYNTBpMPqzbsZ0EPNQiKSuRpMBGb2NeB94G/h8jAzeyHqwFpCyZ5ySvaW07ebagQikrlSqRHcC5wObAdw90VAm6gdrNkajDHUp1uHNEciIpI+qSSCcnffXmNdi48QGoVPwkTQT08Vi0gGS+VWmeVmdgUQM7N+wPeBudGG1TKKtu0F4JgueWmOREQkfVKpEUwATgXiwPNAKUEyaPU+KymlS4ccDmuvW0dFJHOlcgX8qrvfCdxZtcLMLiNICq3a6i276NVVtQERyWyp1Aj+tZZ1P2nuQNJhwdptDNStoyKS4eqsEZjZV4GRwDFm9p9JmzoTNBO1evsq4m2j11tEpAnqaxraBHxA0CewNGn9TuDHUQbVErbv2QfAUflqGhKRzFZnInD394D3zOxpdy9twZhaxIbtwR1DffUMgYhkuFQ6i48xs58DQ4DcqpXuPjiyqFrApp1lAAzSqKMikuFS6SyeBjwOGHAR8AwwI8KYWsTmHUEi6Nkpt4GSIiJtWyqJoIO7vwLg7h+7+78C50UbVvQ27woSQfeO7dMciYhIeqXSNFRmZgZ8bGY3AxuAntGGFb31xXvo3rEdee2y0h2KiEhapZII/gXoCNwG/BzIB66PMqiW8NmOUo7MV7OQiEiDicDd54UvdwLXAphZryiDagmbdpQpEYiI0EAfgZmdZmbfMLPu4fLxZvYEbWDQueWf7aBLXk66wxARSbs6E4GZ/QfwNHA18Bcz+wnwGrAYaNW3jro72TFLdxgiIoeE+pqGLgFOcve9ZnY4sDFcXtEyoUVnR2kF5ZXOkKM7pzsUEZG0q69pqNTd9wK4ezHwYVtIAgDFu4PhJTT8tIhI/TWC/mZWNdS0AX2TlnH3yxrauZmNBCYBWcCj7n5/LWWuAO4hmPVssbtflXr4B2d3WQUAuTmpPEYhItK21ZcIvllj+cHG7NjMsoCHgK8ARcACM5vl7suSygwC7gLOcvdtZtYizyeU7C0H4MjOGnBORKS+QedebeK+hwOr3H01gJnNIOh3WJZU5kbgIXffFh5zUxOPmZKtYdNQj07tWuJwIiKHtCjbRo4B1ictF4Xrkg0GBpvZO2Y2N2xKOoCZjTezQjMr3Lx5c5MDKw6Hl+jSQYlARCTKRFDb/Zk154HJBgYB5wJjgUfNrMsBb3Kf4u4F7l7Qo0ePJgdWsjfoI9BzBCIijUgEZtbY0dmKgN5Jy70IbkGtWeYldy939zXACoLEEKni3WUc1i6L7Cx1FouINHglNLPhZvY+sDJcPsnMfpPCvhcAg8ysn5m1A64EZtUo8yLhSKbh08uDgdWNiP+gfPjZTt06KiISSuUr8WTg68BWAHdfTArDULt7BTABeAVYDjzj7kvN7F4zGx0WewXYambLCJ5a/qG7b238aTROXrssclQbEBEBUht9NObunwQjUSdUprJzd58NzK6x7qdJrx24PfxpMbvLKuhzuKaoFBGB1GoE681sOOBmlmVm/wx8FHFckdq+p5wuHdRRLCICqSWCWwi+sfcBPgfOCNe1WiV7y+mcq0QgIgKpNQ1VuPuVkUfSQtydbXv2cXhHPUMgIgKp1QgWmNlsM/u2mXWKPKKIVY082u0wJQIREUghEbj7AOA+4FTgfTN70cxabQ1hRzjOUGc9TCYiAqT4QJm7/8PdbwNOAXYQTFjTKm0Jh5foquElRESA1B4o62hmV5vZfwPzgc3AiMgji8jWXcGAcxqCWkQkkEpn8QfAfwO/dPe3Io4ncnvKg0cgjuysietFRCC1RNDf3eORR9JCquYiyFcfgYgIUE8iMLNfu/sdwHNmVnPU0JRmKDsUrd2yG4BOeo5ARASov0YwM/zdqJnJDnVVYwzltctKcyQiIoeG+mYomx++PM7dqyUDM5sANHUGs7Qo2btPzxCIiCRJ5daZ62tZ953mDqSl7Cyt0DMEIiJJ6usjGEMwh0A/M3s+aVMnYHvUgUVlzZbd5OWoWUhEpEp9fQTzCeYg6AU8lLR+J/BelEFFqVNuNkXb9qY7DBGRQ0Z9fQRrgDXA31sunOjt3VfJgB4d0x2GiMgho84+AjN7I/y9zcyKk362mVlxy4XYvHbvq6SD7hgSEUmor2moajrK7i0RSEvZXVZBR81XLCKSUGeNIOlp4t5AlrtXAmcCNwGHtUBskfi0pFQT14uIJEnl9tEXCaapHAA8ARwH/DHSqCISTJEc9BOIiEgglUQQd/dy4DLgv9z9e8Ax0YYVjT1hAujXo9VWaEREml0qiaDCzL4FXAv8OVzXKp/I2l1WAaCmIRGRJKk+WXwewTDUq82sHzA92rCiUVYRdHvkZmsuAhGRKg1+NXb3D8zsNmCgmX0RWOXuP48+tOZXGs5FkKsni0VEEhpMBGb2ZeBJYANgwJFmdq27vxN1cM1tb5gINMSEiMh+qTSWPwCMcvdlAGZ2HEFiKIgysCgkmoaUCEREElJpLG9XlQQA3H050CrHcS4rDxJBe81XLCKSkEqN4F0z+x1BLQDgalrpoHNVTUPt1VksIpKQSiK4GbgN+BFBH8GbwG+iDCoqe/YFt49qrCERkf3qTQRmdiIwAHjB3X/ZMiFFp6ppKK+dniMQEalS3+ij/5dgeImrgb+ZWW0zlbUqVU1Deo5ARGS/+q6IVwND3f1bwGnALY3duZmNNLMVZrbKzH5cT7nLzczNLNI7kfZVVHUWq2lIRKRKfYmgzN13A7j75gbKHsDMsghmNrsIGAKMNbMhtZTrRNAHMa8x+z8Y5fEgEWTHLOpDiYi0GvU1lvdPmqvYgAHJcxe7+2UN7Hs4wVPIqwHMbAZwCbCsRrl/A34J/KAxgR+M8opg9NGcLDUNiYhUqS8RfLPG8oON3PcxwPqk5SLg9OQCZnYy0Nvd/2xmdSYCMxsPjAfo06dPI8PYryIeJ2aQpRqBiEhCfXMWv9rEfdd2tfXERrMYwVPL4xrakbtPAaYAFBQUeAPF67SvMq7agIhIDVFeFYsIZjer0gvYmLTcCTgBeN3M1gJnALOi7DAuK4/rYTIRkRqivCouAAaZWT8zawdcCcyq2ujuJe7e3d37untfYC4w2t0LowqovDJOOyUCEZFqUr4qmln7xuzY3SuACcArwHLgGXdfamb3mtnoxoXZPMrVNCQicoBUhqEeDjwG5AN9zOwk4IZwysp6uftsYHaNdT+to+y5qQTcFOuL95KdpY5iEZFkqXw9ngx8HdgK4O6LCWYsa3Xy83LYumtfusMQETmkpJIIYu7+SY11lVEEE7WKeJy+3TRxvYhIslRGX1sfNg95+LTw94CPog0rGmUV6iwWEakplaviLcDtQB/gc4LbPBs97tChYJ8SgYjIAVKZvH4Twa2frV5pRZz8vJx0hyEickhJ5a6h35P0RHAVdx8fSUQR2lehB8pERGpKpY/g70mvc4FLqT6GUKtRXhmnnZ4jEBGpJpWmoZnJy2b2JPC3yCKKUPBAmZ4jEBFJdjBfj/sBxzZ3IC1hX4WeLBYRqSmVPoJt7O8jiAHFQJ2zjR3KdNeQiMiBGpq83oCTgA3hqri7H/Qw0OmmsYZERA5U71UxvOi/4O6V4U+rTQIAFXHXNJUiIjWk8vV4vpmdEnkkLaC8Mk6OmoZERKqps2nIzLLDoaS/BNxoZh8DuwlmHnN3b1XJwd0pr3RyVCMQEammvj6C+cApwDdaKJZIVcaDVq3iPRp9VEQkWX2JwADc/eMWiiVSlWH3xlH5eWmORETk0FJfIuhhZrfXtdHd/zOCeCJTVSPIUtOQiEg19SWCLKAjYc2gtUskAmsTpyMi0mzqSwSfuvu9LRZJxOLx4LdqBCIi1dV3L2WbumJW9REoEYiIVFdfIji/xaJoARVhlSCmRCAiUk2dicDdi1sykKhVNQ3pyWIRkeoy5jHbqhqBmoZERKrLmERQXhn0EWg+AhGR6jImEVTdPhrT7aMiItVkTCKI664hEZFaZUwiqAibhtRZLCJSXcYkgqoagZqGRESqy5hEoLGGRERqlzmJoKpGoEQgIlJNpInAzEaa2QozW2VmB0x4b2a3m9kyM1tiZq+a2bFRxbKrtAKA8op4VIcQEWmVIksEZpYFPARcBAwBxprZkBrF3gMK3H0o8Czwy6jiaR9OUZnXLiuqQ4iItEpR1giGA6vcfbW77wNmAJckF3D319x9T7g4F+gVVTBhFwHZsYxpDRMRSUmUV8VjgPVJy0Xhurp8B3i5tg1mNt7MCs2scPPmzQcVjCfuGjqot4uItFlRJoLaLrlea0Gza4ACYGJt2919irsXuHtBjx49DiqYqhqB6fZREZFq6puYpqmKgN5Jy72AjTULmdkFwE+Ac9y9LKpgHNUIRERqE2WNYAEwyMz6mVk74EpgVnIBMzsZ+B0w2t03RRiLagQiInWILBG4ewUwAXgFWA484+5LzexeMxsdFptIMC/yn8xskZnNqmN3TRZXH4GISK2ibBrC3WcDs2us+2nS6wuiPH6N4wIaYkJEpKaMuZeyaoYyJQIRkeoyJxGENQLlARGR6jImEVTdt6pEICJSXeYkAvURiIjUKmMSQdXto0oEIiLVZVAi0O2jIiK1yZhEULx7H6A+AhGRmjImEXTOzQH2NxGJiEggYxJB1VhDOVkZc8oiIinJmKuiV401lN4wREQOOZmXCJQJRESqyZhEENdzBCIitcqYRKA+YhGR2mVMIkBNQyIitcqYRFB115AmphERqS5zEoHuGhIRqVXmJILwtyoEIiLVZU4iSNQIlAlERJJlTiJAE9OIiNQmcxKB+ghERGqVOYmg6oUygYhINdnpDqDFVM1ZrEwgQHl5OUVFRZSWlqY7FJFmlZubS69evcjJyUn5PRmTCHTXkCQrKiqiU6dO9O3bV8+WSJvh7mzdupWioiL69euX8vsyp2lIfQSSpLS0lG7duikJSJtiZnTr1q3RNd0MSgR6sliq09+CtEUH83edOYkg/K3/+iIi1WVOItCgc9LKvfjiiyxbtuyg3z9r1izuv/9+AO655x5+9atf1Vt+2rRpbNy48aCP1xgdO3Y86DJ79+7lnHPOobKyMrHugQceIDc3l5KSksS6adOmMWHChGrvPffccyksLARg165d3HTTTQwYMIDjjz+es88+m3nz5h3M6SS4O7fddhsDBw5k6NChvPvuu7WWmzlzJkOHDuX444/nRz/6UWL9unXrOO+88zj55JMZOnQos2fPBuD9999n3LhxTYotWeYkgvC37hqS1qqpiWD06NH8+Mc/Trl8SyaCppg6dSqXXXYZWVlZiXXTp0/ntNNO44UXXkh5PzfccAOHH344K1euZOnSpUybNo0tW7Y0KbaXX36ZlStXsnLlSqZMmcItt9xyQJmtW7fywx/+kFdffZWlS5fy+eef8+qrrwJw3333ccUVV/Dee+8xY8YMvvvd7wJw4oknUlRUxLp165oUX5XMuWtIvcVSh5/991KWbdzRrPsccnRn7r74+HrLfOMb32D9+vWUlpby/e9/n/HjxwPBN99du3YB8Oyzz/LnP/+Z8ePHM2vWLN544w3uu+8+nnvuOXbu3MnNN9/Mnj17GDBgAFOnTqVr166ce+65DBs2jPnz57Njxw6mTp3K8OHDmTZtGoWFhTz44IPV4li0aNEB+3n11VcpLCzk6quvJi8vjzlz5pCXl5d4z+uvv87dd9/NEUccwaJFi7jssss48cQTmTRpEnv37uXFF19kwIABfPLJJ1x//fVs3ryZHj168Pjjj9OnTx/WrFnDVVddRUVFBSNHjqwWz8SJE3nmmWcoKyvj0ksv5Wc/+1m9n+PTTz/NH//4x8Tyxx9/zK5du5g4cSL//u//ntI3548//ph58+bx9NNPE4sF34/79+9P//79G3xvfV566SWuu+46zIwzzjiD7du38+mnn3LUUUclyqxevZrBgwfTo0cPAC644AKee+45zj//fMyMHTuCv82SkhKOPvroxPsuvvhiZsyYUa0GcbAypkZQRU1DcqiYOnUqCxcupLCwkMmTJ7N169Y6y44YMYLRo0czceJEFi1axIABA7juuuv4xS9+wZIlSzjxxBOrXTB3797NP/7xDx5++GGuv/76euOobT+XX345BQUFPP300yxatKhaEqiyePFiJk2axPvvv8+TTz7JRx99xPz587nhhhv4zW9+A8CECRO47rrrWLJkCVdffTW33XYbAN///ve55ZZbWLBgAUceeWRin3/9619ZuXIl8+fPZ9GiRSxcuJA333yzztj37dvH6tWr6du3b2Ld9OnTGTt2LF/+8pdZsWIFmzZtqvf8AZYuXcqwYcOq1SrqMmbMGIYNG3bAzxNPPHFA2Q0bNtC7d+/Ecq9evdiwYUO1MgMHDuTDDz9k7dq1VFRU8OKLL7J+/XogaMJ76qmn6NWrF6NGjUp8rgAFBQW89dZbDcabigyqEQS/lQekpoa+uUdl8uTJiaaL9evXs3LlSrp165bSe0tKSti+fTvnnHMOAN/+9rf51re+ldg+duxYAM4++2x27NjB9u3bD2o/9TnttNMS32wHDBjAhRdeCATNFq+99hoAc+bM4fnnnwfg2muvTXx7feedd3juuecS6++8804gSAR//etfOfnkk4Gg3X7lypWcffbZtcawZcsWunTpUm3djBkzeOGFF4jFYlx22WX86U9/4tZbb63zbprG3mUzc+bMlMsmWiLqOV7Xrl357W9/y5gxY4jFYowYMYLVq1cDQVIbN24cd9xxB3PmzOHaa6/lgw8+IBaL0bNnz2Zruos0EZjZSGASkAU86u7319jeHngCOBXYCoxx97VRxKKJaeRQ8vrrr/P3v/+dOXPm0KFDB84999zEvd/Jf6MH++Rzzb/zpv7dz5s3j5tuugmAe++9l86dO9O+ffvE9lgslliOxWJUVFQ0GFdtMbk7d911V+JYDcnLy6v2GS1ZsoSVK1fyla98BQhqDP379+fWW2+lW7dubNu2rdr7i4uL6d69O126dGHx4sXE4/FE01BdxowZw4oVKw5Yf/vtt3PddddVW9erV6/Et3sIHmRMbt6pcvHFF3PxxRcDMGXKlETN5LHHHuMvf/kLAGeeeSalpaVs2bKFnj17UlpaWmtN7WBE1jRkZlnAQ8BFwBBgrJkNqVHsO8A2dx8IPAD8Iqp4VCOQQ0lJSQldu3alQ4cOfPjhh8ydOzex7YgjjmD58uXE4/FqnZ2dOnVi586dAOTn59O1a9dE08CTTz6Z+FYP+7+1vv322+Tn55Ofn19rHPXtJ/l4p59+OosWLWLRokWMHj065fMcMWIEM2bMAIK2/C996UsAnHXWWdXWV/nqV7/K1KlTE30kGzZsqLdpp2vXrlRWViaSwfTp07nnnntYu3Yta9euZePGjWzYsIFPPvmE0047jXfeeYfPPvsMgMLCQsrKyujduzcDBgygoKCAu+++O/EtfuXKlbz00ksHHHPmzJmJzyL5p2YSgKCD/oknnsDdmTt3Lvn5+dX6B6pUneO2bdt4+OGHueGGGwDo06dPouN4+fLllJaWJvoSPvroI0444YQ6P5vGiLJGMBxY5e6rAcxsBnAJkHzbwyXAPeHrZ4EHzcy8tvpUE2mICTmUjBw5khu+mW0AAApQSURBVEceeYShQ4fyhS98gTPOOCOx7f777+frX/86vXv35oQTTkhcFK+88kpuvPFGJk+ezLPPPssf/vCHRCdv//79efzxxxP76Nq1KyNGjEh0Ftenrv2MGzeOm2++udbO4lRNnjyZ66+/nokTJyY6iwEmTZrEVVddxaRJk/jmN7+ZKH/hhReyfPlyzjzzTCDoOH/qqafo2bNnnce48MILefvtt7nggguYMWMGL7/8crXtl156KTNmzODOO+9k0qRJjBo1ing8TseOHZk+fXqiBvDoo49yxx13MHDgQDp06EC3bt2YOHFio8852ahRo5g9e3Zin8n/RsOGDWPRokVA0GeyePFiAH76058yePBgAH79619z44038sADD2BmTJs2LVGTeu211/ja177WpPgS3D2SH+ByguagquVrgQdrlPkA6JW0/DHQvZZ9jQcKgcI+ffr4wXjlg0/9u08t9NLyioN6v7Qty5YtS3cIkTnnnHN8wYIF6Q6jxbz77rt+zTXXpDuMFlVaWuqnn366l5eX17q9tr9voNDruF5HWSOo7bt3zW/6qZTB3acAUwAKCgoOqrZw4fFHcuHxRzZcUERalZNPPpnzzjuPysrKlO76aQvWrVvH/fffT3Z281zCo0wERUDvpOVeQM0u7qoyRWaWDeQDxRHGJNLmvf766+kOocU1dItsWzNo0CAGDRrUbPuL8jmCBcAgM+tnZu2AK4FZNcrMAr4dvr4c+N+wCiMSOf2pSVt0MH/XkSUCd68AJgCvAMuBZ9x9qZnda2ZVtx08BnQzs1XA7UDqz7+LNEFubi5bt25VMpA2xcP5CHJzcxv1Pmtt/xEKCgq8apAokYOlGcqkraprhjIzW+juBbW9J2OeLBZJlpOT06gZnETasowba0hERKpTIhARyXBKBCIiGa7VdRab2Wbgk4N8e3egaTNNtD4658ygc84MTTnnY929R20bWl0iaAozK6yr17yt0jlnBp1zZojqnNU0JCKS4ZQIREQyXKYlginpDiANdM6ZQeecGSI554zqIxARkQNlWo1ARERqUCIQEclwbTIRmNlIM1thZqvM7IARTc2svZnNDLfPM7O+LR9l80rhnG83s2VmtsTMXjWzY9MRZ3Nq6JyTyl1uZm5mrf5Ww1TO2cyuCP+tl5rZH1s6xuaWwt92HzN7zczeC/++R6UjzuZiZlPNbJOZfVDHdjOzyeHnscTMTmnyQeuauqy1/gBZBFNe9gfaAYuBITXKfBd4JHx9JTAz3XG3wDmfB3QIX9+SCecclusEvAnMBQrSHXcL/DsPAt4DuobLPdMddwuc8xTglvD1EGBtuuNu4jmfDZwCfFDH9lHAywQzPJ4BzGvqMdtijWA4sMrdV7v7PmAGcEmNMpcAfwhfPwucb9aqp7Vv8Jzd/TV33xMuziWYMa41S+XfGeDfgF8CbWG86VTO+UbgIXffBuDum1o4xuaWyjk70Dl8nc+BMyG2Ku7+JvXP1HgJ8IQH5gJdzOyophyzLSaCY4D1SctF4bpay3gwgU4J0K1FootGKuec7DsE3yhaswbP2cxOBnq7+59bMrAIpfLvPBgYbGbvmNlcMxvZYtFFI5Vzvge4xsyKgNnA91omtLRp7P/3BrXF+Qhq+2Zf8x7ZVMq0Jimfj5ldAxQA50QaUfTqPWcziwEPAONaKqAWkMq/czZB89C5BLW+t8zsBHffHnFsUUnlnMcC09z912Z2JvBkeM7x6MNLi2a/frXFGkER0DtpuRcHVhUTZcwsm6A6WV9V7FCXyjljZhcAPwFGu3tZC8UWlYbOuRNwAvC6ma0laEud1co7jFP9237J3cvdfQ2wgiAxtFapnPN3gGcA3H0OkEswOFtbldL/98Zoi4lgATDIzPqZWTuCzuBZNcrMAr4dvr4c+F8Pe2FaqQbPOWwm+R1BEmjt7cbQwDm7e4m7d3f3vu7el6BfZLS7t+Z5TlP5236R4MYAzKw7QVPR6haNsnmlcs7rgPMBzOw4gkSwuUWjbFmzgOvCu4fOAErc/dOm7LDNNQ25e4WZTQBeIbjjYKq7LzWze4FCd58FPEZQfVxFUBO4Mn0RN12K5zwR6Aj8KewXX+fuo9MWdBOleM5tSorn/ApwoZktAyqBH7r71vRF3TQpnvMdwO/N7F8ImkjGteYvdmY2naBpr3vY73E3kAPg7o8Q9IOMAlYBe4B/avIxW/HnJSIizaAtNg2JiEgjKBGIiGQ4JQIRkQynRCAikuGUCEREMpwSgRxyzKzSzBYl/fStp2zfukZpbOQxXw9HuFwcDs/whYPYx81mdl34epyZHZ207VEzG9LMcS4ws2EpvOefzaxDU48tbZcSgRyK9rr7sKSftS103Kvd/SSCAQknNvbN7v6Iuz8RLo4Djk7adoO7L2uWKPfH+TCpxfnPgBKB1EmJQFqF8Jv/W2b2bvgzopYyx5vZ/LAWscTMBoXrr0la/zszy2rgcG8CA8P3nh+Oc/9+OE58+3D9/bZ/fodfhevuMbMfmNnlBOM5PR0eMy/8Jl9gZreY2S+TYh5nZr85yDjnkDTYmJn91swKLZiH4GfhutsIEtJrZvZauO5CM5sTfo5/MrOODRxH2jglAjkU5SU1C70QrtsEfMXdTwHGAJNred/NwCR3H0ZwIS4KhxwYA5wVrq8Erm7g+BcD75tZLjANGOPuJxI8iX+LmR0OXAoc7+5DgfuS3+zuzwKFBN/ch7n73qTNzwKXJS2PAWYeZJwjCYaUqPITdy8AhgLnmNlQd59MMA7Nee5+XjjsxL8CF4SfZSFwewPHkTauzQ0xIW3C3vBimCwHeDBsE68kGEOnpjnAT8ysF/C8u680s/OBU4EF4dAaeQRJpTZPm9leYC3BUMZfANa4+0fh9j8AtwIPEsxv8KiZ/Q+Q8jDX7r7ZzFaHY8SsDI/xTrjfxsR5GMGQC8mzU11hZuMJ/l8fRTBJy5Ia7z0jXP9OeJx2BJ+bZDAlAmkt/gX4HDiJoCZ7wEQz7v5HM5sHfA14xcxuIBiy9w/uflcKx7g6eVA6M6t1jopw/JvhBAOdXQlMAP5PI85lJnAF8CHwgru7BVfllOMkmKnrfuAh4DIz6wf8ADjN3beZ2TSCwddqMuBv7j62EfFKG6emIWkt8oFPwzHmryX4NlyNmfUHVofNIbMImkheBS43s55hmcMt9fmaPwT6mtnAcPla4I2wTT3f3WcTdMTWdufOToKhsGvzPPANgnH0Z4brGhWnu5cTNPGcETYrdQZ2AyVmdgRwUR2xzAXOqjonM+tgZrXVriSDKBFIa/Ew8G0zm0vQLLS7ljJjgA/MbBHwRYLp/JYRXDD/amZLgL8RNJs0yN1LCUZ2/JOZvQ/EgUcILqp/Dvf3BkFtpaZpwCNVncU19rsNWAYc6+7zw3WNjjPse/g18AN3X0wwV/FSYCpBc1OVKcDLZvaau28muKNpenicuQSflWQwjT4qIpLhVCMQEclwSgQiIhlOiUBEJMMpEYiIZDglAhGRDKdEICKS4ZQIREQy3P8H1aji0qwyg5MAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "_, _, fpr, tpr, thresholds = inspector.get_roc_curve()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "Run the following command to visualize the AUC curve to determine an appropriate threshold to you use for your model.\n" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABJcAAADUCAYAAAAhisI5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeXxW9Z33/9f3nOtcWzayAmEn7IsBB0FErd5Wq9aO3W2n1drldlrr3Wlnasf7Yaft3dHHtNZfax1pHWvdbdVOXajYWrC0owguCCqCuABiRJawhEByref7+yM5l1dCgIC5uEh4Px+PPJLrOud8zydhsbz7+X6OsdYiIiIiIiIiIiJyJJxiFyAiIiIiIiIiIv2XwiURERERERERETliCpdEREREREREROSIKVwSEREREREREZEjpnBJRERERERERESOmMIlERERERERERE5YgULl4wxtxljthljVh/guDHG3GiMecMY85Ix5sRC1SIiIiIiIiIiIoVRyM6lO4BzD3L8PGB858dlwC8LWIuIiIiIiIiIiBRAwcIla+3/ADsPcsqFwF22w3JgkDFmaKHqERERERERERGRvlfMmUvDgLfzXjd1viciIiIiIiIiIv1EqIj3Nj28Z3s80ZjL6Ng6h+e6fzd4aEeDk7UWay3GmP1eZ7NZHMfB2h6X3P/GeecF6+W/n/9e9/eDe+bX0sP3cND3jTG5j+7nG2NwHGe/83s672D3y1//QLVZa3P36mmd3t7vUK8P9f6hHOr7EBEREREREZHeW7FiRbO1tvZIri1muNQEjMh7PRzY3NOJ1tpbgFsA6qtq7K9//euCFOT7fv49cwFSfvDk+/5+5/V0DUAqlcJxHHzf73IsuN73fTKZDKlUilQq1SUYA8hms7S3txMOh7ucH6wVrBuEaMYYfN/Pve5edxCABWsHYVN+PQcKoA6me8AHdHkdhGMHWjc4ln/OoWoIjudf43le7vsPznFdd7/rgvODY90DvvxwrXuNwdfBOfnrdw8Bg2tc18VxnNzn/HO6r9nT61Ao1ON7+YFk/rpBXfn1BMe7v989IDzYsZ5+jof66OnX9FCfRURERERE5Ogzxrx1pNcWM1xaAFxhjLkPmAO0WGvfLWI9PYYKx5MgkOoeTB0oQDvQed3fz2azPa4bOFD41r22A11jrSWTyeSCuSCkCM7p6XvMr6t77T3ds7fBY/7x3vysDvR1/lrB5/wg8HD0prOup1oCB7qme3DXPUgyxuB53n5rBXUExw7U5RYEYcF9PM/bLxzs/jo/qMwP44L3UqkU5eXlXY71FPx1Xzt/zfw6gutCoVCX4C7/nvkhZv41Pa3fPZzLv05ERERERORYVbBwyRjzW+AMoMYY0wR8H/AArLU3A48B5wNvAG3AFwtVi/SO/hHbf3QPsvL1Ngzs6fz81wcL3PI754J6ugeEQZdd97qDDrnu4V/3oC6/g89am+vuy79//rX56+UfD95LpVK5cOdAAeDBfn6HK/+67kFc0HnWPYg6WLddfojmum5ujVAolAu8goArFArljufXkN9RmN/9lr9efh35oVr3Lrr89YOwMP/c/OvzA7megrNDdb5174ITEREREZFjS8HCJWvtZw9x3AJfL9T9RQay473Lrhjyt6Pmv+5Np1oQlHUP3fLfD8Kx7kFd9/WCcxOJRO787p18+a+DMC9/rfzts92/l4Pds6ffdz1tp+1pq2ygp46u4HP3QAnIdbj11IkWhGjB15FIpMegLP8e+R1xPXWT5R/P3yKayWSIx+NdQrog6Mv/COoBcl/3FJr1VFv38E1EREREpL8o5rY4EZF+Iz8YOF71FKAdLFjrqbOtp/Ar/7r8uXbB1wcK31KpFIlEItfp1v287vUEr4PtmYGDddFZa3Pdc927yg72c8oPkYKOr+5bOvPfC97PPz8Ip1zXJRKJ7BdUeZ7XpSss+Dr/uvzX3a/Pv39P8+DyA7hgnfwQLX87aE/ndw/Ogu42ERERKa50Ok1TUxOJRKLYpUiRRKNRhg8fvt8Yk/dD4ZKIiPTK8dpVc7BtlN2DsyCMyg/FgvOCr/O71/JDsPzjwbFMJoPv+7S0tPR4z+7bU/Pr7f51fhdbT9tAD/b9A/uFcj1tu8wPlKy1uZArCKM8z8t1mOVvt3RdN9cZlj8zrXswFqyXf15PWyrz79H9Iwjhgjq7r5G/7bP7tSIiIgNBU1MTZWVljB49+rj833bHO2stO3bsoKmpiTFjxvTZugqXREREDuJ4CNV6mj12oG2T+d1hvu+TTqdzc9S6nx+cm8lkyGQypNPpLls6g/Wy2WyX0O1QXXL5HWiHMxctf8tm/uvgw/d9XNftMkMMum5xdByHcDjcJZiKRCJEo9Eu7+eHYJ7n5UI0z/O6hHH5T/MM1s4P1oLruj/J81DbLXs6JiIiApBIJBQsHceMMVRXV7N9+/Y+XVfhkoiIyHGup216/Ulvtmx27zILBLPP8gOzoGMsm83mHgbQfetm/lqtra2k0+ke79E9aOv+sIEDfT/w3q9H91As2KKZHxj1FC7lh1JBWBUOh4lGo7mQK7/DKwiyIpFIrtus+0MD8kOy4Fg4HCYSieSCsfzgTaGWiMixqb/+N1/6RiF+/RUuiYiISL/WX7vLuodQ+dsne5pRln9+fijWPfTq6etgdlgymaStrS33uvv2ze5hW/dtl8HX3bdA5g/V777FMOjeyp+7lR9KBcFUPB7PrRF0gwXHu3eP5X90fyCAiIgc+1zXZfr06bnXDz/8MKNHj+7x3I0bN3LBBRewevXq93XPM844g3fffZdoNEppaSm33XYbEydOPKw1br75ZuLxOJdccgl33HEH55xzDvX19QB85Stf4Z//+Z+ZMmVKn9UZDof51a9+xYwZMw56zQ033MBll11GPB5/X/d+PxQuiYiIiBRB91Cst0Pji6V711b+bLFkMkkymewSWgVbIoNusGQy2eXaYLtkcG42m82FW9C1YyuYg9U9XArkP63RGEM4HN5vAL3neUSj0Vx4FXRj5QdYQQgW/MOj+33zQ7Wg+0tEpL974cE/0rq1uc/WKxtcw4kfP++g58RiMVatWtVn9+yte++9l1mzZnHLLbdw5ZVXsmDBgsO6/qtf/Wru6zvuuINp06blwqVbb721z+u8/fbbufLKK1m0aNFBz7/hhhv4/Oc/r3BJRERERI5tB5vdVFpa2mf3ye+2CkKodDoN0GVgfhBkdR+kH5yXSqVyA/Lztzrmd3Vls9ke53cFA+m7d0flP/0xeIpj/rD6INSKRCLEYrEux0tKSnKdXfkdW9FotMeh9flPZBQRKaTWrc1UDh/aZ+vtanr3iK7buHEjF198Mfv27QPgpptu4pRTTulyziuvvMIXv/hFUqkUvu/z+9//nvHjx3PPPfdw4403kkqlmDNnDr/4xS8O+n8AnH766dxwww0APPHEE3z7298mk8lw0kkn8ctf/pJIJMJVV13FggULCIVCnHPOOVx//fX84Ac/oLS0lNGjR/P888/zuc99jlgsxrJlyzjvvPO4/vrree6559iwYQPXXXcd0BFCrVixgv/8z/887Drnzp3LT37yk9zrr33tazz33HO0t7fzyU9+kv/3//4fN954I5s3b+bMM8+kpqaGJUuW8Oc//5nvf//7JJNJGhoauP322/v0v9U9UbgkIiIiIseM/BlQR1N+Z1YmkyGVSuXCKGC/YCqZTObCr/b2dvbu3ZsLvNLpdJetjEG4FcjfDpkfXHWfbRVsN8wPquLxeJfZV8EMrWDboed5xGIxwuFwl+2HwXyt7k9hFBEphvb29txWrzFjxvDQQw9RV1fHokWLiEajvP7663z2s5/l+eef73LdzTffzD/90z/xuc99Lvf39Nq1a7n//vtZunQpnudx+eWXc++993LJJZcc8P5/+MMfmD59OolEgksvvZQnnniCCRMmcMkll/DLX/6SSy65hIceeohXX30VYwy7d+/ucv0nP/lJbrrpJq6//npmzZq137G5c+fmwqX777+fq6+++ojq/NOf/sRHP/rR3Otrr72WqqoqstksZ511Fi+99BLf+MY3+OlPf8qSJUuoqamhubmZa665hsWLF1NSUsKPf/xjfvrTn/K9732vd784R0jhkoiIiIgc9/I7s4KApq8FXVb5nVNBaBW8n991FXy0tLTkzs0PuYLgKlg7WCcIjoIQKxQKEQ6H8X2/ywyrIIyKRCLE43FKSkpynVZ1dXXEYrHczyK4Lgitjnb4JyIDS0/b4tLpNFdccQWrVq3CdV1ee+21/a6bO3cu1157LU1NTXz84x9n/PjxPPHEE6xYsYKTTjoJ6Aiu6urqerxv0Gk0evRo/vM//5N169YxZswYJkyYAMAXvvAF5s+fzxVXXEE0GuUrX/kKH/7wh7ngggt6/b3V1tYyduxYli9fzvjx41m3bh3z5s1j/vz5h1Xnvn37yGazvPDCC7n3H3jgAW655RYymQzvvvsua9as4YQTTuhy7fLly1mzZg3z5s0DIJVKMXfu3F7Xf6T0XwURERERkaMgf2tdoeQ/8TC/CyudTnfZHpgfZrW0tNDc3NxlRlZ+aBXUHnRLBd1QpaWlRCIRKioqKC0tzQVP+VsEgyAqGEwbBFrB9r/8pxqKyPHtZz/7GYMHD+bFF1/E932i0eh+5/zDP/wDc+bMYeHChXzoQx/i1ltvxVrLF77wBf7jP/7jkPcIZhkFduzY0eN5oVCIZ599lieeeIL77ruPm266ib/85S+9/l4uuugiHnjgASZNmsTHPvaxXOB/OHU2NjZy1VVX8fWvf50HH3yQDRs25LbdVVZWcumll5JIJPa71lrL2WefzW9/+9te19sXFC6JiIiIiAwQjuMQDoff9zr5s62Cj3Q6TTKZpL29nXQ6zd69e9m1axdvvfXWfh1YwVMFg613wRbA/OHrwTa94MmAwTa/WCyWC6OA3CD2/K6p4ImCQZjVfXi7iPQ/LS0tDB8+HMdxuPPOO7s85CGwfv16xo4dyze+8Q3Wr1/PSy+9xDnnnMOFF17It771Lerq6ti5cyetra2MGjXqkPecNGkSGzdu5I033mDcuHHcfffdfOADH2Dv3r20tbVx/vnnc/LJJzNu3Lj9ri0rK6O1tbXHdT/+8Y9z7bXXMmrUKH784x8DcNZZZx1WnZ7ncc0119DQ0MDatWtJpVKUlJRQUVHB1q1b+eMf/8gZZ5zRpZaamhpOPvlkvv71r+e+p7a2NpqamnLdWYWicElERERERLp4v7Ovehqenh9SZbPZXIdUOp0mkUiwb9++3Ov864EuA9u7CwKl4Cl+wevy8nJKSkqIRqNEo1FKSkqIxWLE4/HcLKtgPlUQTGkOlUjxXH755XziE5/gd7/7HWeeeSYlJSX7nXP//fdzzz334HkeQ4YM4Xvf+x5VVVVcc801nHPOOfi+j+d5zJ8/v1fhUjQa5fbbb+dTn/pUbqD3V7/6VXbu3MmFF15IIpHAWsvPfvaz/a699NJL+epXv5ob6J2vsrKSKVOmsGbNGmbPng3AlClTDrvOWCzGv/zLv3D99dfz61//mpkzZzJ16lTGjh2b2/YGcNlll3HeeecxdOhQlixZwh133MFnP/tZkskkANdcc03BwyWT/2SM/qC+qsbecd9vil2GiIiIiIgcZd2fFuj7PolEgkwmQzabJZlMkslkcp+DLYHB+ZlMBmNMLqhyHCfXJRUOhykrKyMSiVBSUtJlOHppaWmXYCoIrIIwS6GU9Cdr165l8uTJudcvPPhHWrc299n6ZYNrOPHj5/XZelIY3X8fABhjVlhrZx3gkoNS55KIiIiIiPQLPXVUxePxXl+f30UVdEklk0lSqRTJZJLW1lZ27NiRC6symUwuxAquB3Jb/iKRSC6MKi8vz82iisfjxOPx3Da/4CPY/hd0S4kcCxQESV/Q32giIiIiInJcyH9K3uEIOqbyQ6f8YCqRSLBt27bck/7yO6yg68ypIJDyPI94PJ6bIVVaWko0GqWsrIx4PJ7rogoGpwezpUREjkUKl0RERERERA7iSGdQBU/kCwKp9vZ2kskk6XSaVCrF3r17SSQSGGO6bO3zfT/3FL1QKEQ4HM4FT1VVVQwaNCg3+DwWi1FdXU1ZWVmuK0pP4BORo03hkoiIiIiISAEYY3LDwoPtc73h+35uVlQw7DyVStHW1sbOnTtJpVL7DT0PgqVgdlRNTU2u6ykWizFo0CBKSkpyQ86Dp++JiPQFhUsiIiIiIiLHEMdxiEQiQMdMqaqqqgOeG3REBdvzkskk7e3trFu3LtcNFQRRxhistUSjUSKRSG77XWlpaZdgatCgQVRVVVFeXk4sFtN8KBE5pP73t0T/eridiIiIiIhIwYRCIcrKyg56TvCkvGBWVLA9r729nebmZjZv3pybERUEUsHaJSUlDBo0iEGDBlFTU5MLniorK3NPz9PT8kSk/4VLIiIiIiIi0mvGGDzPw/M8YrEY5eXlBz0/k8nktuHt3buXffv2sX79el599VWSySTWWlzXzXU+lZeXU1lZSXl5ORUVFdTV1VFWVkY0GiUajR6l71L6ix07dnDWWWcBsGXLFlzXpba2FoBnn32WcDjcJ/dZvHgxn/jEJxgzZgzJZJLPf/7zXH311Ye1Rjab5YwzzuDJJ59k/fr1PPvss3zmM58B4JlnnuG+++7jZz/7WZ/VmUgk+OhHP8qPfvSjg17zwgsvsG3bNs4999z3de++pHBJREREREREcoI5UfF4nJqami7HrLUkk0kSiQR79+6lpaWF7du38/bbb5NIJHJzoFzXJRqNUldXx+DBg6mqqqKuro7y8nLKysooKytTx9MxYuHChTQ3N/fZejU1NXz4wx8+4PHq6mpWrVoFwA9+8ANKS0v59re/3eUcay3W2vc9nP7MM8/k4YcfZu/evZxwwglccMEFNDY29vp613V58sknAVi/fj333XdfLlyaM2cOc+bMeV/1da+zra2NxsZGPvaxjx107RdeeIHVq1crXBIREREREZH+xxiT60gaNGgQw4cP73Lc9/3clrs9e/bQ0tLCli1bSKfTXYKn6urq3Ba7aDRKTU0Nw4YNo6KiglgspuDpKGpubmbo0KF9tt677757RNe98cYbfPSjH+XUU0/lmWee4eGHH6axsZHdu3cDcN9997F48WJuvfVWtm7dyte+9jU2bdqE4zjceOONnHzyyQdcu7S0lBNPPJE333yTCRMm8NWvfpUXXngBz/O44YYbOP3003n55Zf50pe+lPu9+vDDDzNy5EhqamrYvXs3V111Fa+//jozZszgS1/6ElOmTOGmm27i97//PWPHjuXll1+mvLwcay0NDQ08++yzZLPZw6ozHo/T2NjIO++8A8Dy5cv51re+RSKRIB6Pc8cddzB8+HB++MMf0t7ezl//+le++93vcu6553LFFVewZs0a0uk0P/zhD/nIRz5yRL8OR0rhkoiIiIiIiPQJx3GIxWLEYrH9BpGn02nS6TStra3s3r2b5uZm3nnnndw2vGDYeDDfqba2liFDhlBdXU1ZWRklJSUaLj7ArVmzhttvv52bb76ZTCZzwPO+8Y1v8J3vfIeTTz6ZjRs3csEFF7B69eoDnr99+3aeffZZrr32Wm688UbC4TAvv/wyr7zyCueffz6vv/46v/jFL/j2t7/NRRddlNv+me9HP/oRN910Ew8//DDQsZ0NOrqbLrjgAh555BEuvvhinn76aSZMmEBNTQ0XXXTRYdW5c+dO1q9fz6mnngrA5MmTeeqpp3Bdlz/96U9897vf5f777+d73/seq1ev5oYbbgDgO9/5Dueeey533HEHu3btYs6cOZx99tlHdVuq/mSKiIiIiIhIwQVzn+LxOIMHD+5yzPd92tra2LNnD62trWzatInXXnst10USCoUoLS2lpqaGUaNGUVFRkVtn0KBBeJ6nbqcBoKGhgZNOOumQ5y1evJh169blXu/atYv29nZisViX85YsWcLMmTNxHId/+7d/Y+LEiTz11FNceeWVAEydOpX6+nreeOMNTjnlFK655hreeustPv7xjzNu3LiDBlz5LrroIq677jouvvhi7rvvPi666KLDrvOEE07g1Vdf5d/+7d+oq6sDYPfu3VxyySW8+eabB73/n//8Z/74xz/mZjUlEgk2bdrEhAkTelV/XyhouGSMORf4OeACt1prf9Tt+EjgTmBQ5zlXWWsfK2RNIiIiIiIicmxxHIfS0lJKS0u7vJ/NZnPDxVtbW9m1axdvv/127ul3juNQUVFBNBqlvr6ekSNHMnLkSAYNGkQ8Hlfg1M+UlJTkvnYcp0v3UCKRyH1tre3V8O9gllG+7h1JgYsvvpi5c+eycOFCzj77bO68805OOeWUXtV92mmncemll7Jjxw4WLFjAv//7vx9Rna+++iqnnXYaH/3oR5k+fTpXX301H/rQh7j88st54403DjhjyVrLww8/TENDQ6/qLYSChUvGGBeYD5wNNAHPGWMWWGvX5J32XeABa+0vjTFTgMeA0YWqSURERERERPoP13Vz2+yqq6u7HEun07S1tdHS0kJ7ezvr1q3jxRdfxPd9Kioqcp1O+VvsgplOcuxzHIfKykpef/11GhoaeOihh3JPlfvgBz/I/Pnz+da3vgXAqlWrmDFjRq/WPf3007n33ns5/fTTWbt2Le+++y7jxo1j/fr1jBs3jn/6p3/i9ddf56WXXuoSLpWVldHa2trjmsYYLrzwQr75zW/S2NjIoEGDjqjOSZMm8Z3vfIfrrruOu+++m5aWFoYNGwbAHXfcccBaPvShD3HjjTfy85//HICVK1cyc+bMXv08+kohO5dmA29Ya9cDGGPuAy4E8sMlCwTPwawANh9qUUvPKaOIiIiIiIgcPzzPo6KigoqKitx7wfa6Xbt2sW/fPl5//fXckGOAWCzW5Ql2FRUVVFdXU1NTo0Hix6Af//jHnHvuuYwcOZIpU6aQTCYBmD9/Pl/72te4/fbbyWQynHnmmcyfP79Xa/6f//N/+Md//EemT5+O53ncddddhMNhfvOb3/Db3/4Wz/Oor6/nmmuu6XLdzJkzyWazNDY28uUvf5kpU6Z0OX7RRRcxd+5c7rnnntx7R1Ln5Zdfzvjx49m0aRP/+q//ype+9CWuu+46zjzzzNw5/+t//S9+8pOfMHPmTK6++mq+//3v881vfpPp06fj+z7jxo3jkUce6dXPo6+YA7WEve+FjfkkcK619iudry8G5lhrr8g7ZyjwZ6ASKAE+aK1d0cNalwGXAVTGSv7uvocfLEjNIiIiIiIiMvBkMpnc1rqdO3eyb98+UqkU2WwW3/fxPI/S0lKGDh1KXV0dVVVV1NfXU11dPeBCp7Vr1zJ58uTc64ULF9Lc3Nxn69fU1PDhD3+4z9aTwuj++wDAGLPCWjvrSNYrZOdST3/6uidZnwXusNb+f8aYucDdxphp1lq/y0XW3gLcAjB0ULW11g6oP9wiIiIiIiJSOKFQiPLycsrLy3PbjKAjdEomk+zdu5eWlhY2b97MunXr8H0f3/cJh8OUl5dTW1vL4MGDGTx4MEOHDqWmpuaQc3T6CwVB0hcKGS41ASPyXg9n/21vXwbOBbDWLjPGRIEaYNtBV/YtuAqXRERERERE5MiFQiFCoRAlJSVdnmCXzWZJJBK0trbS0tLC1q1bWb9+PalUCmMM5eXluVlO9fX1DBs2jMGDB+N5XhG/G5HiKWS49Bww3hgzBngH+AzwD93O2QScBdxhjJkMRIHth1rY+j7Gdfq4XBEREREREZGOQeIlJSWUlJQwZMiQ3PvZbJZ9+/bR3NzMnj17eOutt0in0xhjKC0tZciQIQwZMoTJkyczdOjQLk8/ExnIChYuWWszxpgrgMcBF7jNWvuKMeaHwPPW2gXAvwC/MsZ8i44tc5faXgyBKtScKBEREREREZEDcV03t70ukMlk2LdvH9u2bWP79u2sX7+ep556ilgsxvDhwxk3bhxjx45l8ODBRCKRIlb/Ho2aOb4VIlMpZOcS1trHgMe6vfe9vK/XAPMOc1Ws7x/6NBEREREREZECC4VCXZ5aZ60lmUyyc+dOtmzZwptvvom1lrKyMurq6hgyZAgNDQ2MHj2aeDx+1OuNRqPs2LGD6upqBUzHIWstO3bsIBqN9um6BXtaXKEMraiyt95xO15JrNiliIiIiIiIiByUtZa9e/eyY8cOWlpaaGlpIZlMEovFaGhoYNy4cUyYMIHq6mocp/DjX9LpNE1NTSQSiYLfS45N0WiU4cOH7zcj7Fh9WlzBqHNJRERERERE+gNjDGVlZZSVleXeSyQSbN++nU2bNrF27VpCoRA1NTVMnDiRCRMmUF9fTyxWmIYKz/MYM2ZMQdaW45fCJREREREREZGjKBqNMmLECEaMGEE2m2Xnzp1s376dp556iiVLllBaWkp1dTUNDQ1Mnz6d+vp6bWGTY1o/DZf611Y+ERERERERkZ64rkttbS21tbVYa2lvb2fnzp3s3LmTv/3tb/ztb39j+PDhDBs2jGHDhjFq1Ciqq6txXbfYpYvk9NNwSZ1LIiIiIiIiMrAYY4jH48TjcYYPH47v++zYsYMtW7awatUqli9fjjGGyspKxo0bxwknnMCoUaP2m50jcrT1u3DJWrCZbLHLEBERERERESkox3FyXU0A2WyWlpYWmpubWblyJcuWLaOqqoqRI0cyduxYpk+fTmlpaZGrluNRv3ta3JDySjv/5z+nbNiQYpciIiIiIiIiUjR79uxhx44dNDc309raSjQaZejQoYwfP56ZM2dSWVlZ7BKlHznunhbnq3NJREREREREjnPl5eWUl5czZswY0uk0W7ZsYfPmzbz55pv89a9/Zfz48UyaNImpU6cSj8eLXa4MYP0yXMqmM8UuQUREREREROSY4Xle7gl0mUyGbdu2sXHjRlatWkV5eTmNjY1MnTqVsWPHahi49Ln+Fy5ZdS6JiIiIiIiIHEgoFKK+vp76+npSqRSbNm1i+fLlLFu2jFGjRjF9+nSmTJlCdXV1sUuVAaL/hUsG/Kw6l0REREREREQOJRwOM27cOMaNG8fOnTvZsGEDf/jDH1i0aBEnnngi0z5Wg9UAACAASURBVKdPZ9SoUYRC/S8ekGNH//vdY8FPq3NJRERERERE5HBUVVVRVVVFNptl06ZNrFixgmXLljF06FCmT5/OSSedRHl5ebHLlH6o/4VLBvyMOpdEREREREREjoTruowZM4YxY8awZ88eNm3axJ/+9CeWLFnCtGnTmDVrFmPHjsVxnGKXKv1EvwuXLJq5JCIiIiIiItIXysvLmTZtGpMmTeKdd95h9erVvPjiizQ0NNDY2Mi0adOIxWLFLlOOcf0uXAKwCpdERERERERE+kwoFGLUqFGMHDkyN5vpgQceYPHixcycOZMTTzyRurq6Ypcpx6h+GS5l0toWJyIiIiIiItLXjDFUV1dTXV1NIpFg/fr1/OUvf2Hp0qXMmTOHD3zgA5SVlRW7TDnG9MtwyabTxS5BREREREREZECLRqNMmTKFSZMmsXHjRhYvXsyKFSuYN28ep556KtFotNglyjGiX4ZLmZTCJREREREREZGjwXEcxo4dy4gRI3jttdd47LHHeO6555g9ezazZs2ioqKi2CVKkfW7cMkYQ1bb4kRERERERESOKs/zmDp1Kg0NDbz++ussXLiQv/3tb5x88smcdNJJ1NbWFrtEKZJ+Fy4B+AqXRERERERERIoiGo0yffp0Jk+ezFtvvcXSpUtZvnw5J554IqeddhrV1dXFLlGOsv4XLhnIptNYazHGFLsaERERERERkeNSKBSioaGBMWPGsGnTJp5++mlWrFjBKaecwuzZsxUyHUf6X7iEIZ1MYbNZTKgfli8iIiIiIiIygDiOw+jRoxk+fDhvv/02Tz75JM888wyTJk1izpw5jB49Ws0hA1z/S2cMGAzWt8WuREREREREREQ6hUIhxowZw6hRo3jnnXdYvXo1q1atorGxkQsuuICysrJilygF4hRycWPMucaYdcaYN4wxVx3gnE8bY9YYY14xxvymF2tifR+bzfZ9wSIiIiIiIiLyvjiOw4gRIzjttNOYMmUKK1eu5KabbuLFF1/EWjWKDEQF61wyxrjAfOBsoAl4zhizwFq7Ju+c8cD/BeZZa3cZY+p6s7af9bG+X4iyRURERERERKQPGGMYMmQI1dXVrF69mt/+9re88sornH766QwfPrzY5UkfKuS2uNnAG9ba9QDGmPuAC4E1eef8b2C+tXYXgLV22yFXNQY/k8HPKlwSEREREREROdZ5nsfMmTPZvn07a9eu5eWXX2bmzJmcc845DBo0qNjlSR8oZLg0DHg773UTMKfbORMAjDFLARf4gbX2T4da2GLJpjN9VaeIiIiIiIiIFFhtbS3V1dU0NTWxcuVK1q5dy8yZMznxxBPVydTPFTJc6mkUfPfNlSFgPHAGMBx40hgzzVq7u8tCxlwGXAZQUVKKAfxUqs8LFhEREREREZHCcRyHkSNHUl9fz4YNG3j22Wd5/vnnOe200zjjjDPwPK/YJcoRKORA7yZgRN7r4cDmHs55xFqbttZuANbRETZ1Ya29xVo7y1o7KxaNAZBJKFwSERERERER6Y9CoRDjx4/ntNNOY8iQITz++OPccsstrFu3rtilyREoZLj0HDDeGDPGGBMGPgMs6HbOw8CZAMaYGjq2ya0/9NKGbCrdp8WKiIiIiIiIyNFljGHcuHHMnTuXnTt3ctttt/HQQw/R1tZW7NLkMBRsW5y1NmOMuQJ4nI55SrdZa18xxvwQeN5au6Dz2DnGmDVAFrjSWrvjYOsaA8Z1yCSThSpdRERERERERI6isrIyZs+ezebNm1m2bBnr1q3j/PPPZ9q0aThOIftipC8Ya7uPQTq21dfW2cs/cAH1jZMZPqex2OWIiIiIiIiISB9KJBKsWbOGlpYWxo8fzxlnnMHYsWOLXdaAZ4xZYa2ddSTX9s/4zxj8lJ4WJyIiIiIiIjLQRKNRTjzxRBobG3nrrbf49a9/zWOPPUZ7e3uxS5MDKOTT4grG9UKk9ZtKREREREREZMCqqqrilFNOoampib/85S+88sornHrqqZx00kmEQv0yzhiwet25ZIw51Rjzxc6va40xYwpX1sE54TCpfQqXRERERERERAYyYwwjRozgAx/4AMlkkgcffJBf/epXbNmypdilSZ5ehUvGmO8D/wr83863POCeQhV1KKFomExbApv1i1WCiIiIiIiIiBwlkUiE2bNnM3v2bN555x1+8YtfsHz5cvrbHOmBqredSx8D/h7YB2Ct3QyUFaqoQ3EjYdLJFNl0ulgliIiIiIiIiMhRVlFRwbx586isrOShhx7innvuoa2trdhlHfd6u0kxZa21xhgLYIwpKWBNh+TFIuxNpcim0oSikWKWIiIiIiIiIiJHkeu6TJ06ldraWlatWsW2bds444wzmDlzJo7TP59b1t/19qf+gDHmv4BBxpj/DSwGbi1cWQfnhD38bJZMm+YuiYiIiIiIiByP6urqOP3000kmkzzwwAPcfffd7Nixo9hlHZd61blkrb3eGHM2sAeYCHzPWruooJUdhOOFMBiSe9soaguViIiIiIiIiBRNNBpl1qxZ7Nq1ixdffJFt27Zx3nnnMXXqVIwxxS7vuNHbgd4/ttYustZeaa39trV2kTHmx4Uu7oD1OA64hlSr9lWKiIiIiIiIHO8qKyuZN28eyWSSe+65h3vvvZfNmzcXu6zjRm+3xZ3dw3vn9WUhh8vxQiRb9xazBBERERERERE5Rniex+zZsznhhBNYt24dt9xyC0uXLiWth4EV3EHDJWPM14wxLwMTjTEv5X1sAF46OiX2LBSN0r6zpZgliIiIiIiIiMgxpqamhnnz5lFdXc2CBQu46667aG5uLnZZA9qhZi79Bvgj8B/AVXnvt1prdxasql7wSmIkWlrxMxmcUG8feiciIiIiIiIiA53jOEycOJH6+npWrlzJ7bffzllnncWMGTP0RLkCOOhP1FrbYq3daK39rLX2LaAdsECpMWbkUanwAMJlcVLt7aTbEsUsQ0RERERERESOUWVlZZxyyikYY7j//vt58MEHSaVSxS5rwOlVy48x5iPAT4F6YBswClgLTC1caQcXisfwsz5tzbuIlJcWqwwREREREREROYaFQiFOOOEEtm3bxrPPPsvWrVs555xzGD9+fLFLGzB62wt2DXAy8Jq1dgxwFrC0YFX1ghNyMY5D+47dxSxDRERERERERPqBuro65s6dy7Zt27jzzjtZsmQJ1tpilzUg9DZcSltrdwCOMcax1i4BZhSwrl4JlURp3bK92GWIiIiIiIiISD9QUlLC3LlzGTZsGI8++iiPPPIIyWSy2GX1e72dhL3bGFMK/A9wrzFmG5ApXFm9EykrZd/2nWSTKdxIuNjliIiIiIiIiMgxznEcGhoaiMfjLF26lI0bN3L++eczYcKEYpfWb/W2c+lCoA34FvAn4E3gI4UqqrdiVYNItSVI7GopdikiIiIiIiIi0o8MHTqUefPm0drayl133cWjjz5KIqGHhh2JXoVL1tp91lrfWpux1t4JzAfOLWxph+bEwhjXYc9mbY0TERERERERkcMTj8eZPXs2o0eP5qmnnuLXv/4127ZtK3ZZ/c5BwyVjTLkx5v8aY24yxpxjOlwBrAc+fXRKPGh9eCUxWjZtLnYpIiIiIiIiItIPGWMYOXIkc+bMYdu2bdx66628+uqrxS6rXzlU59LdwETgZeArwJ+BTwEXWmsvLHBtvRKrqaR1azOpvW3FLkVERERERERE+qmSkhLmzJmD67rcc889LFq0SMO+e+lQA73HWmunAxhjbgWagZHW2taCV9ZL0apyWtY3seftLdRMHlvsckRERERERESknwqFQsyYMYO33nqLxYsXs3HjRj7xiU9QVVVV7NKOaYfqXEoHX1hrs8CGYylYAjCuS6gkxq4Nm4pdioiIiIiIiIj0c8YYRo8ezZw5c9i4cSP/9V//xQsvvIC1ttilHbMOFS41GmP2dH60AicEXxtj9hyNAnujZEg1LU1bSe7ZW+xSRERERERERGQAKC0tZd68eYRCIX73u9/xu9/9jn379hW7rGPSQcMla61rrS3v/Ciz1obyvi4/1OLGmHONMeuMMW8YY646yHmfNMZYY8ysI/kmolUVZDMZdr6+8UguFxERERERERHZTygUorGxkQkTJrBixQpuvvlmmpqail3WMedQnUtHzBjjAvOB84ApwGeNMVN6OK8M+AbwzBHfy3GIVFWw/bUN+OnMkS4jIiIiIiIiIrKfYcOGceqpp9LS0sLdd9+tbXLdFCxcAmYDb1hr11trU8B9QE9PmPt34Dog8X5uVlZfR9uOFlo2bX4/y4iIiIiIiIiI7CcSiXDKKacQDoe5//77WbRoEZmMGlygsOHSMODtvNdNne/lGGNmAiOstY++35u5sQheSYytL6/Dz2bf73IiIiIiIiIiIl04jsP06dOZMGECTzzxBL/5zW9oa2srdllFV8hwyfTwXq5nzBjjAD8D/uWQCxlzmTHmeWPM8/va2w94Xtnoelre2UrLW+peEhEREREREZHCGDFiBH/3d3/H6tWrufPOO9m1a1exSyqqQoZLTcCIvNfDgfzUpwyYBvzVGLMROBlY0NNQb2vtLdbaWdbaWSWx2AFvGC6NEyqJ8+4La9S9JCIiIiIiIiIFM2jQIObOncumTZu47bbbjutB34UMl54DxhtjxhhjwsBngAXBQWtti7W2xlo72lo7GlgO/L219vn3c9OKMcPYs2U7za+ufz/LiIiIiIiIiIgcVElJCaeeeip79uzhrrvu4s033yx2SUVRsHDJWpsBrgAeB9YCD1hrXzHG/NAY8/eFum8oHiVWW0nTMy/SvnN3oW4jIiIiIiIiIoLneZx88skYY7j33nt58cUXi13SUVfIziWstY9ZaydYaxustdd2vvc9a+2CHs494/12LQXKR9eTyWTY+LfnyKbSfbGkiIiIiIiIiEiPXNdl5syZhMNh/vu//5vly5cfV0+SK2i4VCzGcaia1EDL5q00LV+F9f1ilyQiIiIiIiIiA5jjOMyYMYPKykoeeugh/vu///u4CZhCxS6gUEKxMIMaRvLuy+uIlJUyeMYkjOnpAXYiIiIiIiIiIu+f4zhMmzaNuro6Vq5cie/7fPrTnyYUGrDxCzBAO5cC0eoKSofVsemZlexYe3wO1RIRERERERGRo6uuro4ZM2awcuVKHnzwwQHfwTSwozOgdNhg/EyWDU8+j3FdqieOKXZJIiIiIiIiIjLAVVdXM2PGDFasWEEymeQzn/kMnucVu6yCGNCdS4HyUfWEK0pZ/9dn2PbSOs1gEhEREREREZGCq62tZcaMGbz44ov8/ve/J50emA8dOy7CJYCKhhFEayvZ8NQKmpavwh/gLWkiIiIiIiIiUnzV1dXMnDmTFStW8PDDDw/IgGnAb4vLVzZiCKFomHdeWEOyZS8jT5tFuDRe7LJEREREREREZACrra2lsbGR5557Dsdx+NjHPobjDJx+n+MqXAKI1VbhRiM0r9vAvuadDJ/TSFXDKIw7cH5RRUREREREROTYUldXxwknnMCzzz5LWVkZH/zgBwdMwDQwvovDFC4rYfDMydiQwxuLl/HmoqW072wpdlkiIiIiIiIiMoDV1dUxceJEFi1axOOPP44/QGZCH3edSwHjugwaN4pUSys7179NS9MW6mdOpnbqeELRSLHLExEREREREZEBaPjw4Rhj+Otf/4rv+5x33nn9voPpuA2XAuGKMmobJ7H3nW28tfxFtq19kyHTJlIzeSxuJFzs8kRERERERERkgBk2bBi+7/Pkk09SXl7OqaeeijGm2GUdseM+XAIwjkPZiCGUDK6mtWkrG5Y+z9bVrzHkhIlUTRitTiYRERERERER6VMjRowglUrx+OOPA3DaaacVuaIjp3ApjxP2qBg7nLLhg2ndtIX1//McTStWUztxDDWTxhKtrOjXSaKIiIiIiIiIHDsaGhrwfZ/HH3+cWCzGrFmzil3SEVG41AMn7FExbgRlo4bStrWZzS++yrsvraNyZD01k8dSPmwIbtgrdpkiIiIiIiIi0s+NHz+ebDbLI488QiKRYN68ef2usUXh0kE4XojS4UMoqR9Mctcedr+zlR0b3iZSVkLV6OFUjhlB6ZAaHE8/RhERERERERE5MhMnTsR1XRYuXEg0Gu13HUxKRXrBOIZodQXR6gqyiSTt23ex9dU3ePfldUTLSqkYMYTyYYMpG1qHVxrvdwmjiIiIiIiIiBSPMYbx48eTTqd59NFHGTJkCMOHDy92Wb1mrLXFruGw1NfW2R9888pilwFAtj1Be3ML7bt2k02kcEMepbWVVIwYSvmwwcRqKrV9TkRERERERER6xfd9nnnmGQYNGsSXv/xlSktLj9q9jTErrLVH1DKlcKmP+JksqZa9tO/YTWrvPmwmSyQeo2RwDeVDaympqyJaOYhQLKLOJhERERERERHpUSKR4Omnn2bEiBF88YtfJB6PH5X7vp9wSdvi+ogTcnNb56y1+Ikkid2ttG7bwe63N+NnsoQiEeKDyiiprSZeU0m8ppJwWYkCJxEREREREREBIBqNMmfOHJYtW8Yf/vAHPv3pTx/zmYHCpQIwxuDGopTEopQMrQUgm0qT2rOX1O69bH19A/aV17DWEo7FiJTGiddWEq+qIFZdSXRQOV48inGcIn8nIiIiIiIiInK0lZSUMH36dF544QUmTJjAzJkzi13SQSlcOkrcsEesppJYTSVAR3dTMkWydR/p1jZ2bGhi29o3sdbihrz3AqfKCqKDyolVlhOKxwhFwwqdRERERERERAa4wYMH09zczGOPPcaIESOoqakpdkkHpHCpSIwxuNEI8WgEaquAzsApnSGzr41Uaxst72xl54YmbCYDGLxYBC8eI1ZRRnRQGZHyMqKVZYRLSwhFIxoeLiIiIiIiIjKATJw4kWXLlvHQQw9x6aWX4nnH5r/7FS4dQ4wxuGEPN1xBpLIi9771ffxEivS+dlJt7ezZ1szOpnfxM1nwfdyQRzgeJVJa0hk6lRIpLyVaWY6nbicRERERERGRfikUCjFz5kyWLVvG0qVLOeOMM4pdUo8ULvUDxnFw41HceJQolbn3rbXYTJZMWzupfe20t7XTun4XNpPpCJ6AkOcRLokRKS0hXF5CpDROuLSEcGmccEkcJ+x1BlrHZvopIiIiIiIicjwrLS1l7NixLFmyhIaGBkaMGFHskvZT0HDJGHMu8HPABW611v6o2/F/Br4CZIDtwJestW8VsqaBxBiD8UKEK8oIV5R1OWazPtlUmkxbgnRbO21797GneSc2m8VmslhrcVyHUDjSsd0uFiVSVkK4JI4Xj+KVxAmXxHCjYdxwGDccwgkpixQRERERERE52saMGcP27dt56KGHuOyyy4hGo8UuqYuCpQXGGBeYD5wNNAHPGWMWWGvX5J22EphlrW0zxnwNuA64qFA1HU+M6xCKRQjFIkSrK/Y77mey+KkU6fYkmfYEifZ29rbs6QieMln8bBZjwY14hLwwoUgYLx7p6HoqiRGKR/Hi0Y4wKhbFjYRxw56234mIiIiIiIj0McdxaGxs5Omnn2bRokVccMEFGGOKXVZOIVtRZgNvWGvXAxhj7gMuBHLhkrV2Sd75y4HPF7AeyeOEXJxQjFA81uNxay02nSGTSJJpT5JNpWjbu4/Wnbs7A6gMNmuxWIzjEAp7eJFIx3a70jheSZxISQyvNE64rAQvEsGNhnG80DH1B0BERERERESkP4jFYkyePJnly5czduxYpk6dWuyScgoZLg0D3s573QTMOcj5Xwb+2NMBY8xlwGUA5SWlfVWfHIQxBhP2CIc9wuU9/8yt73d0QCVTZBIpMskk7W1t7N3Vgp9OY7M++D4WcFyXUDhMKBrumP9U1tEB5ZVE8eIxvFgMLxrB8UK5D4VQIiIiIiIiIu+pr69n27ZtPProo9TW1lJXV1fskoDChks9JQO2xxON+TwwC/hAT8ettbcAtwDU19b1uIYcfcZxcMMObtjDKyvp8RybzeJnsmQSKbKJJNlkirbW1o75T5kMNuODtYDF0jGA3A17hDyPUCzaMQ+qJEYoGun8CBOKRvGiYdxoBMd1O8KokKsteSIiIiIiIjLgTZ8+nWXLlvHggw/yhS98gVis5x1JR1Mhw6UmIH+E+XBgc/eTjDEfBK4GPmCtTRawHikC47q4rosbCUPFATqgOgMom8mSTabIpNL4qQzJVIK2vXvxN2ew2Sz4tqMbiiCKAjcUwvU8QmGPUCxCOB7rmDUViXTMi4qECUWCUCqCEwphQm7HtkAvhOO6R+1nISIiIiIiIvJ+ua7LiSeeyPLly1m8eDEf+chHil1SQcOl54DxxpgxwDvAZ4B/yD/BGDMT+C/gXGvttgLWIsewIIAiAqGSGJGDnGt92/HEu6yPzWTIpjJk02my6TSpVJrE9jayuSAqP5DqiKM6wqiObXcdXVLhXBeUG+4Io9xIxwDzoFPK9TycUAjHc3PhlLbsiYiIiIiISLHE43EmTZrE008/TUNDA1OmTClqPQULl6y1GWPMFcDjgAvcZq19xRjzQ+B5a+0C4CdAKfC7zn+sb7LW/n2hapL+zzgG44TAAwj36jewtbYzjMqSTaXx02my6Qx+Ok0y3TEnys9k3psRFQRY1nbe08F13VzA5LqhjgAqGu7YuhcN54VSnV1Sna+dkItxXRzXxbiOtu6JiIiIiIhInxg6dCjvvvsuCxcuZPTo0cTj8aLVYoJ/QPcX9bV19gffvLLYZchxIOiQyqazHQPK0xmynd1SfiaDn87kQimb9bG+D37Hn6f8bXtOyMUNebidW/HcsIcb7QyiIh5uyOvYohd2CXlh3LCHE/ZwvY4Qy3EdBVQiIiIiIiKyn2QyydKlSzn55JO58MIL39cuG2PMCmvtrCO5tpDb4kT6NeN2dh2FPSDaq2u6bNvLZjs6pIIgKp0lk8mQamvD7mklm850dEf5HUPNbdZ2DjeH9+KpjiftOaGOrYMmCKq8EG40/N5WPtfFBMFVZ6DleKGOr8MdQ9JdL9QZUjmd35ujmVMiIiIiIiL9WCQSYfLkyTz33HNMmjSJSZMmFaUOhUsifajrtr3D/wNmfYv1gy6ojo4oP5PNdUn52Sx+Oks6kyHZkug4luncwmdtx4ypzg4qi809stECBtO5Tc/BcTqHmrsOrhfGDXd0SbnhMG7YxQl5uJ7bMW/K6xia3jGrquN9x/Pe66Tq7KbKfdY8KhERERERkaNm6NChbNmyhYULFzJs2DDKysqOeg0Kl0SOIR3hlEvHmLIjZ/OCpqA7ymayZDPZzsCqI6iymSyZTJZUWwrb2oqf8Tuv8cEnN4Oqp44qg+kMqjpCJcd1OjurOkKoUNjDDYdwvDChcMcg9JDn5Z7W1zEoPZhlFWwfDOWFVW7nz8PJBVkiIiIiIiKyv2nTpvHUU0/xxBNPvO/tcUdC4ZLIAGSMAdfsF8gc6R94m+uI8rt0V9lsZ+eU73cEU9ksfjZLJpsh1ZroCLCyed1Y1r4XfFmLsflxVWftnQPUTRBaeS6u07lF0XNzW//cSBg35ILrEvI6wivHfW87oOkMroJZV04o1BFWda7tBCFWZ9eViIiIiIhIf+V5HpMmTeL5559n8uTJTJw48ajeX+GSiBxSX3VUddfRFZUfWNncVkDrZ94LrDI+vp8lk0ljkylsJoOf9d/bDmg7wy9LLsTqyKyCAevB99G5JdB9r9sKx8F1HZyQh+O5uW2AHZ87Qim3M7QyIadzALvbeczDCTm4Ye+9eVbduq40gF1ERERERI6GoUOHsnnzZhYuXEh9ff1R3R6ncElEisY4BnAxBZgrnuu2st2Dqwz4fm6LYNCBlclmsYk0/j4f6wcdV9nOAIxcCNb9iYC578U4GMd0dE/lhUxdBqx7oa4zrDqHtQfbAp3g3M5tg27n+yYUeq/LKhdaKbwSEREREZGupk+fzlNPPcVjjz3Gpz71KZyj9O8FhUsiMiC9123V94Kwis6ZVtb3IeOTzb4XWPmZTMfndJaMn8W2pfCznU8T9LP4vsVY2/nZx/rkzbYiN5C9564rk9vOF4RZHcFUCMfrmG3lhD1c14X8Jw26edsLQ51PD/Q6tw52BljkhVb59zHGKMgSERERETnGhcNhGhsbWblyJWPGjGH27NlH5b4Kl0REDlNH8ALdtwm+379Q8+dR5Z76Z33I+vi+xWayWNsxnN23nUFVtiPUyqbS+IlEZ4dWFj/rAzbXeWWt3zHjyr7XfdXTtkFjTMf/u+GY3LB24zo4pvNz0F3VGVQFWwcdz829n+vCch2MG+rYOhjq2FbouKHckwoP2JFlOkMuPXlQREREROSwVVdXM3jwYBYvXszo0aOpq6sr+D0VLomIHCNyg9iBvp5vlc/m5lTZ94asZ338zi2CQbDlZ7IdQVc223nMz20ftPnD3Tu7tfLDrPz1O94w+20lBDq6rhx3v61+TjC7yjWdWwtDHYFXKIQb6gy53I7B7a7ndn4dPL3QxQl1rpMLud4Lv0zIzXViBffEGG01FBEREZEBY/LkySxbtowFCxbwuc99jlgsVtD7KVwSETnOGGM6w5Su7xcizuoxyNqvIyuDbzu2FnZ0ZHWGW9ksvvXJJlP47YnONTrmYfmdWxLxfcC8N1ur8ymEXWroPh8L07nVz8G4Jhc05XdsBR1YJhTqeO11dmp1DnEPQirHdd4LtlwXYzoHxXduJezYtuh2dn0FTyp039uC2BlqYdS5JSIi/3979xsjV3Xecfz33BnbgGInks2LkHWB2mxqbFdA3SQWtFBBKgcBfgEpREobEAK1KqX0n9SqVUsTqQqt0igVtJQQRBKlzR8qtRZNTNSCARkM2ED5pySyDA1uUpIS4EUSvN57n74459x7ZnbWO57dmV3Pfj+S8Z2ZO3fOGl/t8uN5ngMAC6coCp133nnat2+fdu/erZ074cnV7QAAEmtJREFUdw51/hLhEgBgaGYLsoapcxfBLNSaUaWVhr1XTQVWHNxelmUY+n50qh4GX1dqpesqBmXx97r9UJ2hUK+KrTRDK1RLWedA+KKQWlZXbxVFS8XKNDurXc/aaoKqzhlcnTsXhtlj4XEMuVIgFgOwJuQqOoOurtcAAABwYjn55JO1ZcsWPfnkk1q/fr22bds2tM8iXAIAjJUUaEkayk6Ec6mDJ/fYFZhXbMXnyyrMxYo7ElYx4KqrubxUWcaqrmpK+nHVBFsxxApVW+mjsnbEOlxTPSS+e0C8JJkUWg/zCi7rbBUsUvAVK6+KVudsLUuthkXRtB9ms7rqFsXU/thqZmwVsSostSnWw+Traq6sVZKgCwAAYCCnnnqq1q1bp4ceekibN28eWnsc4RIAAAvICpNi9dJiN7Y1oVPVGXTFlsKqTBVb3oRb7vKqDIVZWUVXOT2tamoq7JJYNhVcrrDz4Yzr13PjswqyHlVcUvxzsqKjiiuFTHWgVFgYLG/WtCHGYKsOv4qwG2KRhWYqCrXiOSqKEJh1tzAWVs/+SvO66vbEFJxZdytjrPRSMwxfJtoaAQDAkjM5Oam9e/dqz5492rFjx1B+TiFcAgBgTKXAQ2r1DLpGXdjVs00xBl2p7bAqPYZhKcRSHXCp8ljpldoWj4b3uccqMNWtjaGkKwZsUseuiZJmhmDqXeGV6wi+YjVVmF3QtDeG56yu4iraWatiquyqjy22K6YQK87jMsVWxraKGJCFGV5FE6jlrZBFsy5lazNZz9lekpjvBQDAMrJq1ao6YNqwYYMmJycX/DMIlwAAwEikqq6lFGX0bClUHj55c05ZqUrhlVchzKrndjXnhjCsVFm5jh6dkh/JA7Tw3iqFXvVQeslVxbWoXkfH58fqr/Tn19Hm2CP4MnUOis9nfKUqKzOFiq56TlcIu4qi1QyhT5VjdVtjbJvM2xxbrVgZVtTVXkoBmBVSO7ZGdg+x71XxlVWtKT+ftkgAAAY2MTGh1157Tffff79uuOEGrV69ekGvT7gEAACWraa6a2mFXrPpOdOrqqRYtVVVcR5XFnRVVRo67yEcKytJHp+X3Ev50en6Gvnw+o6Qq8qqwOogLDxvlpaVz/vq3QhpeQukqQnA6qqqGF5lwVMIolQHYalqrJn31Q4tjXFGWD4/rL5+1/NF3SrZeY7ijpLpecXwLA3Br8O52doju6vIAABYIrZu3aq9e/dq9+7duvLKKxd09zjCJQAAgBPEUprpNZfO6i81x6XHECsGWZ7N+4ptknLVLY8h6KpUlSkIK+OA/KreAdLl8kqSPGuFzIKutAbFgfjuMlkcku91u2Q8o/7njEH43RViUtN62D0QP2udtFYhWV4hVjSVYdlQfKX5X2YdA/BnhGRFq64oq2eGtUym5tqS6lbKpvIrVg9m1WDd1WGEYgAwvlauXKmtW7fq6aef1qZNm7Rly5YFuzbhEgAAABZcx86Ni7yWufTecTFWXqXjukLMuyrEFF+rsjAsPueuMrZCqqrkR6v6dXW0U8b3dFWLNYPx43uU1jnL13GMofl1VVbdDtkrFFMYnJ8CrTgvrKPCq13IZCrahVS0siH5TSBWzxZLQVodhKVZY9nssa73pvbHGXPB6nCMIfoAMB/r1q3TmjVr9PDDD2tyclIrV65ckOsSLgEAAGBZO+HaI7vbDzsqtFIolR1XoT0yBV6pvbJugczaIFPgVVaV5KX8yFHlbZTyUCmm0uNmlFW9JssH6XuqElM2MywFjenYsuOZmVl3C2UzSyyvsiqkIlzL2mHG14xQLA6/DztEttTKBuHXwVTcWVLZQP7UFtlq5YP5m90q6yH72a6WPeeK5TtfAsASsGnTJu3bt08HDhzQ9u3bF+SahEsAAADACSQPw5p/Ln29WiU7d27sCspS+2O9i6TqNsrU0ljPIatCAFZWZdg9cvpoM3sszhZrArEyrKeSZKEtsm6RlNUtl5ZViR1rN8lQrZVaE7sH0qdZXfFXO7Y1pkqtdr5zZKGi1ewSWaQALA7Gt3ZW5WVpNlirCbYKi+eFz0i7TCq1X6b3ZfPCZuwySQgGLAunnHKKTjvtND344IPavHmz1qxZM+9rEi4BAAAAGLoTqVUymbVlMlVnpdlhZVm3RaqsVMXKMOWzxSqpqkp5VcrL6RCYVd7xen2NGLRZCsg8VnelarCwuB5pV1MHNlsQZhZmgeWzuOoWRmvaGa2u4mrFQEpqrWjXQVg9UD9rg+xoieyeE5af0y7qGWEp9GoqwXpUgXXsJEkrJLAQNm7cqEcffVSPPPKILrvssnlfj3AJAAAAAHpY6i2Ts7VI5hVheShWlVU2I6yzEizMEovnKOws6WUZQ7IyVn15HKifqsmyIfpVDLbia4pVYHGDy+zPL5yXqsHSM4nJZK04fD5riaznfqVh+bEF0lphR8fWihWx+iu2RrbbzXEKz1a06xlg4XqpXbLoHI7faquVBu9nbZRpYD8tjxgH7XZbGzZs0P79+7V9+3atXbt2ftdboHUBAAAAAEboRGmRnHX3yB4zwrys4o6SXu8MGXaajLtKlrGyKwZh1dFpWVnK3z5S70SptCtlDNcs+6wm8PJ658i4gtAW2aMFsm59zIOujllbMZyKgVbRbnfsCtlqr1DRLsJ57azqKw21b4Vgq2i1QtBlYWh+0WrH1sks6OpubSTowjxMTEzo0KFDeuyxx3T55ZfP61pDDZfMbIekz0hqSbrb3T/Z9foqSV+Q9AuSXpd0tbu/Msw1AQAAAABGZym3RKYh9jMqv+pgS6Gaa7rMhuKHEKuKu0FW02GOV1lVKqdKVW8fidcow8yvtJukhwH4ocorBV/5GmJDY0y2UudfCrpCjhiDqVTBlbc4trKh9itaarXb9bEVhVorVqhoxxZFK8LzKeCKLYjtk1bF9+WzwFrN3K4UrKU15K2VBFsnnKIodOaZZ+rAgQO64IIL5nWtoYVLZtaSdIekD0o6LOkpM9vl7i9lp10v6Q1332hm10i6TdLVw1oTAAAAAACJFaEFT1r84CuFTO6dv9fHZaWqqjrneXUFXXVFV1lqamqqrvoKz5dKVV+qXLLQumguVV7Jj5ayVhMQzajgUhokX3RWcGWth1a0YtVVCLSKdksyU2tFO4RU7TSEvog7Mcaqr1i1FYbaN3O/wvmt+nNTVVg+L2xGq2I6Rl8mJib08ssv6/HHH5/XdYZZufQ+SQfd/ZAkmdmXJe2UlIdLOyXdGo/vk3S7mZnXjbsAAAAAAIy/FHQtVizSVGz1CLdioFVVzSyuOtwqy/h8eH16upQficFWVcpLV1WVnbtCysMQ+6qSTLGCK7YsNts0htfUPJTS4yJUV1kMoKyQTKEyK2sZVN6CmOZ2tVM1VufujIqztYp4LYsVXu4eqryUV2wVskL151oecMUKsnwwfQrBiqylURZ2mJQpVp/Fry4dp2q/rPKveVw/6Ph3OONvz2x/mbL3FUWh008/Xfv37+/vL8oshhkuvUfSq9njw5LeP9s57j5tZm9JWivp/2a7aFmWOvzKfy/wUgEAAAAAwLykQrC6AKoVf+8/esgH0nfM6pI3LYVVJU+ztaqpMIsrtjdWlUtl1zXSdaXm+dSCWOcscQZXvT1j+oKaXRgtza2PoVCd8SgPf7KDHhVUs24U0H1uOsc6nuhx+mzP93h9lrCp8kqvvjFrDNOXYYZLvZbdXZHUzzkysxsl3RgfHvnE3Xe8MM+1ATh+63SM4BfA0HDvAYuH+w9YHNx7wOJ476BvHGa4dFjS+uzxhKTvzXLOYTNrS3qnpB91X8jd75J0lySZ2X533zaUFQOYFfcesDi494DFw/0HLA7uPWBxmNnAvXHDHOf+lKSzzOxMM1sp6RpJu7rO2SXpY/H4KkkPMm8JAAAAAADgxDG0yqU4Q+kmSQ8oNFre4+4vmtnHJe13912SPifpi2Z2UKFi6ZphrQcAAAAAAAALb5htcXL3r0v6etdzf54dvy3pw8d52bsWYGkAjh/3HrA4uPeAxcP9BywO7j1gcQx87xldaAAAAAAAABjUMGcuAQAAAAAAYMwt2XDJzHaY2bfN7KCZ/XGP11eZ2Vfi60+Y2RmjXyUwfvq4937fzF4ys+fM7D/N7PTFWCcwbua697LzrjIzNzN20QEWQD/3npn9Wvze96KZ/dOo1wiMqz5+7vwZM3vIzJ6JP3teuhjrBMaJmd1jZj8wsxdmed3M7O/iffmcmZ3Xz3WXZLhkZi1Jd0j6kKSzJX3EzM7uOu16SW+4+0ZJn5Z022hXCYyfPu+9ZyRtc/efl3SfpL8e7SqB8dPnvSczWy3pZklPjHaFwHjq594zs7Mk/Ymk8919s6RbRr5QYAz1+b3vzyR91d3PVdj86e9Hu0pgLN0raccxXv+QpLPirxsl/UM/F12S4ZKk90k66O6H3H1K0pcl7ew6Z6ekz8fj+yRdbGY2wjUC42jOe8/dH3L3n8SH+yRNjHiNwDjq5/ueJH1CIdB9e5SLA8ZYP/feDZLucPc3JMndfzDiNQLjqp/7zyWticfvlPS9Ea4PGEvu/oikHx3jlJ2SvuDBPknvMrN3z3XdpRouvUfSq9njw/G5nue4+7SktyStHcnqgPHVz72Xu17SN4a6ImB5mPPeM7NzJa139/tHuTBgzPXzfW9S0qSZ7TWzfWZ2rP/bC6B//dx/t0r6qJkdVtiF/HdGszRgWTve/yaUJLWHtpz56VWB1L2tXT/nADg+fd9XZvZRSdskXTjUFQHLwzHvPTMrFFrArx3VgoBlop/ve22F1oCLFKp1HzWzLe7+5pDXBoy7fu6/j0i6190/ZWbbJX0x3n/V8JcHLFsDZS1LtXLpsKT12eMJzSyBrM8xs7ZCmeSxSrsAzK2fe09mdomkP5V0hbsfGdHagHE21723WtIWSXvM7BVJH5C0i6HewLz1+zPnv7n7UXd/WdK3FcImAPPTz/13vaSvSpK7Py7pJEnrRrI6YPnq678Juy3VcOkpSWeZ2ZlmtlJheNuurnN2SfpYPL5K0oPuTuUSMD9z3nuxNecfFYIl5k4AC+OY9567v+Xu69z9DHc/Q2He2RXuvn9xlguMjX5+5vxXSb8iSWa2TqFN7tBIVwmMp37uv+9KuliSzGyTQrj0w5GuElh+dkn6jbhr3AckveXu35/rTUuyLc7dp83sJkkPSGpJusfdXzSzj0va7+67JH1OoSzyoELF0jWLt2JgPPR57/2NpHdI+lqcof9dd79i0RYNjIE+7z0AC6zPe+8BSb9qZi9JKiX9kbu/vnirBsZDn/ffH0j6rJn9nkJbzrUUFADzY2b/rNDqvS7OM/sLSSskyd3vVJhvdqmkg5J+Ium6vq7LvQkAAAAAAIBBLdW2OAAAAAAAAJwACJcAAAAAAAAwMMIlAAAAAAAADIxwCQAAAAAAAAMjXAIAAAAAAMDACJcAAMCyZGZrzezZ+Ot/zex/4vGbcdv5hf68i8zs/uN8zx4z29bj+WvN7PaFWx0AAMDgCJcAAMCy5O6vu/s57n6OpDslfToenyOpmuv9ZtYe9hoBAABOBIRLAAAAM7XM7LNm9qKZfdPMTpbqSqK/MrOHJf2umZ1qZv9iZk/FX+fH8y7MqqKeMbPV8brvMLP7zOxbZvYlM7N4/sXxvOfN7B4zW9W9IDO7zsy+Ez/7/BH9OQAAAMyJcAkAAGCmsyTd4e6bJb0p6crstXe5+4Xu/ilJn1GoePrFeM7d8Zw/lPTbsRLqlyT9ND5/rqRbJJ0t6WclnW9mJ0m6V9LV7r5VUlvSb+WLMbN3S/pLhVDpg/H9AAAASwLhEgAAwEwvu/uz8fiApDOy176SHV8i6XYze1bSLklrYpXSXkl/a2Y3K4RR0/H8J939sLtXkp6N131v/LzvxHM+L+mXu9bzfkl73P2H7j7VtQYAAIBFxawAAACAmY5kx6Wkk7PHP86OC0nb3f2n6vRJM/t3SZdK2mdml8xy3bYk63NN3ud5AAAAI0XlEgAAwOC+Kemm9MDMzom/b3D35939Nkn7Jf3cMa7xLUlnmNnG+PjXJT3cdc4Tki6KO9ytkPThhfoCAAAA5otwCQAAYHA3S9pmZs+Z2UuSfjM+f4uZvWBm/6Uwb+kbs13A3d+WdJ2kr5nZ8wo71d3Zdc73Jd0q6XFJ/yHp6YX+QgAAAAZl7lRYAwAAAAAAYDBULgEAAAAAAGBghEsAAAAAAAAYGOESAAAAAAAABka4BAAAAAAAgIERLgEAAAAAAGBghEsAAAAAAAAYGOESAAAAAAAABka4BAAAAAAAgIH9P0B//IMUf2tVAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "inspector.visualize_auc(fpr,tpr,thresholds)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "You can also plot a heatmap of the confusion matrix and use the threshold slider to get a sense for the trade-offs at various thresholds." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "70ad54ff76b14bc1b67935dbff30f01f", "version_major": 2, "version_minor": 0 }, "text/plain": [ "interactive(children=(FloatSlider(value=0.5, description='Threshold', max=1.0, step=0.05), Output()), _dom_cla…" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "\n", "inspector.display_interactive_cm()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "#### Global Feature Importance\n", "\n", "The blueprint will run an Amazon Clarify job to obtain Shapley values. You can view the global feature importance of your models by navigating to the \"Model Explainability\" reports as shown below.\n", "\n", "\n", "![Global Shap Report](img/global-shap-report.gif)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "The Amazon SageMaker Clarify process produces Shap values for each of records used in your automl job. This notebook provides you with utilities to visualize feature impact of single predictions.\n", "\n", "The following utility function will load a sample of your prep dataset for a specific blueprint run. Run the cell to preview your data." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
agejobmaritaleducationdefaulthousingloancontactmonthday_of_week...campaignpdayspreviouspoutcomeemp.var.ratecons.price.idxcons.conf.idxeuribor3mnr.employedtarget
056housemaidmarriedbasic.4ynononotelephonemaymon...19990nonexistent1.193.994-36.44.8575191.00.0
157servicesmarriedhigh.schoolNaNnonotelephonemaymon...19990nonexistent1.193.994-36.44.8575191.00.0
237servicesmarriedhigh.schoolnoyesnotelephonemaymon...19990nonexistent1.193.994-36.44.8575191.00.0
340admin.marriedbasic.6ynononotelephonemaymon...19990nonexistent1.193.994-36.44.8575191.00.0
456servicesmarriedhigh.schoolnonoyestelephonemaymon...19990nonexistent1.193.994-36.44.8575191.00.0
..................................................................
758673retiredmarriedprofessional.coursenoyesnocellularnovfri...19990nonexistent-1.194.767-50.81.0284963.61.0
758746blue-collarmarriedprofessional.coursenononocellularnovfri...19990nonexistent-1.194.767-50.81.0284963.60.0
758856retiredmarrieduniversity.degreenoyesnocellularnovfri...29990nonexistent-1.194.767-50.81.0284963.60.0
758944technicianmarriedprofessional.coursenononocellularnovfri...19990nonexistent-1.194.767-50.81.0284963.61.0
759074retiredmarriedprofessional.coursenoyesnocellularnovfri...39991failure-1.194.767-50.81.0284963.60.0
\n", "

41188 rows × 21 columns

\n", "
" ], "text/plain": [ " age job marital education default housing loan \\\n", "0 56 housemaid married basic.4y no no no \n", "1 57 services married high.school NaN no no \n", "2 37 services married high.school no yes no \n", "3 40 admin. married basic.6y no no no \n", "4 56 services married high.school no no yes \n", "... ... ... ... ... ... ... ... \n", "7586 73 retired married professional.course no yes no \n", "7587 46 blue-collar married professional.course no no no \n", "7588 56 retired married university.degree no yes no \n", "7589 44 technician married professional.course no no no \n", "7590 74 retired married professional.course no yes no \n", "\n", " contact month day_of_week ... campaign pdays previous \\\n", "0 telephone may mon ... 1 999 0 \n", "1 telephone may mon ... 1 999 0 \n", "2 telephone may mon ... 1 999 0 \n", "3 telephone may mon ... 1 999 0 \n", "4 telephone may mon ... 1 999 0 \n", "... ... ... ... ... ... ... ... \n", "7586 cellular nov fri ... 1 999 0 \n", "7587 cellular nov fri ... 1 999 0 \n", "7588 cellular nov fri ... 2 999 0 \n", "7589 cellular nov fri ... 1 999 0 \n", "7590 cellular nov fri ... 3 999 1 \n", "\n", " poutcome emp.var.rate cons.price.idx cons.conf.idx euribor3m \\\n", "0 nonexistent 1.1 93.994 -36.4 4.857 \n", "1 nonexistent 1.1 93.994 -36.4 4.857 \n", "2 nonexistent 1.1 93.994 -36.4 4.857 \n", "3 nonexistent 1.1 93.994 -36.4 4.857 \n", "4 nonexistent 1.1 93.994 -36.4 4.857 \n", "... ... ... ... ... ... \n", "7586 nonexistent -1.1 94.767 -50.8 1.028 \n", "7587 nonexistent -1.1 94.767 -50.8 1.028 \n", "7588 nonexistent -1.1 94.767 -50.8 1.028 \n", "7589 nonexistent -1.1 94.767 -50.8 1.028 \n", "7590 failure -1.1 94.767 -50.8 1.028 \n", "\n", " nr.employed target \n", "0 5191.0 0.0 \n", "1 5191.0 0.0 \n", "2 5191.0 0.0 \n", "3 5191.0 0.0 \n", "4 5191.0 0.0 \n", "... ... ... \n", "7586 4963.6 1.0 \n", "7587 4963.6 0.0 \n", "7588 4963.6 0.0 \n", "7589 4963.6 1.0 \n", "7590 4963.6 0.0 \n", "\n", "[41188 rows x 21 columns]" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "df = bprunner.get_prepped_data_df(execution_arn)\n", "display(df)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "Now, you can use your model inspector to get the feature impact for a specific prediction. The cell below shows the features that influence the prediction for row number 31614. You can change the row id to obtain predictions for other records in your dataset." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABGoAAAEACAYAAADr4JCdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdd5hbV5248fdqNKPpM+7xxL3bKTg9IYlJnIQUpxNY0wNLyNL3R1mHALuBhQSzC2QXdhMIJUAAsySkkV6d3nuxHdfYcW/TPF3394c0M7LjotFIHo39fvzo8Ui6ujr3e+6RdL/3nHODMAyRJEmSJElS34v0dQEkSZIkSZKUYKJGkiRJkiQpT5iokSRJkiRJyhMmaiRJkiRJkvKEiRpJkiRJkqQ8YaJGkiRJkiQpT5iokSRJkiRJyhMmaiRJkiRJkvKEiRpJkiRJkqQ8YaJGkiRJkiQpT5iokSRJ+6fK2VV9XQRJkqQdmaiRJEndKmcvp3L2x3bx3OVUzr69B+u6nsrZv8pW0XLgVipnfy+TF46eu/rh0XNXfzvbBZIkSYr2dQEkSVI/UTfvyr4uQpZ9GniYytlQN+9f+7owkiRJYKJGkiT1F5WzC6mb19bD15wEPLSHpb5D5ew7qZv3VKZF6wuj564uXDGnpmfxkCRJec9EjSRJ2tEoKmc/ABwDLAc+S928J6icfQVwAnXzTgWgcvYBwHXADGAdMBf4FTCWunnLk+uKUTn7OuCDQCPwPerm/aLrnSpnnwhcBUwDtgD/C/yEunlhMslyP/Ap4LvAEKCih9vyePJ1OzMcuAO4HXi6h+vtMnru6kOBq4HDSGzDb4CrVsyp6Rg9d/XPgKIVc2ouTS77KDBqxZya0cn7c4AZK+bUzErePx/4DjAeWAN8f8Wcmj8mn7sY+DbwC+ArQC1wUKblliRJ+clEjSRJ2tGngfOABcB/Ar8DJu5kuT8CW4GRQDHwfztZ5iLgH4BLgfOBv1A5+27q5q2gcvZBwJ3Ax4C/J9/jLmAD8Pvk6wuAM0kkQXreeyTRA2fjTp+rnP1X4Dbq5n2xx+tNGj13dRVwH/DzZDnHkUj+tAD/QSLR9NPksuXAdGDL6LmrJ62YU7MIOJVEDBg9d/VpwK9JxOlx4EjgntFzV69cMafmkeRbjgFqSMQqyLTckiQpf5mokSRJO/oFdfNeB0hOBvzP77pCUuXsEcBMYDx18+qAOipn/zvwvh3W9SB1825L/v03KmdvJZGsWAF8DvgrdfNuTT6/gMrZPwc+QXeiBuAy6ubV9maDRs9dHQDlK+bU1Kc8/GHq5q3tzXqBWUAriZ4vIfDm6Lmr5wJfJZGoeQgYOXru6nHAVOBZ4C3gtNFzV68Ajk8uC4leMv+1Yk7No8n7z4yeu/oGEvHoTNS0AZetmFPT0styS5KkPGWiRpIk7WhNyt+Nyf93HHJ0YPL/t1MeW7GHdXWur3NdY4GZVM6+MOX5CLAy5X58h/s9lkzS/Dq57ou7nuh9kgYSvYmWJ5M0nZYkH2fFnJq60XNXP0ei58xUEr1vFgMfJdFjqW7FnJpXk68bC5w8eu7qr6asqwB4NOX+GpM0kiTt20zUSJKkTLyT/H8UsDTl755YAfyGunlf2M0yIXXzwt08v1spSZojSfQAyraVwOjRc1cHKcmacWyfXLqf7kTNp4BlwC+BRcADKcutAK5fMafmP3bzfvFsFVySJOUnEzWSJKnn6uatonL2w8APqZz9j0AJiYlue+J/gflUzr4buBsIgUnAEOrmzc9SST9AIjkCsGH03NW7Wm7iijk1izNY/x0kJhK+fPTc1f9BolfMHBIT/na6n8SwplbghRVzauKj565eRmLenn9OWe5q4Lej565+CniCRG+aQ4BgxZya5zIomyRJ6ocifV0ASZLUb30EKAVWAY8Bf00+nt7QnLp5rwFnk0hWrAHWA9ez66s0ZeJW4GbgVRI9XYbs4rZ0VyvYnRVzamqB95PoMbMOuIfE/Do/SVnsSRK/uR5cMaems0fM/UBl8v/Odd0LfJbE3DYbScTkp0B5JmWTJEn9UxCGGfcmliRJ6lY5+3QSiZGS3gxXyrbRc1dHgXlA/Yo5NZ/a0/KSJEl9yUSNJEnKTOXs95AYrvQqiSE/fwHeoG7eJ/u0XDuRTNZUr5hTs/NLdUuSJOUJ56iRJEmZGghcBwwHaoG7gK/1aYl2YcWcmnYSw4kkSZLymj1qJEmSJEmS8oSTCUuSJEmSJOUJhz5JkqQdrQWG9XUh+oF1wAF9XQhJkrRvceiTJEnakT8O0hf0dQEkSdK+xaFPkiRJkiRJecJEjSRJkiRJUp4wUSNJkvbo6quvZunSpX1djD3auHEj1157LVdeeSVPP/00S5YsYd68eWm99rrrrmP9+vU5LqEkSdLumaiRJEn7jMcff5wxY8Zw+eWXc8wxx/DAAw9wwgknpPXa9773vTz00EM5LqEkSdLumaiRJEn7jNraWoYOHQrAO++8Q3NzMyNGjEjrtZMnT2b58uXU19fnsoiSJEm75eW5JUlSWt555x3uuusu6uvrmTJlCmeffTbRaJSmpiZuvvlmVq1aRTweZ9SoUZx99tlUVlYC8NJLLzF//nwaGxspLS1l5syZHHrooQC8+OKLPP744zQ0NHDggQdyzjnnUF1d/a73vuGGG5g0aRJHH330Lsv3u9/9juXLl/P2229z9913c/zxxzNmzJiu51euXMmf//xnLr30Uqqqqli7di3XX389n/nMZxg8eDDRaJThw4ezZMkSpk+fnt3gSZIkpangiiuu6OsySJKk/HLFjg889dRTbNq0iY997GMcf/zxPPPMM9TV1TF27Fja2tooLi5m1qxZHHvssbz11lssWrSIgw8+mNbWVn7/+99z8cUXc/rpp3PQQQdRXFxMWVkZCxYs4KGHHmL27Nmcdtpp1NXV8dhjj3H44Ye/q0BhGPLSSy/t9LlO06dPZ/ny5Zxwwgl86EMf4plnnmHYsGGMGjUKgKqqKpqbm3nmmWc46KCD+NOf/sTxxx/PxIkTu9axevVqtm3bxoQJE9KN1XfTXVCSJCkdDn2SJElpOfroo6mqqqKkpIQZM2bw6quvAlBaWsq0adMoLCwkFosxY8YMli9f3vW6IAhYv349bW1tVFRUdA1Neu655zjhhBMYMmQIkUiEE088kbVr17J169Z3vfeUKVN2+dyuNDc3E4vFtnvspJNOorm5meuuu46KigqOOuqo7Z4vKiqiubk57feQJEnKNoc+SZKktHQOZYJE75TOuVza2tq4++67Wbx4cVeSo6WlhXg8TlFRERdddBFPPPEEt912GyNHjuT0009n8ODB1NbWcvfdd3Pvvfd2rTcMQ+rr6981/CkWizFp0iRee+21tCcHLikpoaWlZbvHCgoKmD59OnfddRenn346QRBs93xrayvFxcXpB0WSJCnLTNRIkqS01NXVdf1dW1tLRUUFAE888QSbNm3ikksuoby8nLVr13Lttdd2LTthwgQmTJhAW1sbDz74ILfddhuf/vSnqays5MQTT+yar2ZPDj74YObPn8/o0aMZOXLkHpcfNmwYmzZtetc2zJ8/n8MOO4x7772XSy65hGi0++fQhg0b0i6PJElSLjj0SZIkpaVzXpqmpiYeffRRDj74YCDRCyUajVJcXExTUxMPP/xw12saGhpYuHBh1zJFRUVEIomfH0ceeSSPPfYY69evBxJDlV5//fVdvv/EiRPZunVr2pfQnjhxIitWrOi6H4Yht9xyC4cddhjnnnsu5eXl262rvb2dNWvWMH78+LRjIkmSlG32qJEkSWk55JBD+MMf/kB9fT2TJ09mxowZABx77LHcdNNN/OhHP6KiooLjjjuOBQsWAInkyBNPPMHf/vY3giDggAMOYNasWQBMnTqV1tZWbrzxRmpra4nFYowfP56DDjpop+8fjUaZOnUqL774YlrlHT58OLFYjFWrVjFixAiefvppGhsbmTlzJkEQcP7553PNNdcwadIkRo8ezcKFCxkzZkxXTyFJkqS+EIRh2NdlkCRJ+WWf+XGwZMkSnn32WWbPnr3HZa+77jrOO++8rsmO0xTseRFJkqT0maiRJEk78sdB+kzUSJKkrHKOGkmSJEmSpDxhokaSJEmSJClPmKiRJEmSJEnKEyZqJEmSMhcmb2v7uiCSJGnf4GTCkiRpR/44yIwTC0uSpF6zR40kSdrRur4ugCRJ0v7KHjWSJClf9bcfKfaokSRJvWaPGkmSJEmSpDxhokaSJPU78+bN4/DDD6ekpISBAwdy0UUXsXjx4j2+7mc/+xnTpk0jFosxdOhQPvWpT7Fu3fYjvb785S9z+OGHU1hYSBAEBIEdZSRJ0t7j0CdJkpSvdvoj5de//jWf+cxnABg7diybNm2irq6OoUOH8vLLL3PAAQfsdGXf+c53+P73vw/AxIkTWbVqFU1NTUyePJkXXniB0tJSAKqrqykoKCAWi7FmzZpEQdL7vWRGR5Ik9Zo9aiRJUr/R2trKZZddBsAHPvABli5dyptvvklFRQXr16/nyiuv3Onr1q1bx9y5cwH42te+xqJFi3jqqacIgoCFCxdy7bXXdi37yiuvsGnTJj7ykY/kfoMkSZJ2YKJGkiT1G88++ywbN24EEokagJqaGo499lgA7rnnnp2+7v7776etrW271x166KFMmDDhXa8bNWpUbgovSZKUBhM1kiSp31i5cmXX30OHDu36e9iwYQC8/fbbWX2dJEnS3maiRpIk9Ru7mitmT3PIZPo6SZKkvc1EjSRJ6jdShyWtX7/+XX+PHDkyq6+TJEna20zUSJKkfuOoo45i0KBBANx0000ArF69mqeeegqAM844A4ApU6YwZcoUfv7znwNwyimnEI1Gt3vdK6+80nVJ787XSZIk9TUvzy1JkvLVTn+k/PKXv+TSSy8Ftr889+DBg3n55ZepqakhCBJXyv63f/s3rrjiCgAuv/xyrrrqKgAmTZrEypUraWpqYuLEibz44ouUlZUBcNJJJ7Fq1So2bdrE1q1bARg/fjwAf/zjHznmmGN2VV4vzy1JknrNHjWSJKlf+exnP8sNN9zA9OnTWb16NUEQcMEFF/D4449TU1Ozy9f94Ac/4Oqrr2bKlCksW7aMsrIyPvGJTzB//vyuJA3A8uXLWbJkSVeSBmDJkiUsWbKEpqamnG6bJEmSPWokSVK+6m8/UuxRI0mSes0eNZIkSZIkSXnCRI0kSZIkSVKeMFEjSZIkSZKUJ0zUSJIkSZIk5QkTNZIkSZIkSXnCRI0kSZIkSVKeMFEjSZLy1bq+LkAP9KeySpKkPLbPJGqCIDgjCIKFQRAsDoLgsp08HwuC4C/J558OgmBMynPfTD6+MAiC05OPFQdB8EwQBC8HQfB6EATfTVl+bHIdbyXXWbQ3tnFfs5fr7PogCJYFQfBS8jZ9b2zjvibbdZbyXEEQBC8GQfD3lMdsZ720l+vLNpYFuaizIAiWB0HwarJenkt5fGAQBPcl29h9QRAMyPX2ZeAAIKivr6e+vh4g6O0tm+sCgiAIzgyCYFEQBA17oc6uCILgnZR2dlbmod1/5aidVQdBcGMQBAuCIHgzCILjko/3h3aW9/ZyndnOeinb9RUEweSU+ngpCIK6IAj+OfmcbSwL9nKd2cbSEYZhv78BBcASYBxQBLwMTNthmc8D1yb/ng38Jfn3tOTyMWBscj0FJH6AlSeXKQSeBo5N3v8/YHby72uBz/V1DPrbrQ/q7Hrgor7e7v58y0Wdpbzuq8CfgL+nPGY761/1ZRvL0zoDlgODd/J+PwIuS/59GTC3r2Owq1tdXV1YV1cX5tu6+qDOrgC+3tf10Z9vOayz3wGfSf5dBFQn/+437Sxfb31QZ7azPKyvHda/FhidvG8b6391ZhtL47av9Kg5GlgchuHSMAxbgXnAeTsscx6JD2SAG4FTgiAIko/PC8OwJQzDZcBi4OgwoSG5fGHyFiZfMzO5DpLrPD9XG7YP22t1lusN2Y9kvc4AgiAYAcwCftW5EttZVuy1+lLW5KTOdiN1XbaxzOztOlPvZb3OgiCoBGYAvwYIw7A1DMOtO1mX7Swze7vO1Du5/lw8BVgShuGKnazLNpaZvV1nSsO+kqg5EFiZcn9V8rGdLhOGYTtQCwza3WuDRPf+l4D1wH1hGD6dfM3W5Dp29V7as71ZZ51+EATBK0EQ/DQIglg2N2Y/kZM6A64G/gWIpzxvO+u9vVlfnWxjvZOrOguBe4MgeD4Igs+mLDMsDMM1yXWtAYZmaTv2J3u7zgC+mGxnv7GLf0ZyUWfjgA3Ab4PEsNBfBUFQllzGdtZ7e7vOwHbWG7n6XOw0G/hzyn3bWO/t7ToD29ge7SuJmmAnj+3Yk2JXy+zytWEYdoRhOB0YQSL7fnCa76U925t1BvBNYApwFDAQmJNJofdzWa+zIAjOBtaHYfh8Bu+l3dub9QW2sWzIyecicHwYhocDZwJfCIJgRuZF1A72dp1dA4wHpgNrgB/3uMTKRZ1FgcOBa8IwPAxoJDEEQ9mxt+vMdtY7ufpcJEjMV3gu8NeMS6ed2dt1ZhtLw76SqFkFjEy5PwJYvatlgiCIAlXA5nRem+wK+TBwBrARqE6uY1fvpT3bm3VGGIZrkkOjWoDfYvfyTOSizo4Hzg2CYDmJbpYzgyC4AdtZNuzN+rKNZUdOPhfDMOz8fz1wM911sy4IguHJdQ0n0RNRPbNX6ywMw3XJExJx4DpsZ5nIRZ2tAlal9OK9kUQSAGxn2bBX68x21mu5/I1/JvBCGIapV9mzjfXeXq0z21h69pVEzbPAxCBxlZgiEt2rbtthmduATyb/vgh4MAzDMPn47ORM1mOBicAzQRAMCYKgGiAIghLgVGBB8jUPJddBcp235nDb9lV7rc6S9zs/wAMSY1dfy+nW7ZuyXmdhGH4zDMMRYRiOSa7vwTAMP2Y7y4q9Vl9gG8uSXHwulgVBUAGQ7Nb/frrrJnVdtrHM7NU662xnSRdgO8tELj4b1wIrgyCYnHzNKcAbO1mX7Swze7XObGe9lvX6Snndh3n3EBrbWO/t1TqzjaUpzIMZjbNxA84CFpGYafpbyce+B5yb/LuYRJerxSR2nnEpr/1W8nULgTOTjx0KvAi8QmLn+deU5ccl17E4uc5YX29/f7zt5Tp7EHg1+fgNJK8O5a1v62yHdZ/E9lcRsp31r/qyjeVhnSXb0cvJ2+ud60w+Nwh4AHgr+f/Avt7+Xd3y9apPfVBnf0i2s1dI/Dge3td10x9vufhsJNGF/7lk3dwCDEg+3m/aWT7f9nKd2c7ys75KgU1A1Q7vZRvrf3VmG0vjFiSDJUmSlJfq6+tDgIqKip2Nhe+zdUmSJOXCvjL0SZIkSZIkqd8zUSNJkiRJkpQnTNRIkiRJkiTlCRM1kiRJkiRJecJEjSRJkiRJUp4wUQMEQfDZvi6DesY663+ss/7HOut/rLP+xzrrX6yv/sc663+ss/7HOss+EzUJ7lj9j3XW/1hn/Y911v9YZ/2Pdda/WF/9j3XW/1hn/Y91lmUmaiRJkiRJkvJEEIbhLp8844wzwo0bN+7F4vSNDRs2MGTIkL4uhnrAOut/rLP+xzrrf/bVOovH4wBEIr0/v5TNdWXDvlpn+yrrq/+xzvof66z/sc4y8/zzz98ThuEZO3tut4kaYLdPSpIk5Vp9fT0AFRUVebUuSZKkXgh29UR+nE6SJEmSJEmSiRpJkiRJkqR8YaJGkiRJkiQpT5iokSRJkiRJyhPRvi6AJEnS7jjxryRJ2p/Yo0aSJEmSJClPmKiRJEmSJEnKEyZqJEmSJEmS8oSJGkmSJEmSpDxhoiZXtrVAGPZ1KSRJknqltT0k7m+aHouHIS3txi0Tze0hoftcj7V1hHTEjVtPhWFIs201Iy221Zzxqk/ZVtsIb6yEjjj87UlYvKavSyRJkjr98SsAfGTexj4uSP/x4feUMbQswgurW3l2VasHNGk69IBCzpxUwvKtHTyyrJk19R19XaR+IRaFLxxbSVNbyBMrmnllbRvucek5Z0oJEwcX8sqaVp5a2UJDq5FLx9gBUf7h0DJW17Uzf1kzK7baVtMRCeBzx1QQjcC4gVEGlUWIBEFfF2ufYaIm27Y0JJI0AJNq4Ff39W15JEnSuzy+orWvi9AvjBsYZXR14ufiMSNjXPdsI5u2xfu4VP3D7EPLCIKAsQOiPLTUfS5dFx5UQlFBQFFBwNShRfzv0419XaR+YXBphG+cWEhBJODIETFueGkbSza393Wx+oXTJ5YAUFMZpaQwwuMrmvq4RP3DSWNjVMQSA3Q2NMYZVOZgnWwymtk2fCB0JhIPHg0Th/dpcSRJkjJ13tSSrr8fX9FikiZN04cXMmZAIsHV1BZy76LmPi5R/1AQwNlTuve5W9/0gDlds6aUUBBJHIS8urbVJE2aRlUXcFhNEQAd8ZDb3efSdt607rY6qNTeNNlmoibbYoUwuKr7/vnH9l1ZJEmSMjSwJMKMsbGu+7e84QFMus6fVtr19/2Lm2hscwhKOo4fHWNIWQEAW5rizF9qgisdxVE4Y1Jx133bavpSk9FPvt3K+kaT0ek4eFghEwYVAok+CvamyT4jmgsHDuz++8RpMLiy78oiSZKUgVlTSogmz9C/tq6VxZs8Q5+OUVUFHHFg4gx9PAy5zTP0aTs/5Qz9nQubaPOYOS2nTiimvChxWLe6rp1nVznMLh3VxQEnj0tNcG3rw9L0L6m9aQaURrq+K5Q9JmpyobwEKpNnUqIFcM5RfVseSZKkHohF4cyUM/S3eoY+beemHMA89XYraxvMNqRj2tBCJg5OnKFv7Qi5c6H7XDoiAZw7tbsH161vNjn5cprOmlxCYUEiwbBgQxsLN5qMTkdNRQFHjyjquj/Y3jQ5YVRz5cBB3X+feQQUF/ZdWSRJknrglPHFXZNErq7r4BnP0Kelaocz9Le+6Rn6dKX2pnlwSTN1LaYb0nH0iCKGVySGi9W3xHlgicPF0lFUkEjUdLI3TfrOnVrSNR9NRSygOGpvmlwwUZMrA8uhOJlpLC+G06b3bXkkSZLSELD9vA23vbmNuMfMaTlzUuKKRQCLNrbxxnrP0KdjeEWEY0Z2n6F3uFj6UudDumtRMy3ucmk5aVwxVcWJQ+F1DR08+bbJ6HSUFwWcOqE7GT3E3jQ5Y2RzJQi2n6vm/GMSfRMlSZLy2FEjiqipTFyxqMEz9GkrjOx4ht5kQ7rOmVradYb+uVUtrKzt6OMS9Q8TB0U5aFii135bR8gdC9zn0rFjMvr2N5tMRqfpjEnFxJI9aIqjUFbk8W2umKjJpaHVEE2GePhAOGZS35ZHkiRpD1KHoNz9VjPNnqFPy/vGFTOgJPG7b0NjB0+saOnjEvUPZUUBp41PmdDV3jRpS53Q9dHlLWxucj6kdBxeU8So6kQyeltrnHsXm4xORzQCZ0/p3ueGlBUQeEnunDFRk0sFEThgQPf9C47ru7JIkiTtwfiBUQ45IDEEpT0e8nfP0Kft/B3O0Hd4hj4tZ0wsprgwcbC3bEs7L69p6+MS9Q9DyiKcMDrWdf9WE1xpS01w3fNWM01tNtZ0nDgmxqDSxHxI0QhUlZikySUTNbk2fGCifx3AwaNgYk2fFkeSJGlXUg9gHlvewqZtnqFPx2HDCxk9IHGGvqkt5N63PEOfjoJg+zP0tzqha9rOnlJCQXJahVfWtrJ0s13f0jGmuoDDahLJ6A6T0T2S+v0wuCzSNVxRuWGiJtdihTCosvv+eUf3XVkkSZJ2obo44MQx3WfoPYBJX2qy4aGlzTR6hj4tx42KMbgscYa+riXOI8scLpaOogJ4/8Tu4WK21fTNSmmrz6xqZX2jyeh0TBsaZfzA7qsYDyw1jZBrRjjXwhCaUmYRX7mx78oiSZK0Cw2tIbXN3QctNZUFfVia/uWduu7Jb41b+t6p6+4FUlYYUF3ioUk6WjtgQ0N3Wz3QfS5tqW3VuKVvbX2c9pQZl1vbTUbnmp+GuVa7DRqT3V+b2+Cu5/u2PJIkSTvRHt/+zHzqZX+1e7cvaKIjeRAzfXgRYwZ4AJiOZVs6eHlN4oRmQSTgnJTeDtq9W1KGiZ09paTr+iXavXvfamZbWyLJNao6ymE1hXt4hQA2N8V5NKXH20Z7IuWcTTrXVm/q/vuBl6HOromSJCk/3b2omebksJ1xA6MceoAHMenY0Bjn8ZSrPJ0/1SRXulIvY/7+icWUFDrvRToeWd7C5m2J3iGDSgu2m1hYu7atLeS+lDmkTEinL3XC6q3NIa3OmJ5TJmpyqakFNjd037/l6b4riyRJ0h40tIbcvyT1IMYeDulKPYiZMTbWdalu7d7z77SyqjYxBKqsKMJpE4r38ApBogfcHQttq5lI7QGXuFS3PeDSsWRzO6+s7Z7SY5O9anLKb5Bcemdz999PL4J3Nu16WUmSpDxw25tNxMPEQcxRI2KMcB6HtCza2M4b6xOXli4sCJg12YRDOkLg1pReNedOKSFip5q03L2oiZbkXCHjBxVy8DB7wKVjXUOcJ9/uTjicN9UkV7pSe8Bt2hbvSngp+0zU5EpbO6zf2n3/lqf6riySJElpWlPfwTOrug9izvUgJm2pl5c+c3IJMXNcaXloaTN1LYmz88MqCjh2ZFEfl6h/qGsJedAecBm59c3utnryuGKqi80OpuO5Va1dk4DHQ9jSZK+aXDFRkytrtyb2XoAla+Hl5X1aHEmSpHSl9nCYOb6YypgHMel4amUra+sT84ZUxiKcPM5eNelo6YC7Fnbvc+c5b0jabksZcnfUiCJqKswOpmPBhnYWbOjuAXfmZJNc6QjZfp/b2BgnDO1VkwsmanIhHsKalGFPN9ubRpIk9R+vrWtj8abEQUwsGnDGJA9i0hEPtz+IOW9aCaa40nPHwmbakpOTThtayOTB0T4uUf+wqq6DZ1clJrKOBIE94HogNSF91qQSisxxpeWBJc3UJ3vAtXYkenYp+0zU5MLGWmhNdAljUz088lrflkeSJKmHUucimDW52Mv/pun+xc00tiYOYkZURTniQIfxpGNLU5xHUi7/e57DeNKW2lZPnVBMeZHpwXQ88XYL6xsSPeCqS//GycQAACAASURBVCKcNNYecOloaYe7FnUPufNS3bnhV262heH2kwjf/mxiWnZJkqR+5PEVLWxsTBzEDCwtYMYYL/+bjqb2kHvfct6QTNySMm/Ie0fFGFrmoUo6XlnbxtLNiZPEiR5wJhzSsbMecErPHQuaunrANbaGbGv1eDfb/PTLttpt0Jj8cm5ug7ue79vySJIkZaA9Dn9f0H0Qc77zhqQt9fK/7xlexNgBjqlIx/ItHby0JjGRdUEk4OwpHjin65aUiazPnlJiD7g03be4uSvJMKo6yuE1XjkrHZub4jy6vLsHnL1qss8mnG0barv/vv8lqG/a9bKSJEl57O63mmluSyQcRg8oYIwJh7RsaIzz+Irug5iTnFQ4bZ3DeBpa45w4pshLdafp0eUtbNqW6AE3qLSAQw8w4ZCObW0h9y7u7gFnW03frSm9kepaQi/VnWXBHmZpNto9FYawYBX86G9w+Hhobt3zayRJ0l5R/0+nAXD9q31ckH5k7IAo0Qjc8FIjx4x0+FO6KmIBYwdE+f0LjUyvcZ6anjjkgEIeWNxMWzzkwEonFU7XiMoCyooCbnixkaNsq2mLRQOmDink9y80cOhw22pPTB1SSBDAi6tb+fqMyr4uTn+0y1S0n3zZFgQwdWTi/4k1fV0aSZKUomLZJpb+/kmm/dMFfV2UfmdASYRpQz1L3xPtcSgtCoxbD3XE4dhRMR5Z1kJVcYQDK+3J1RNVttUea+0IKS60rWaivCigwHE6WWeiJleCACLusZIk5ZUg8Cs6Q8YtM8YtM6VFAceOKuKJFS1UFMeoLjaI6XKfy4xxy0zg8MScMFGTK5EACtxrJUnKK0Gin7G/xXvOuGXGuGVuYEmE6cMLeebtVk4eFyMW9bd1OtznMmPcMuM8UrlhoiaXTC9KkpRfOnvU+BXdYwHGLRPGrXdGVUepawl5ZlUrJ44uImIw98h9LjPGLTOBx7w5YaImV/wVKElS/okEBARE/GHZY0Fg3DJh3HrvkGGFPPl2Ky+vbefwmkIPDPfAfS4zxi0zHvLmhomaXHGQoyRJ+SfZo8bf4j0XYNwyYdx6LwgCjh5ZxENLW1i2pYPxgzyE2R33ucwYt8wYstzwUy5X7DsnSVL+CQK/ojNkZ+HMGLfsiEUDThhdxINLW6gqDhha7pWgdsV9LjPGLTPGLDdM1OSKLV2SpPzjHDUZM8GVGeOWPZXFEY4bVcTTK1uZOT5GeZG913fGfS4zxi0z9kLKDRM1ueLQJ0mS8k/EHjWZMsGVGeOWXcMrCjhoaJQnVrRyyvgYhV5l9V3c5zJj3DJjzHLDRE2u+CtQkqT8EyRuTkaaGeOWGeOWXRMHF7K1OeSZVW0cP7rI+O6EMcmMces5Y5YbJmpyxZkKJUnKP85RkzHPNmfGuOVAEHDUiMTkwq+va+PQ4UV9XaK84j6XGeOWGUOWGyZqcsWhT5Ik5R/nqMmYCa7MGLfciBQEzBgT4563mhlY2s6oag9rOrnPZca4ZcaY5YafaLliS5ckKf/YoyZjJrgyY9xyp7Qo4KRxMR5Y0kxlLMLAUk+SgvtcpoxbZhxEkhsmanLFli5JUv6JBI5OzlCAccuEccutgaURjh5RxCPLWzhjUjElhQbbfS4zxi0zpkdzw0RNrjj0SZKk/NPZo6avy9EPGbfMGLfcGzsgSl1znMeWt3Dq+BgF+/nJUve5zBi3zJjcyg0TNbliv2pJkvJP8opPEb+je8y4Zca47R3Ta4qYv6yF51a3cezI/ftKUO5zmTFumYnsx20tl0zU5IpDnyRJyj/JH5R+Q2fGuGXGuOVeEAQcPzrG3YuaWbihnalDC/u6SH3KfS4zxq3nzNPkhomaXLFHjSRJ+cfJhDPmOajMGLe9pzgacMr4GHctamZASYSayoK+LlKfcJ/LjHHLjDHLDRM1ueIcNZIk5Z+Il+fOlAmuzBi3vauqOMJJY2M8vKyFsyYVU1m8//0ed5/LjHHLjCHLDRM1OeMlJSRJyjtdPWr8ju6pRILLuPWUcdv7aiqjHDw0zitr25gxtrivi7PXuc9lxrhlxpDlhomaXDElK0lS/gnsUZMpf9pkxrj1jeqSCJua4vtl7N3nMmPcMmNyKzdM1OSKQ58kSco/QeLHuL8rey6ws3BGjFsf2Y/buvtcZoxbZoxZbpioyRVP10mSlH+coyZjnm3OjHHrGxH235/j7nOZMW6ZsWtCbpioyRVbuiRJ+ScICAjsqp2BIDBumTBufSOSjPv+GHv3ucwYt8wYstwwUZMrDn2SJCn/OEdNxjwHlRnj1jeCYP+N/f663b1l3DJjzHLDRE2u+CtQkqT8kzz15xnADDh/Q2aMW5/onG9kv4z9/rrdvWXcMmLMcsNETa7sr7OXSZKUz+xRkzHPNmfGuPWNSLD/njd1n8uMccuMw8Vyw0RNrgQBFDj0SZK0j/ngXPiHE+Gi9+699/zLo3DL0/Dnr/d+XZH8/jH+g4fqGFoW4ZKjyzN6/eE/W8ctHx/EqOrs/8TbXw96e2t/idvx167nLx8eyIiq/Di8iPTB0Kdctr+e6E/73Bdv3cLpk4o5Z2pJXxelX8Utnxiy3MiPT9J9kS1dkrQv6ouJHzrP1mXjPfO8R813ZlZ2/f3sqla+dU8t9/7jkB6tI5Kj7cvHBNc1TzWwsraDK0+v6vW6/vGmzcyaXMyFB5dmoWTd8jFuufDk54b2dRG20xn3vR37vnjPHfWnfe5/zx/Q10Xo0p/ilk+MWW6YqMkVhz5JknKhvQOiBX1bhr6Y+CFb36tBkFhVHp4D7IiHFKT84g26/u9ZWYPkv2zLx7hlGqNdrSsXscvHuO0POmO+t2Ofq/bXszLsve1uj4dE95EjddtqZjzkzQ0TNbniVZ8kST1x3Dfgk6fATU/AO5vgfQfDTz8DLy6BL18HnzoFfnUvnHgQ/Ncl2782Hof/uRPmPQr1TXDCNLjy41BdBsvWwYxvwo8/Df9xMzS1wuUXwdQR8I3rYc1muOh4+O5HEuv68yNw4xMw+UC45SkYVg0/+Bi8d2ri+c6ESSSSeN///nvifZtb4eRD4XsfgYoS+OiP4fTD4RMnd5dz5rfg8g/CqdNh0Tvwb3+CV9+GwRXwLxfCWUcmlttUB1/7DTy9CCbWwPFTE2+cje/VZKKmN8cVp/9mPbPfU8btbzaxqraDMyYV8+X3lvPt+2p5cXUbhxxQyI/PqqaqOMJX79jCC6vbaGkPmTQ4yndmVjJhUCEA37p3K8XRgNV1HTz/Thv/dU41f1/QxLDyAj5zVBlfuHULrR1w7DXrAPj7JwazrqGDH86vZ9nmdmLRgFMnFPMvMyooLOjeoJz1qMnCetfWd/DD+XW8sLqVeAhnTirhmydVcN2zjdz02jZa2uH40UV886RKKmIR3qlr54zfbuT7p1Xx8yfraW4P+fhhZXz26HIeW97Cr55rBOChpesYWVXATR8dzM2vb+O3zzeyriHOgJIInz6yjA8d0t1L5sElzfzvUw2squtgQEmEb51UyQurW3lhdRuvrG3jR4/Wc97UEr51cuWuNqNHstWDa2ex+8RhpVzxQB0LN7YRAO8dHeNbJ1dSGUu0lZ7sq52x/teZlVzzdANhCBcfUcYnDy8D4NW1rbvd9w75r7Xc8cnBjKqOsrUpzrfvq+W5d1oZU13Ae0fHeG5VK7//0KCuZb9zciW/e6GRLc1xzppcwrdOqiDI4hFfZzvobexP/816PnhIKbcvaGJDY5yZ44r5zsxKYtGA3z7fyO9faCQI4EvHlW/3vo8sa+ZnTyZ6fJUXBVx4UAmfP7YCgM/fuoXjRxfx0ellXe9z4Q0b+cJx5cwcF+NHj9Rz58JmWjtChlcUMPeMKiYOLky7zNnY53a13a+sbeWye2r5yHtK+cOL2zhuVBFXnV7N/KWJ7V1d18G4QVG+c3Ilk4cU8qtnG3hjfRs/mdXdc+aH8+sIQ/jmSZV86sZNnD2lhA8cXEo8DHf5WfDsqhYuu6eWB/5x6HZlvOLUKo4bFePVta18/6E6VmztIFYQMGtKMf8yo2dtOFtt9VfPNnDT601s3hbngIoIXz6uglMmFNMRD/nJY/Xc9mYTZUURPnF4KVc9XM+LXxpGNBJQ3xLnPx6p59HlLUQCOG9aCV84tny7BH4+MlGTGyZqciWf+1VLkvLTHc/CH78KsUI4/0q48XGYMBw21EJtIzz9nxAP3/398st74cFX4G/fhAHl8K0/JJIgP/ts97KvLocn5sJjb8Bn/wdOOgT++i/Q0g6n/SucezQcNTGRiHl+MZx9JLz63/D3Z+GSn8NT/wlVpXRdFiMSwJ8ehb89CTfOgYHl8KVfwhV/hp/+I3zoeLj+Qbh4ZuL9X1kOmxtg5qGwrTmRyLnsIvjj1+CNlfDh/4QpIxLbe/kNUF4CL10Ny9Ynlh1/QHa+VyPZGfp0/+JmfnXhADpCuOiPm1iwoY1/P62K8QOjXHrLFv788jY+f2w5M8bG+MH7qyiMBPz4sXouu7uWv31sMJAI9Z0Lm7nm/AFMH15IWwfcsaCJIIDyogjXnj+Ay+6p5cHPdB+YbNoW57L3VXDQsELW1XfwT7ds4S+vFPCJw7sP+HL1E6S3Ca6OeMgXbtvCMSOL+OEZQygIAl5b18ZtbzZx6xtN/PaigQwsiXD5PbVc9XAdPzyjms7U3ItrWrnj4iEs39LO7HmbOG1iMTPGxvjsUWW8XdvB3DOqu95ncFmE/z1vACOrCnjunTb+6ZbNHHpAIdOGFvLK2la+dW8tP51VzbGjitjQGKexNWTG2ApeWt3K2VNLuCgPhz7tKnYEcMlRZRx5YBENrSH/fMcWrnmqgW+e1H2Amu6+2hnrZ1e1cufFg1lV28Gnb9rMlCFRjhsVIxoJ0t73rny4jtLCgPmXDOWdug4uvXkzwysLtovDI8tb+MuHB9HQGvKhP2/k5HExThwT612gUmRzlOYdC5v55QUDKSkM+MKtW/jlsw0cXlPE715o5NcXDuDAqgL+7f66rveNBFBaGHDV6VVMGBTlrY3tXHLzFqYOKeSUCcWcP62Y65/fxscPS8RuwYY21jd28L6xMZ58u5UXVrdyxycHUxELWLqlg8pY0KPtyOV2HzeyiE2NceqaQ+7/9BDiyfL/6/11/M+51Rw0rJDbFzTz5du3cMcnh3D2lBJ+8UwD21rjlMcidMRD7lnUzH+fU901j1BnzG55Y9efBcEutqszMTZ3fj0fP6yMc6eW0NgaZ/Gm9h7HIFtxG10d5Q8fHMjgsgj3LGrmm/fUcldNIQ8uaeGxFS3c9NHBlBQGfPWOrdttw7fvrWVQaYS7PjWYpraQz9+6lZqKJj50aHY/k7LNrgm5YVxzJXVgrDdv3rx587anG8A/ngY1A2FQBbx/OrzxNkSS3yffuABKiqAs9u7X3jAfvnkRHDgQSovg6+cnEiykJHW+el7i9adNh8IofOA4GFIFIwbBMZPg9bcTywZBohfNZ05LJIw+8F4YNRQefrV7XZ1luvkp+NyZMGYoVJYmesvc8lTifc86MtFrZuWGxLI3PQHnHwNFUbjvJRh3APzDCYn708fCmYfDnc9BRwfc8wLMuRDKiuHgUd0TF2cjzik9ajKuqgA+Nr2UoeUFDK8o4IgDCzl0eCEHDSukuDDgtAkxFmxoIxLARQeXUhGLUFwY8KXjylm4sZ3G1nhXqGeOj3HkgUVEIwElhcF2B5dd4U5570MOKOSwmiKKCgJGVkf50KGlPPdO6y6Xz+Yt6OXrX1vXxobGDv5lRgXlRRFKCgOOGlHEHQuaufiIMkZXR6mIRfjqCRXctaiZeBh2bdMXjy2ntDBg2tBCpgwuZNHGtq4y7VifJ48rZsyAKAWRgGNGFnH86BgvJGN08+tNXHhQCSeMSSQehlcUMGFQtKtee7tv5CJuu4vd2AFRThgTo7gwYHBZhIsPL+P51P2hB/tqV6yPK6e8KMKUIYVcMK2UOxc292jfC8OQ+xY386XjyikrCpg0OMp500retV9fclQZ1SURRlQVcMyIGItSypFPsSeAj76nlAMrCxhYEuGfjinnroXN3PtWMxdMK2HykELKiyJ8cYceNceOijFlSCHRSMDUoYWcNbm4K16nji/m7a3tvL01kUz4+4ImzpxUTHE0oKgAGltDlm9tJwhg4qAow8oL8ma7O5//8nvLKS4MKC0MuPG1bXzo0BKm1xRRWJDoPVQUDXhlXRsjqgqYNrSQB5cmeok8s6qVksKAw2qK3tXu0vks2LGMna+NFsDKre3UNsepiEW61r+34xYJ4MzJxRxQUUA0EjBrSgmjBxTw2ro27nmrmU8cVkZNZQEDSiJcclRZ1zZt3tbBoytauPzkSsqLIgwpK+Diw0u5a1FzVsqUy1s2e8Kpmz1qcsWhT5KknuhMkHR+d5TGYF0tBBEYVAmlxbt+7Tub4JNXb/+9EwSJHiydjw1LmbCxuGj79yougm0tiftBAMMHQEHKPDgjB8P62uTzJMoUicC6rTBqcPd6Rg2B1nbY3AhDq+DsoxPJnK+cC7c+A3/4f4ll39kMz74Fkz/f/R7tHTD7RNjUAB1xGJGy3pFD4KVl2fleTR7Z9/Z35eCySNc6iqMBg0tT7hcGbGsLiYchVz/ewN2LmtncFO860NjaHKeyOLEtwysK3lWWrimAgu77nZZtaeeHD9fz+ro2mtpDOuIhBw0r3G6ZgN5v3670Zr3rGjqoqSzYbpgWwPrGDg6s7I7fgVUFtMdh87Z4VwyGlKfGF5rawu23OeXvR5a18D9PNrB8SztxoLktMewsCBLDh2aMje1yO3I1/VJv17mr2G3a1sEPHqrnuVWtNLaFhCFUxoLt3i+dfTV1fxu+XV1EeGtTG0GQ/r63pTlOexyGV3bv28MrC94VhyFl29fptvYwq7HvrMtsrHO7mFRGWN/QwfqGDg4aFu16fETnNibf++U1rfz40Qbe2tRGWwe0doScMamYIIBYYcAZk4u5fUETXzyunDsWNPNf51QTBHDc6BgfnV7Kvz9Yx5r6Dk6dUMycGRWUx3r2+Zer7SaAASWJ5HOnNXUd3PpGE398aVvXY20dIRsaOwgCOHtKCXcubOKCg0q4Y2EzZ08pflf7DYL0Pgt29Xn5g/dX8d9PNHDW9RsYUVXAF44r5+Rxu/ne3IVsxO2WN5q4/vlG3qnrAGBba8jWpjjrGzq2+8xPbRdr6jto74AZv1zftZ54uPPviHwTyfPy9VcmanLFoU+SpJ6KpHx3pHYV6Px/V2oGwrWfgyMmvPu5peu6150q9XsqSLkfBLBmy/bLv7MpkbzZ8RTmAdWJ5zqXXb050UNmaGXisdknwld/DdPHJebLOWJ8YrkRgxJz7fzlG+8ub1t74rVrtsC4Ycn1btr5NmQiSEwWGenFL9+AgCDoXkeQPKPYdT95RHHHwmYeWNLM9R8cyIjKAupbQo78n3Vd79/5mtSyBEHQ9XxB8vHU57/7QB3ThhTy07OrKS+KcP3zjdy9qGm7ZXZcZ7YEvVxvTWWUNXVx4iHbTT46tLyANfXxrnWvre8gGoEh5QWsrU8c6ER2iG9njCJdV/FKPNfaHvLl27fwozOrOWV8MYUFAZ+7ZTMklx9eGWVVbXyn2xFJiX029TZusOvY/fSxBgLg9k8OYUBJhPveauZ7D9ZuH6s09tXU+K6rDxk/qLMu4sneHEHa+97g0gKiEVjfEGfswGhynfGu51OX7S5X9mMfyUJbh8Q61jWk7J8NcYaWFzC0vIC1qfttw/b76tfvrOVjh5Xy6w8MJBYN+MFDtWxp6l7+woNK+cZdWznqwBglhQFHHNg97OviI8q5+IhyNm3r4Cu3b+U3z2/jn4+vSL/MWdjndrXdO7Y5gOGVUT53TBGfO3bnZTxrcjFzH6ljfUOc+xc385cPD9ppe97dZ8GGbXGa28Ou5zriIVu2hV3bOm5gIVefPYB4GHLvW8185fatPPOFYZQWpp/gykbc3qlr5zv31fK7iwZxWE0hBZGAc3+/AUhs37rG7u1b19DdLmoqoxRF4enPD+t3kzPneyKpv7LLR650/oj15s2bN2/e0rnB9v2ug5TvE/bwnXLxTPjBXxMJjUiQmIz3nheSz+/k9cGO91MfI9FT5tf3JSYLvuUpWL4eTn1P92s7y/mB98I1d8OqjdDYDFf+FS48LtEHPRLAeycnesr8+18Sc9Z0vt9ZR8DCdxLz23R0JG4vLoUlaxLDrc46Ev7jb9DSBgtXwf89nr3v1SDYLsyZVtW7qo7t3oKAxFnUWEHAwJIILe2JSSRTqzk1lJGdPDakrICtzfGuoVKRILHOilhARVHAss3t/Pnlxu3WseNulM1bb6tg+vBChpZH+PGj9TS3xWnrCHlxdSvnTClJnH2ubaepLc5PHqvnrMnFFBV0z8uxq6YypCySPGudGBrRHoa0dsCg0ghFBfDosmYeX9HStfyHDinhpte38dTbLUDI+oYOlm1ODD8ZXBphVV173sVtd7FrbA0pKwqoLg5Y39DBr59r2C5e6e6rqc9f83Q9Le0hize1cdNrTcyaUtKjfa+wIOD9E4v5+ZOJ9Szb3M4tb2zb7X66s7aQD3HvLOsfX9rGuoYO6prj/OLpBmZNKWbW5GJufr2JJZsSk4X//Mnt23dja8iA4sQwtVfXtnL7m83b1ccRByaG5vxwfh3nTyvpevy1ta28sraVjnhIWWFALBpQ0AfbvqvtDnbylTL70BL+/Mo2XlnbSkBIc1uc+Uub2daW+OwaXJYY3vbNe7YyoqqASYMLd7ov7O6zYPzAKC3tIfOXNtMRD7nm6QZaO8Ku9dz25ja2NHUQjQRUJXssFkaCvR635raQgERPtkgAf3ttG29tTAxjO2tyMb9/oZH1DR00tMT51bPd7fWAigJOGB1j7vw6GlvjQMjK2naeXdWS1XaRq5uyzx41uWKPGklST+34yzX1iD71O2X5ejjhMnj2x4meLl+clVj2gqsSQ5SGVCXmoJl1ZPfrdvxO2vF7KrVHzdETE4mUCZfCAQPg919JTBbcVc7k+i6eCeu3wqx/TyRVTn0P/PATKesNEvPQ/Ohm+PPx3Y9Xl8FNc+Dbf4Rv/h5C4JDRiatLRQL4yafg87+AqZ+HSQfCR9+XuAJUNr5Xg95f9Qm2r5KA7cPZ+f8HDi7hsRUtnHDteqpKInz1+HL+9PL2BwU7liX1sYmDo5wzpYSTr1tPRwj3fnoIl59UweX31nLds41MGxrl7CklPPF2y3bryNUP597+tIkUBFx34UC+90AtM365niCAc6eW8J2ZlWxo7OAjf9lESzucOKaIK06t2m47dnZQFwlg1pQSbn2ziSN/nrjq0+2fHMK/zqzky7dvpbUj5JTxxZw6obgrpofVFPGjM6r5wUN1rKrtYHBZhO+eWsXEwVE+dUQZX79rK396aRsXHFTCv51S1euYpZa3N3YVu68cX87X79zKYT9bx+jqAi44qITfPNf4rn1qT/tqanyPGRnjlF+tJx4m5pF539hET4+e7HvfPbWSb9xVy3HXrGPswCjnTC3h1bVtu9xPd9YWeqvz4ywb6zx3agmfunEz6xo6OG1CMV86roKSwoBPHdnOx/9vM5EAvnpiBbe92T2fyPdOq+TKh+v57gN1HD2yiFlTiqlrjm9XngsPKuEnjzXwiwtKuh5vbAv5/oN1rKztIBaFE8fE+OzRZT3ajlxu98trWt9VV+8ZXsRVp1fx3QfqWL6lneJowJEHFnHMqKKu5c6bVszX7qzlsvdV7PSrJxLAPxxassvPgqriCN87rYrL760lHsJnjyrjgIqCrrI8uqyFqx6up6kt5MDKAv77nAGUFPYsCNmI2+QhhXzmqDI++KeNRAK4YFppV1Luw+8pZfmWDs7+3QbKYxE+eXgpT69spTA56vjHs6r50fx6zvztBhpaQ0ZVF3Dp0eUeUu6ngjAMd/f8bp/Ublz518TEjpIk9Se/exD+7zG441+zs74/PJS4fHe21pcF4VU3+h2dgZ8+Vs//OyH94RdK6C9xW1nbzom/2MDirx+Q9aEXVz1cx4bGOD+ZVb3nhbPkrY1tLNjQzjlTS3q1nuOvXc/cM6o4IYtXpOp002vb+PPL27jxo4Ozut5s7HO53O58tbfb6kNLm/n2vXU8/k9D97xwnrv68Xr+3wk9uxy6gO5+0+9ij5pccsCeJKm/6fzuysZ32LYW+PX98IWz8uo7MYC8Kk9/0Tm/iXqmv8Sts4xBcv6k3licnEB3ypAoL69p4/9ebWLuGVV7Nw7J7ejteyZ6IfV+PTtqaotzw4vb+PjhZVlfdzb2uVxtdz7LdVttbgt54u0WZoyNsbExzn8/3sDpE4v3qxgrfSZqJElS9t39Alz8X3DKoYm5bCTtNxpbQ7502xbWNcQZXJq4DPH7J/b8Cjz7qvlLm7n0li2cMDrGedN61+NH/UdIyE8fq+eLt22hOBowc1yMr56Y/73t1DdM1EiSpG6fnJm49dYZh8Pa3/V+PZL2ipFVUVbMqcnKut4zvIhHLh2WlXX1tcc/l/3teN+4YhZ8dXjW15tNudju/V1JYYTbPzmkr4uhfsKrPkmSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCecTFiSJEnSPmXioCgTBnmoI6l/CsIw3N3zu31SkiRJkiRJPRbs6gmHPkmSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWS0MsLIAAAAiBJREFUJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJOUJEzWSJEmSJEl5wkSNJEmSJElSnjBRI0mSJEmSlCdM1EiSJEmSJP3/du3QBgAgBIJgPqH/mr8EHFkxY8+gN0QINQAAAAARQg0AAABAhFADAAAAECHUAAAAAEQINQAAAAARQg0AAABAhFADAAAAECHUAAAAAEQINQAAAAARQg0AAABAhFADAAAAECHUAAAAAEQINQAAAAARQg0AAABAhFADAAAAECHUAAAAAEQINQAAAAARQg0AAABAhFADAAAAECHUAAAAAEQINQAAAAARQg0AAABAhFADAAAAECHUAAAAAEQINQAAAAARQg0AAABAhFADAAAAECHUAAAAAEQINQAAAAARs+zv5AoAAAAAfNQAAAAAVAg1AAAAABFCDQAAAECEUAMAAAAQIdQAAAAARAg1AAAAABEfepsFa4uB7doAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "The ground truth was 0.0. The model predicts 0.005339. \n", "\n" ] } ], "source": [ "description = inspector.explain_prediction(data_row_id = 31614)\n", "print(description)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "#### Post Training Bias Analysis\n", "\n", "The blueprint uses Amazon SageMaker Clarify to generate post-training bias metrics. The animation below how to navigate to the bias reports generated for your best model.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Bias Report](img/bp-bias-report.gif)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "### [Optional] Deploy Model\n", "\n", "You've inspected your model. You've approved the quality. You're now ready to deploy as a hosted endpoint on Amazon SageMaker to serve real-time predictions.\n", "\n", "You can run the cells below to deploy a hosted endpoint." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'EndpointArn': 'arn:aws:sagemaker:us-west-1:803235869972:endpoint/automl-bp-ep-2021-04-08-21-50-52',\n", " 'ResponseMetadata': {'RequestId': '08a96083-0750-46aa-9f46-066e9b103064',\n", " 'HTTPStatusCode': 200,\n", " 'HTTPHeaders': {'x-amzn-requestid': '08a96083-0750-46aa-9f46-066e9b103064',\n", " 'content-type': 'application/x-amz-json-1.1',\n", " 'content-length': '100',\n", " 'date': 'Thu, 08 Apr 2021 21:50:52 GMT'},\n", " 'RetryAttempts': 0}}" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "now = strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n", "\n", "epc_name = f\"automl-bp-epc-{now}\"\n", "ep_name = f\"automl-bp-ep-{now}\"\n", "\n", "model_name = bprunner.get_best_model_name(execution_arn)\n", "ep_instance_type = \"ml.m5.large\"\n", "\n", "variant_config = {\n", " \"VariantName\":\"v1\",\n", " \"ModelName\":model_name,\n", " \"InitialInstanceCount\": 1,\n", " \"InstanceType\": ep_instance_type\n", "}\n", "\n", "sm.create_endpoint_config(\n", " EndpointConfigName = epc_name,\n", " ProductionVariants=[variant_config])\n", "\n", "sm.create_endpoint(\n", " EndpointName= ep_name,\n", " EndpointConfigName=epc_name\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can monitor the progress of your endpoint deployment from Amazon SageMaker Studio:\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "#### Clean up\n", "\n", "Once you're done with your endpoint, you can uncomment and run this line of code to delete your endpoint." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#sm.delete_endpoint(EndpointName=ep_name)" ] } ], "metadata": { "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Data Science)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:eu-west-1:470317259841:image/datascience-1.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 4 }