{ "cells": [ { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "### Multi Objective Optimization with Credit Risk Analysis\n", "Here we will optimize our AI model for multiple objectives - Performance with Area Under Curve (AUC), and Fairness with Statistical Disparity (SD)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%%capture\n", "#Cell magic to hide outputs \n", "\n", "#Install dependencies\n", "from io import StringIO\n", "import os\n", "import time\n", "import sys\n", "import IPython\n", "from time import gmtime, strftime\n", "\n", "import boto3\n", "import numpy as np\n", "import pandas as pd\n", "import urllib\n", "\n", "import sagemaker\n", "from sagemaker.s3 import S3Uploader\n", "from sagemaker.processing import ProcessingInput, ProcessingOutput\n", "from sagemaker.sklearn.processing import SKLearnProcessor\n", "from sagemaker.inputs import TrainingInput\n", "from sagemaker.xgboost import XGBoost\n", "from sagemaker.s3 import S3Downloader\n", "from sagemaker.s3 import S3Uploader\n", "from sagemaker import Session\n", "from sagemaker import get_execution_role\n", "from sagemaker.xgboost import XGBoostModel\n", "from sagemaker.sklearn import SKLearnModel\n", "from sagemaker.pipeline import PipelineModel\n", "from sagemaker.inputs import TrainingInput\n", "\n", "\n", "session = Session()\n", "bucket = session.default_bucket()\n", "prefix = \"sagemaker/sagemaker-amt-credit-risk-model\"\n", "region = session.boto_region_name\n", "\n", "# Define IAM role\n", "role = get_execution_role()" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# !pygmentize processing/preprocessor.py" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## Train XGBoost Model\n", "In this step, we will train an XGBoost model on the preprocessed data. We will use our own training script with the built-in XGBoost container provided by SageMaker.\n", "\n", "Alternatively, for your own use case, you can also bring your own model (trained elsewhere) to SageMaker for processing with SageMaker Clarify\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Set up XGBoost Estimator\n", "\n", "Next, let us set up: \n", " 1. Pre-defined values for Hyperparameters for XGBoost algorithm\n", " 1. XGBoost Estimator for SageMaker\n", "\n", " " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "hyperparameters = {\n", " \"max_depth\": \"5\",\n", " \"eta\": \"0.1\",\n", " \"gamma\": \"4\",\n", " \"min_child_weight\": \"6\",\n", " \"silent\": \"1\",\n", " \"objective\": \"binary:logistic\",\n", " \"num_round\": \"100\",\n", " \"subsample\": \"0.8\",\n", " \"eval_metric\": \"auc\",\n", " \"early_stopping_rounds\": \"20\",\n", " \"output_data_dir\": \"/opt/ml/output/data/\",\n", "}\n", "\n", "entry_point = \"train_xg_amt_multi.py\"\n", "source_dir = \"training/\"\n", "output_path = \"s3://{0}/{1}/{2}\".format(bucket, prefix, \"xgb_model\")\n", "code_location = \"s3://{0}/{1}/code\".format(bucket, prefix)\n", "\n", "estimator = XGBoost(\n", " entry_point=entry_point,\n", " source_dir=source_dir,\n", " output_path=output_path,\n", " code_location=code_location,\n", " hyperparameters=hyperparameters,\n", " instance_type=\"ml.c5.xlarge\",\n", " instance_count=1,\n", " framework_version=\"0.90-2\",\n", " py_version=\"py3\",\n", " role=role,\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "### SageMaker AMT (HPO)\n", "\n", "Now it's time to run the HPO job to train and find the best model \n", "\n", "#### NOTE: THIS CELL WILL RUN FOR APPROX. 5-8 MINUTES! PLEASE BE PATIENT." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "%store -r train_data_path\n", "%store -r val_data_path\n", "%store -r model_path" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "No finished training job found associated with this estimator. Please make sure this estimator is only used for building workflow config\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "...................................................................................................................................................!\n", "!\n" ] } ], "source": [ "# Automatic Model Tuning (HPO)\n", "\n", "# output_data_dir = 's3://sagemaker-us-east-2-921553072635/sagemaker/sagemaker-amt-credit-risk-model/data/output/'\n", "\n", "from sagemaker.tuner import IntegerParameter, CategoricalParameter, ContinuousParameter, HyperparameterTuner\n", "\n", "hyperparameter_ranges = {'eta': ContinuousParameter(1e-3, 1, 'Logarithmic'),\n", " 'min_child_weight': ContinuousParameter(1e-6, 10, 'Logarithmic'),\n", " 'gamma': ContinuousParameter(1e-6, 10, 'Logarithmic'),\n", " 'max_depth': IntegerParameter(2, 10, 'Linear')}\n", "\n", "objective_metric_name = 'validation:auc'\n", "\n", "tuner = HyperparameterTuner(estimator,\n", " objective_metric_name,\n", " hyperparameter_ranges,\n", " max_jobs=100, # 100\n", " max_parallel_jobs=10, #16\n", " )\n", "\n", "tuning_job_name = \"xgb-tuner-{}\".format(strftime(\"%d-%H-%M-%S\", gmtime()))\n", "\n", "# inputs = {'train': train_data_path, 'validation': val_data_path, 'output_data_dir': output_data_dir}\n", "\n", "inputs = {'train': train_data_path, 'validation': val_data_path}\n", "\n", "tuner.fit(inputs, job_name=tuning_job_name)\n", "\n", "tuner.wait()\n", "\n", "tuner_metrics = sagemaker.HyperparameterTuningJobAnalytics(tuning_job_name)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'xgb-tuner-27-00-32-14-052-be514283'" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tuner.best_training_job()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "xgboost_model_data = (\n", " \"s3://{}/{}/{}/{}\".format(bucket, prefix, \"xgb_model\", tuner.best_training_job()) + \"/output/model.tar.gz\"\n", ")" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "xgboost_inference_code_location = \"s3://{}/{}/{}/code\".format(bucket, prefix, \"xgb_model\")\n", "\n", "xgboost_model = XGBoostModel(\n", " name=\"xgb-model-{0}\".format(str(int(time.time()))),\n", " model_data=xgboost_model_data,\n", " entry_point=\"inference.py\",\n", " source_dir=\"inference/xgboost/\",\n", " code_location=xgboost_inference_code_location,\n", " framework_version=\"0.90-2\",\n", " py_version=\"py3\",\n", " role=role,\n", " sagemaker_session=session,\n", ")" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "EndpointName = MULTI-2023-01-27-0044\n" ] } ], "source": [ "from datetime import datetime\n", "\n", "endpoint_name = f\"MULTI-{datetime.utcnow():%Y-%m-%d-%H%M}\"\n", "print(\"EndpointName =\", endpoint_name)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-----!" ] } ], "source": [ "#Create Endpoint Instance\n", "\n", "#Instance Count\n", "initial_instance_count=1 # Example\n", "#instance Type\n", "instance_type='ml.m5.large' # Example\n", "\n", "FinalModel = xgboost_model.deploy(\n", " initial_instance_count=initial_instance_count,\n", " instance_type=instance_type,\n", " endpoint_name=endpoint_name\n", ")" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'s3://sagemaker-us-east-2-921553072635/sagemaker/sagemaker-amt-credit-risk-model/data/preprocessed/val/val_features.csv'" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#Validation\n", "n = val_data_path+\"val_features.csv\"\n", "test_dataset = S3Downloader.read_file(n)\n", "n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "44157\n", "221\n" ] } ], "source": [ "print(len(test_dataset))\n", "test_dataset = test_dataset.split(\"\\n\")\n", "print(len(test_dataset))" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "for i in range(len(test_dataset)-1):\n", " test_dataset[i] = np.fromstring(test_dataset[i], dtype=float, sep=',')" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "#Set up predictor \n", "from sagemaker.serializers import CSVSerializer\n", "predictor = sagemaker.predictor.Predictor(\n", " endpoint_name,\n", " session,\n", " serializer=sagemaker.serializers.CSVSerializer(),\n", " deserializer=sagemaker.deserializers.CSVDeserializer(),\n", ")" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "#Make Predictions on the test dataset\n", "Multi_Optimized_Predictions = []\n", "for i in range(len(test_dataset)-1):\n", " Multi_Optimized_Predictions.append(predictor.predict(test_dataset[i]))" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[['0.9327612']]\n", "[['0.86414945']]\n", "[['0.9440027']]\n" ] } ], "source": [ "#See first 3 predictions and how they look. We have the double brackets meaning each element is a 3 dimensional element, we will make them a single dimension in the next cell \n", "print(Multi_Optimized_Predictions[0])\n", "print(Multi_Optimized_Predictions[1])\n", "print(Multi_Optimized_Predictions[2])" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "#Turn each item into a single dimension element in the list \n", "Predictions = []\n", "for i in range(len(Multi_Optimized_Predictions)):\n", " Predictions.append(Multi_Optimized_Predictions[i][0][0])" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.9327612\n", "0.86414945\n", "0.9440027\n" ] } ], "source": [ "#Look at first 3 processed predictions. They are now i nthe format we want\n", "print(Predictions[0])\n", "print(Predictions[1])\n", "print(Predictions[2])" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "#Each prediction value is between 0 - 1. But the target variable \"Credit Risk\" is either bad (0) or good (1) there for we will round up/down to ensure every value is either 0 or 1\n", "for i in range(len(Predictions)):\n", " if (float(Predictions[i]) >= float(0.5)):\n", " Predictions[i] = 1 \n", " elif (float(Predictions[i]) < float(0.5)):\n", " Predictions[i] = 0" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1\n", "1\n", "1\n" ] } ], "source": [ "#Look at first 3 predictions now to confirm we will be able to see they are either 0 or 1 \n", "print(Predictions[0])\n", "print(Predictions[1])\n", "print(Predictions[2])" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "71\n", "149\n" ] } ], "source": [ "#Check how many predictions were 0 (Bad Credit) and 1(Good Credit)\n", "print(Predictions.count(0))\n", "print(Predictions.count(1))" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAATkAAAD3CAYAAABmdCOBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAtcklEQVR4nO3deXxcZfXH8c83yWRPW7pAKaXsOxQQkH1XoIqyqiyCgKAoiIgoyiJlERAUWRVR+LEXkB1BBYGCrLIVWnYoS0sXKKVLkiaZJOf3x3NDp+lMMkkmuTOT83695pXJ3O3MzL1nzvPcTWaGc84Vq5K4A3DOuf7kSc45V9Q8yTnniponOedcUfMk55wrap7knHNFLfYkJ2l1SSapLPr/n5K+l8P5nyrpb1mMN1nS0blabk9IukrSGXEsO59IOkLSk10M31HSWwMcU6+XKWmcpHpJpTmO6QNJX8nlPHO1zM7bcz/Escw6Ei1r7a6mySrJSTpE0gvRFzY7SkQ79DXgdMxsgpldHy23y5U+GmeypKYotnmS7pK0csr8zjOznCav6AtfEi1zjqTrJNVmOe1y78nMjjWzc3oRR8cKVZ/yeKWn8+krSbtEcdzV6fVNo9cn93K+y6zAZvZfM1uvi/El6ReS3om+n48kXSCpor+W2RUz+8jMas2srTfT90a0Lpqkb3Z6/ZLo9SMGKpa+kDQxivfLfZ1Xt0lO0knAJcB5wErAOOBPwD4Zxu+XDN6N482sFlgbqAV+PwDL/Ea0zM2AzYFfD8AyMxkWbUy1ZrZpTyfO0Xf2KbCdpBEpr30PeDsH887WZcAPgMOBOmACsBtw+wDGkA/eJnz2wBff77eA92KLqAckCTgMmE/K++itLpOcpKHA2cBxZnaXmTWYWdLM7jezX0TjTJR0h6SbJC0CjpA0VNI1UdX3saRzO0p2SaWSfh9VXdOBr3da5mRJR0vaALgK2DaqUBZ092bMbAFwDyHxdMxvoqSboueVUZyfSVog6XlJK6V53ytLelXSyVkscw7w707L/JWk9yQtlvS6pP2i19O+p+jX99yU6Y+R9K6k+ZLukzSmuzjSvIcx0bTzo3kd0+kzWeY7SzP91yW9LGmRpBmSJnazyBbCZ39QNH0p8G3g5pR5LteUUYZuAklPRE9fiT6r70QV48wM73cd4MfAoWb2jJm1mtlrwAHAXpJ2i8a7TqF74OHo+3lc0mrZLlOhiv9FtH40ROv5Sgqtm8WS/iNphc7vV1LHd97xaJL0QTReSco685mk2yUNT1nmYZI+jIad1s33AHA/sH1HHMBewKvAnJR5lkg6PZrvJ5JuiLb3bpfZXbw5sCMwBvgpcJCk8r7MrLtKblugEri7m/H2Ae4AhhFW6uuBVkJltTmwB9CxIh8D7B29viVwYLoZmtkbwLHAM1GFMqybGFCoIvYH3s0wyveAocCqwIho/ks6zWN14HHgCjPrtiKUNJZQMaQu8z3CFzUUOAu4SdLK2bynaGM8n5AgVgY+BG7tLo40JgEzCSvLgcB5knZPGd75O+usgVARDSP8EP1I0r7dLPOGaBqAPYHXgFm9iB0z2yl6umn0Wd3WzSS7AzPN7H+d5jMDeBb4asrLhwLnACOBKUTvvwfLPCCa37rAN4B/AqdG8ysBTkjzfjq+81pghSimSdHgE4B9gZ0J39fnwJUAkjYE/kyobMYQ1tux3XwWTcB9RD84hO/khk7jHBE9dgXWJLSArshymRnjzZHvERJ1x+e/d19m1l2SGwHMM7PWbsZ7xszuMbN2YAhhoz8xqvw+Af7I0g/828AlZjbDzOYTNui+ukzSQmAeYUX7SYbxkoT3tLaZtZnZi2a2KGX4hsBk4Ewzu7qbZd4jaTEwA/gEOLNjgJn93cxmmVl7tKG8A2Tbt3AocK2ZvWRmzYRm8LZR8s1kXlSZLpB0sqRVgR2AU8ysycymAH8jrLQdvvjOzGxJ5xma2WQzmxoNf5WwQe7cVeBm9jQwXNJ6pN+w+tNIYHaGYbOj4R0eMLMnos/3NMLnu2oPlnW5mc01s4+B/wLPmdnL0fzuJvyAd+Uywo9IR4X0Q+A0M5sZzWMicGBU8R4I/CMl3jOA9ixivAE4PKrOdiZU2akOBS42s+lmVk9Yzw7KcpldxdsnkqoJTetbzCxJ+CHuU5O1uyT3GTAyi+BnpDxfDUgAszs2POAvwIrR8DGdxv8w+3AzOsHMhgLjCb+SmX7pbiQ0LW+VNEvShZISKcMPBT4mfLDd2dfM6oBdgPVJ2YgkHS5pSsr735hlN7KujCHlM4lWwM+AVbqYZqSZDYsev4/mMd/MFqeM82GnecygC5K2lvSYpE+jH5Bjs3wPNwLHEyqE7loAuTSPUPmms3I0vMMX7z36fOcTPrNszU15viTN/xl3Qkn6IWGdOSQqCiBsM3enrC9vAG2EPvBlthczayCsD10ysyeBUcDphITV+YdsmfUsel6W5TK7ijcjSYemNNf/mWG0/QitwAej/28GJkga1dW8u9JdknuGUPru2814qZcymQE0s+yGN8TMNoqGzyY0FzuMy3K+3TKzqcC5wJWSlGZ40szOMrMNge0IZfDhKaNMJGwMtyjL3f5m9jhwHdHOjqh/56+EDX1E1CSdBnTE0917mkVYiYjmV0OoPj/OJp6UeQyXVJfy2rhO8+gujlsITZ5Vox+Qq1j6HrpyI6Fv7EEza+w0rCH6W53y2ugs5pmNR4FV1WlvXFShbQM8kvLyqinDa4Hh9LJZ3ROSdiQ0k/cxs4Upg2YAE1K2l2FmVhlVistsL1GlM4Ls3AT8nPQV9TLrGWH9aCUk7O6W2VW8GZnZzSk7yCZkGO17hB+JjyTNAf5OKJoO7mreXekyyUVfxG8ISWNfSdWSEpImSLowwzSzgYeAP0gaEnVSriWpo6lzO3CCpLFRx+ivughhLjC2hx2P1xOqxm92HiBpV0mbRAlsEaH5mrp7P0kolWuAGyVlexzhJcBXJW0WTWuEvY1IOpJQyWX7nm4BjpS0mcKhD+cRmkMfZBlLRz/U08D5CjtbxgPfJ33fWyZ1hGqwKUoch2S57PcJzaPlOsjN7FNCov2uwg6oo4C1upjdXEJ/UTbLfZuQiG+WtE00/42AO4H/mNl/Ukb/mqQdou/gHMLn21G5ZL3MnoiS7W3A4VGsqa4CfqulO0BGSeo4euEOYO+UeM8m++NbLyP0HT6RZtgk4GeS1ogS/XnAbVHXVHfL7CreXpO0CqFvdW/CjrzNgE2B39GHJmu3H5aZXQycRCh7PyVk8eNZvo2f6nCgHHid0Cl5B0ubEn8lNBlfAV4C7ko3g8ijhM7rOZLmdTFearwthC833cG1o6NYFhFK7McJv3adp9+fkCivzSbRRRvvDcAZZvY68AdCFTwX2AR4Ktv3ZGaPRLHfSfhFXYul/Zk9cTCwOuEX+25CP+PDPZj+x8DZUb/jb+jBYRhm9qSZZaqMjgF+QWj+bERIxplMBK6PmkXfzmLRxxP6Hm8C6oF/EfpYD+g03i2EPtT5wBaEboreLjNbuxOtfylNtteiYZcSquaHos/7WWBrAAt7iI+LYp5N2J7S7mHuzMzmm9kjZmkvGnktoep+Anif0GL7SZbLzBhvHx0GTDGzh8xsTseDsD2Pl7RxN9OnpfTv37niJOk6wl7Y0+OOxQ2M2E/rcs65/uRJzjlX1Ly56pwral7JOeeKmic551xR8yTnnCtqnuScc0XNk5xzrqh5knPOFTVPcs65ouZJzjlX1DzJOeeKmic551xR8yTnnCtqnuScc0XNk5xzrqh5knPOFTVPcs65ouZJzjlX1DzJOeeKmic551xR8yTnnCtqnuScc0XNk5xzrqh5knPOFbWyuANwxUeSgDpgJDAi+jsSGA60A81AU/S3GagH5kePz4B683tluhzxJOd6TFIpsD6wVTVsUwWrAyu2wYgWGFoCtQloHwItK0DrKGAlKB0FZQYsgfaORxPYYkJ2WwBli6E8CSU1UkMlfNACz9fD88CrwDQzWxzT23YFym8u7boUVWVrAltVwXZVsPNiWG8kJLcGdoDaVVm2XBsBVPZhmS3APOBNYCrwPDS+CMn3oLoSPi+HaQvh6VZ4hZD83jOztj4s0hUxT3JuGZISwK7lsFst7NIAm9QAW0HrTlC3FWhLYIUYYmsD3iUkvinQ/j+ofwVKFkCiGh5fADcBD5jZ/IGKSVJbFJKiEI83s6d7MP1EQvP892mGHQ78Mpq3gGvTjZflclYH/mFmG0vaEjjczE6QtAvQ0pOYC403V11H83PHOvheFRywFtg+ULs1lGwFjI47wEgpsF70ODDsNBsCoRPvH7DHJNjucUiMkKZ+Dtcb3GtmM/o5rCVmthmApD2B84Gd+zpTSROAE4E9zGyWpErgsDTjlZlZa0/mbWYvAC9E/+5C6BMt2iTnldwgFTVDt6mBwwwOXgVKj4Ka70DJGnEH1weNwMPAbdB4P5SUwYwGuCkJdwGv5XqHhqR6M6uNnn8LONTM9pVUC9xLKHoTwOlmdm803mnA4cAM4FPgxc4VmqQngIlm9miaZU4mJKXtgfuAycDFQC2hpX+Emc2WtAVwbfSxPAlMiCq5XYCTgeOBZwkV6KfAT8zsv7n5ZPKImfljkDwITZ4vVcHFtfDpqrD4TEi+CWZF+EiCTQY7DppHQX0dzKmEc4FROfxM24AphC7EhcAW0etlwJDo+UhCS1vAFoTmbTWhEn0XODnNfOcDQzMsczLwp+h5gpDwRkX/f4fQrIXQX7lz9Pwiwo4bCNXbP6LnE9Mtv5ge3lwdBCRVlcBRdfDrGhh2OJQfColNCFtdsSojtBt3hvLLoXwK1FwBP58EJ9VJk+rhfDN7t4+LSW2ubgvcIGljwkd7nqSdCIfNrAKsBOwI3G1mjdE09/VyubdFf9cDNgYeDsU5pcBsSUOBYWb2eDTejcCEXi6roPnBwEVM0gqV0plVMGd3uOBBWGUW1PwOEuMp7gTXmYDNgWug8n2oOh4Oq4Wpw6QHJH05F8sws2cIVdso4NDo7xZREpzL0p3O2TSZXyNUfZk0RH9FaIZvFj02MbM9ote9LwpPckVriPTbSvh4fzjlBRjyENTuwOBKbJmsBJwPidlQORH2GgWPDZNelPR1Sb3eJiStT6ikPgOGAp+YWVLSrsBq0WhPAPtJqpJUB3wjw+zOBy6UNDqad4WkE9KM9xYwKqoikZSQtJGZLQAWStohGu/QDMtZTDhwu2h5kiteR14PVbdA1YZxR5KnaoEToeRjqP4TfGlduLUO3pd0hKTyLGdTJWmKpCmEJuT3LByzdzOwpaQXCAnmTQAzeykabwpwJ5C2o9/MHgSuBP4j6TXgRdIcDWFmLcCBwO8kvRLNd7to8JHAlZKeAZZkiP9+QtKdImnHLN9zQfG9q0WqRPr+dnDJk2Fbdlkw4D/AWVA/BRY3wDFm9kDMYbk+8iRXpCSV18DMh2DUdt2P7jr5F3A0NCyG5xbBD8zsvbhjcr3jzdUiZWYtS+A3p4YDPV0P7QW8BzW/gp2qYWq1dIGk6rjjcj3nlVwRk1RRDbMeg+E52X04SM0EjofGR2BBPRyScliGKwBeyRUxM2tugYmnLz3cwPXCWOAeqL4ZxgyHB+ukayUNiTsulx2v5IqAJFmGL1JSZTXMfhKGbT7QgRWhBcAJ0HQnNDTCYWb2z7hjcl3zSq6ASapVmU6jjIUqS3sMFWbWlIRzTvNqLieGATdA5b0wYiTcUSOd35dj61z/80quAEkqo4RjKeE81qaUTajmHhaSZEzH6UKdxq+uhlnPwNDxcQRcpOYCX4OGd+DJxfAt8wt65iX/BSowkr5COe8whgs4mjoOopqNgNUoQ/wg3TRm1piE87xvLrdWAp6Gmv1h51p4RdJaccfklueVXIGQtBYV/JkE2/N1qlmfZc/RmgX8H59H1VxTmulrq2DW81C30YBFPTgYcAW0nQKNS2A/M3sk7pjcUl7J5TlJCSV0Ngmmsj278lOq2YDlT0IdA6xKOeLodPMxs/o2uPA34dpiLocE/ARKH4S6IXB/hXRidL0+lwe8kstjkjahnDtYmbHsTzVDu5lgJnA986NqrjnN/IZUwccvQ+16/RPyoPc+sAc0zoV7FsNR6b4HN7C8kstDksqU0JkkeI69WIcjskhwEA7oGkM54sh0g81sUTv8YaJXc/1mDeBlqN4J9q2D/3VcRcTFxyu5PCNpLcq5n9Gsxv5UM6yHM5gB3MCnJFnFzJJp5j+sCma+CjVr5yRil047cDokL4dZ9bCVmX0ad0yDlVdyeUTSviSYwu6sx5G9SHAAqwKjqSLNTU8AzGyBwaVnZb70jsuBEuA8SBwPY2rhSUnD4o5psPJKLg9IKiPBRST4AYdQzdg+zvAD4Bbm0sJYS3MnJ0nDK2HG61BdyDetKQQGHA/NN8Jbi2F7M/MLJgwwr+RiJmk05TzNGI7huBwkOAj3sx9FDXBIusFmNl9w5Tmw3KEmLrcEXA4V+8G6deE+DH2577brBa/kYiRpIxI8xjYMY1cSOf3JmQ7cyixaGGdp7i4vaWQlfPQWVI3L4WJdem3At2HJw/DsYtgruqKvGwBeycVE0i4keIa9GcnuOU5wEHbzDWcI4RZ1yzGzeYK/nAt+iMMAKAVuhaptYetauDO6obcbAF7JxUDSwZRzDQdRxZr9uKB3gdv5iBbWzFDNrVQF778DVav0YxhuqSbgK9D4CtxXH25E3R53TMXOK7kBpoROoZq/8f1+TnAAawHDGA4ckG6wmc0tgWvP92puwFQC/4bqdeEbNXCVnxnR/7ySG0BK6CxqOZkjszy4NxfeBu7kfZpZO13VIGnlKpg+HSr9qNWBsxDYAho+hJ8lzf4adzzFzCu5AaKEzqSGk/n+ACY4gHWAOkYB+6QbbGazS+GGC8A7wgfQUOB+qCmHSyT5NRP6kVdyA0AJnUENv+JoqmO5je+bwN28SzPrpruCsKSxVfDOB1C54sBHN6hdA3YifFgPG5qZH6DdD7yS62dK6DSqY0xwAOsBNYwG9k432MxmlsItF3k1N+COAu0JK9XCVXHHUqy8kutHKtXRVHMpP6CauG978jpwL2/SzIYZqrnVquDNGVA5IobwBrNFwAbQOAuONLPb446n2Hgl108k7UGCyzgiDxIcwPpAFWMJtxRdjpl9WAZ3/AGWO6nf9a8hwL1QXQ3XSPIz7XLMK7l+IGk8CZ7mu9SwWtzRpJgG3M80mhmfoZpbsxpemwmVK8QQ3mD3B2g7C95cDJunu4KM6x2v5HJM0iokeIR9qM6rBAewIVDJGsDu6Qab2fRSuPcSWO6kftf/ToLSrWGNargw7liKiSe5HJJURTmPsBPD2Hi5C5THrwTYnRoquCjTQaiL4Yw/QOvCAQ6tK03Al4FNgY2AM6PXf0FohY8H9iPcEzXbaQFOiaY9POW1G4FLcxd6jwiYBNVV8ANJe8YURtHxJJdL5VzFmoxjB8riDiWjjYBy1gF2TjfYzN4phQcuy6NqrgJ4FHgFmAL8C3gW+CqhBf4qsC5wfg+mXQg8HU3bBkwlXGDvOuDH/fQ+sjGSkOhq4HpJVTGGUjQ8yeWIpO9SyYHsR1Ue1nBLlQK7UUNF5ibRIjj9Ikjmy01EBdRGz5PRQ8Ae8MWvyTaEW1xkO20J4XgZIyS3BHARcEL0PE5fBXaBuspQbLo+8iSXA5I2IMFfOIRqKuKOJgvjgQQbStox3WAze1Pw8BWhyMkLbcBmwIqEJLB1p+HXAhN6MG0d4YTezQkXbBkKPE+G00JicAVUl8AvJOXiCoODmu9d7SNJNZQzjb0Yx5cK6EfjRYyHeNaabLt0gyVtPAT+NwuqagY6ti4sIPS/XQ5sHL32W+AF4C6Wv1Njd9N2OBo4DngReIjwO3B6jmLurdMgeQU8uNBs35hDKWiFs1Hmq3IuZR1WYvMC+yw3RZSyqaRt0w02s2mCyVeFe7LkjWHALoS+NYDrgX8AN9N1gks3bYeXo7/rAjcAtxP6+t7pY6x9dSokyuGrknaIOZSCVlgbZp6RtDulHMzeed4Pl04ZsCtVVHBBplEWwqnnQnPcJ1R+ytI9p0uA/xD2qv4L+B1wH1Ddw2lTnQGcTeiv62iflxD/fRtrgIuheghc6Zdk6j1Pcr0kqY4Ek9ifagp1H9jmiBK2lLRlusFmNsXgqatjruZmA7sSmpBbEfrV9gaOBxZH/28GHBuNPwv4WjfTdrgnen0ModLbFtiEUBVu2i/vpmcOAUaEKwN+Pe5YCpX3yfWSKvQX1uMwDijYFBc8RzuP8pg12VfSDZa0xQrw31lQ5Xdgice9wGEwfTGs41cS7jmv5HpB0raUcBgTCjzBAXyJEsR2kjZPN9jMXmyH564JR1u4GHwTWCPsHD4o7lgKkVdyPSSphApe4+usz/i4o8mRp2nncR62Jkt78r6krUfCozOhuhCOkClGjwHfhDn1MDbd/TpcZl7J9dxhDGMsm8QdRg5tSQmwk6S078rMnkvCy9d5NRebXYFVw76Ir8YdS6HxJNcDkmpI8Ef2prbg9qZ2pRzYkXIqOC/TKAvhlDNhiV8aIz4/g7qhcFLccRQaT3I9UcaprE0Fq8YdSD/YilKMr0jaIN1gM3uqCabd6NVcbA4CkrCjpDFxx1JIPMllKTq95mfsmfGQrMJWAexAggp+m2mUhfDL06Exb87cH2TqCImuPJyg4bLkSS5b5ZzFVpQyLO5A+tGXKcWYIGnddIPN7PFGeGvSQMflvnAcVCbgeEm+7WbJP6gsSBpDO4ewPeVxx9KvKoHtKKOCczKNshB+eSo0+O69eHwJWDV8U3vEHUuh8CSXjQS/ZnP0xTV7itnWlNHONyWtlWGMRxfDe363lficCLXD4Gdxx1EoPMl1Q9JIjO+zQ0FcRKnvqoBtKKOcs9INNjOLqrl6P/Q+HoeAmmFnSSvHHUsh8CTXnTJ+zsZoQO96H7dtKcM4QNLqGcZ4aD7MuGsgY3JfiHZAmO+AyI4nuS5El5/+CTsyuE7brCbshChnYrrBZmaL4Je/9mouNsdBZTn8xHdAdM8/oK59i1UwBuPdlrcjgfEdSZmOCnzgE5h934AG5TpsAdSFzoViOvemX3iS60olJ7PNoNjdsLwawule5ZyRbnBHNfcrqPejg+MxAcoEu8UdR77zJJeBpI2BtUh7xNggsT3ltHNYF0fY3zcbPnlgQINyHfYMNwHfN+448p0nuUzKOYGtKKc07kBiVAt8CZHgtHSDzax9EZzi1Vw8dgXq4cuS8vcWmHnAk1wakipp51C2yOP7pw6UHajAOFLS6Axj3PURfPbvAQ3KAYwCxoY7K6a9srMLPMmltwcr0lrUp3BlawiwGSUk+FW6wWbWvhh+7dVcPL4OlWWQ9qrOLvAkl04Fh7MpdXGHkTd2pALjB5JGZRjj9vdg4aMDGpQD2APKh+TP7WLzkie5TiSV08YENiiqK8b1zVBgPCKR/o7uZtZWH1VzAxzZoLcTsBjGSxpcx3L2gCe55e3OCFoZEncYeWYnKjF+JGl4hjEmvQn1jw9oUG4IsC40EW405tLwJNdZBYexmTdVlzMM2AhRxsnpBptZawOc+muv5gbcN6C63K9KkpEnuRSSRDtfYz1vqqa1M1XACZKGpRtscNOr0PjkwEY16O0OZbV+X9aMPMkta33KKSNTg2ywGw6sTwml6e8zYGbJRjj9VK/mBtQGQBOsFncc+cqT3LJ2Y03/TLq0C1WIn0lK22tpcP1L0PTcQMc1iK0MtEBNdEEJ14lv0Kkq+QZrF8ENo/vTSGBdSijlp+kGm1nLEpjo1dzAKQFWhEZg9ZhDyUue5CKSSmhlB19NsrAr1YhfSkp78YJ2uOZZSL440HENYqtDO7BG3HHkI09yS21MFTaoLo7ZW6OAtSmhhOPSDTazpmY46zRoGODIBq11w91zV487jnzkSW6pLRgbdwgFZFeqKeFUSWlv0dgGV/8XWqcMcFiD1XpQVQnrxB1HPvIk1yHBlxlLTdxhFIyVgDUopYQfpRtsZkta4NzTvZobEGsA1bBh3HHkI09yHcrYhtF+fFyP7EYNpZyeaa9eK/z5UWibNtBxDUKrAwZrxh1HPvIkR3QQcAvrsVLckRSYlYFxJBDHpBtsZg2tcP4ZYc+f60drAEvCN+I68SQXjCOBDdILnffNbtRQxsRMJ4gn4cp/Q/sbAx3XIDMKaIeKTMcvDmae5IKNGUUy7iAK0irAWBKIo9INNrPF7XDRmV7N9SsBK8ISYFzcseQbT3LBaoykPO4gCtZu1FLGWZLSfobNcOn9YO8MdFyDTFU4Vs4vudSJJzmAUtZgBV85em1VYGUqgcPTDTazhQZ/nBgqDddPol+YRLxR5B9PcgDlrM8w37PaJ1+hlnJ+KyntRtYMF98FNn2g4xpEog/eWySdeJILVvczHfpoHLAi1cCh6Qab2eeCy87yaq7feCWXnt+NCqCNMZ7kcmB3apnEeZJuMrPWzoOXwO//Didshm+J/eFTqMArueUM+iQnSYhhfvhIDqwOjKCO2RwE3NR5sJl9ViEdcSbsOeCxDQLt0Aw8G3cc+UZmg/tGcpJqKOVzzvDiIifeA25jJi2sbmZtcYfjnPfJwTASfoxczqwJrMAw4FsxR+Ic4EkOYAgJvOLIFdGxp/UCSb5+udj5SgjVJGiPO4iisjYwlBHAfnGH4pwnOajy3rgc66jmKrjQqzkXt0G/dxUo81TfD9YFalmRZvaX9EDc4RSpZjPzVkg3PMlB0nvk+oGAvajlVm7B/GySnDNKqOBmMpxK55byJAet3iPXT9YBPzSnn7wB3MeqcYdRCLyh5knOFaJweKu3QbLgSQ6StHtzyhWYkN5aYo6iIHiSC5WcJzlXWJqAdubFHUYh8CQHC2j2vklXYJYArcyNO4xC4EkO5tFChffLuYKyhCRtXsllY9AnOTNroYRmmuKOxLkeaCAJfB53GIVg0Cc5AMpY6LdZcQWlgVY8yWXFkxxACZ/5fd5dQanHwPvksuFJLphDfdwhONcDCygH/JYZWfAkB5BkKvMZ3FcPdYWjBUiSAGbFHUoh8CQH0MprzPUbrLgC8TmQYK6fnJ8dT3LB23ziVwd2BeJzoIT34w6jUHiSC97ic7+5tCsQnwOtvB53GIXCk1zwCe2YH0biCsLHNJLkxbjDKBSe5AAzM8qZ7jvkXUGYSSt4ksuWJ7kOrUzmY9/D6vJcC7CIKmBa3KEUCk9yHZI8wft+tJzLc3OAct43s+a4QykUnuSWepaZlHkt5/LaLKCdp+MOo5B4klvqI9ppYWHcYTjXhQ+pp4Un4w6jkHiSi5iZUcYLzIg7EucyaAemUwo8GncohcSTXKol3Mc7fuaDy1OfAMZCM/MDgXvAk9yyHuRt8H45l5fewwC/h20PeZJLYWbvYixgTtyROJfGmyymhfvjDqPQeJLrrJ17eNsvhu7yTBKYRSUwOeZICo4nuc6S3MPrfrycyzPvAgmmmpnv/+8hT3LLe4J5lLM47jCcSzGFBpq4Ou4wCpEnuU7MrIky7mOa735weaIZeI8y4K64QylEnuTSaeZqXvAmq8sTbwNlPG9mfgvCXvAkl95kFtHKJ3GH4RzwMotp4q9xh1GoPMmlYWZtGNcxxa8W7GLWAHxIOXBv3KEUKk9ymbRyLS+T9INJXKxeoo0y7va9qr3nSS4DM5tGO9N5M+5I3KDVBjxDM81cFHcohcyTXFeaOZv/+sEkLiZvA+1MN7OX4g6lkHmS69rdzKOZj+MOww1KT7GYJs6PO4xC50muC2bWShsX8aTf4sYNsE+AubQDd8QdSqHzJNeddq7mHcSiuANxg8pkGmnnYjNriTuUQudJrhtmtoASruNJ/Jr6bmB8CrxNO21cEncoxcCTXDZaOJuXaPdqzg2IR2nE+J2Z+RqXA57ksmBmcxB/43Gv5lw/+xR4h3bauDTuUIpFWdwBFIwkZ/MK32d7YHjcwUSWAPcROqkF7AM8C3Sc4dgEVAI/6jTdPODvKf9/DuwKbAs8DLwDjAb2j4a/Ei1rm5y/A9fZIzRiXGBmfuhSjsjML7aRLZXpTNbhlxxEddyxAHA3MA7YAmglXFixKmX4v4EKYJcu5tEO/AE4hpAQbwGOAu4EdiAk9FuA7wKlOY3edTYTuJ4FJBnnSS53vLnaE238nvdo5qO4AyFUaR8CX4r+L2PZBGfAa8Am3cxnOiGRDSNUg23RtEnC2vEUsDWe4PqbAf+gniQneYLLLU9yPWBmDST5MffQQFvMwXwOVAP3AFcRTt9OPdjgQ6AGGNHNfKYBG0fPK4ANovmtQKjsZgHr5ypol9FUjPl8DFwfdyjFxpNcz91GA6/yXMxprh2YDWwFHAuUwzK3HJ5G91VcK/AWsFHKazsQ+vD2JNzdc1fgReB24PFcBO6W0wQ8SBMtHGFmfkmIHPMk10NmZjRzBI/RQpzXhRgSPcZG/29ISHoQmpxvsGzySuddYGWgNs2wjnmNIOx4+DZhB8dnvQ/ZZfAIzbRzl5k9G3coxciTXC+Y2dsYl/JgjKd71QFDWbondTowKuX5yGh4V6aSudrrqOI6+ugg9Nn5FfZy6yNgCk20cGLcoRQrT3K91crZTGcBr8UYwwTCXtA/AXOAHaPXU/vZOiwCbkr5v4WQDDdIM983gFUIlWIVoVr8EyHJjc5R7C7cu+HvNJLkCL+0ef/xQ0j6QNLWlPMYx1PFkLijcQXnPpqYxn3WbN+JO5Ri5pVcH5jZc7Tze+6gwa8g7HrkPWAq9bTww7hDKXae5PqqlbOZw7s8G/tBJa5QNAJ30kiSg81sQdzhFDtPcn1kZq20sD+P0cScuKNxea8duI1GklxjZv+JO5zBwJNcDpjZdJIcw000+uU1XZceJclsppHkpLhDGSw8yeWImU2imb8yiUZvuLq03gSeYxEtfMPMWuMOZ7DwJJdLSX7OXF7gX35JJtfJZ8BdLCHJ3mbmty0fQJ7kcsjM2mhhX6Ywj5d9f6uLNAI30EArP/ezGgaeJ7kcM7PPSfJVHqSR6XFH42LXQkhwjfzV2uzPcYczGHmS6wdm9gZJ9uZWGv12hoNYG2FP6nweJMnP4w5nsPIk10/M7HFaOIgbWIL3wAw+RjijYQYv0sKhfnWR+HiS60dmdj8tHMt1NLIg7mjcgDHgYVp4g/dp4Wtm5pc1iJEnuX5m7XYDzZzGNTTGemkmNzAM+BfNvMAHtLCTmdXHHdJg50luAFirXcISJnI1jX49tiJmwAM08zLv0cK2fmWR/OBXIRlAKtUPKeePHEkVK8UdjcupduB+mniNt6MKzuv2POFJboBJOogKruUwqr64qq8rbK3A3TTxDq/Twi5+I5r84kkuBpL2ppzbOJBq1o07GtcnjcDNNPIp/6WFA8ysIe6Q3LI8ycVE0nYkeIBdqGU7ylDcEbke+wy4nkaW8DeS/MwPE8lPnuRiJGkc5fyH9RjLPlRRFndELmsfAJNYQpKTrM2uijscl5knuZhJqqGc2xnOznyXmrR3znL5w4BnaeNRlpBkfzN7OO6QXNc8yeUBSSWUcQ5l/IxvUcVacUfk0moE7qKRj/iIFvY2s/fiDsl1z5NcHpG0OwluZwtq+Qrl3nzNIzOAW2mkhRtIcqKZ+eW0CoQnuTwjaSTlTGII23IQNYyMO6JBrg14mjaeoJEkh5vZPXGH5HrGk1wekiRK+DGlXMhXqWRLSvzclBjMBu6kgcW8SjMHm9mHcYfkes6TXB6TtAEV3MIKrMP+1LBi3BENEklgMkmeo5k2forxf+YbSsHyJJfnJJVQwrGUcCFbkWAXyqmIO6oi9j5wN4008zjNHGVmfg+2AudJrkBIGk0FV1DKBPakmk3wyyvk0nzgnzTyAQ0k+aGZ3R13SC43PMkVGEk7UsGfqGF19qKWdcDPluiDBuBxmnmJNowLaOP3ZrYk7rBc7niSK0CSBHyTci5lJCPYi1rGxR1VgVkCPEcrT9GKuIUWTvOmaXEqqAaPpJUk3SJpuqQXJT0jab8czXuypC3TvJ6QdIGkdyRNk/Q/SRP6sJwjJF0RPT9W0uEpr4/JZh4W3EsLazGLn3Aj87iWet4lHJHvMlsE/IsWLqaJZ7iTJJtas33fE1zxKpjDTaPq5R7gejM7JHptNeCb/bzoc4CVgY3NrFnSSsDOaeIrNbMe3VbabJlzHo8ApgGzejB9G3CdpEl8xMH8nTOpZiQ7U8vGFNC3OwA+A/5LE9MwxPUkOd/MPoo7LNf/Cqa5Kml34DdmtlyCiYZXAn8GtiRc4eskM3usi9ergP8DNgTeAFYHjjOzF1LmWU041n0NM1uUZpn1wMXAnsDPo3mcAJQDzwE/NrM2SUcCvyYcefU20Gxmx0uaCNQTTve+DviY0JDatjf9QtEPwR5UciawKdtQwWaUMqyncyoSrYS71j/LYuYg4HJaudiv2Du4FNJv/UbAS10MPw7AzDaRtD7wkKR1u3j9R0CjmY2XND7DvNcGPkqX4CI1wDQz+42kDYBTgO3NLCnpT8Chkh4GzgK2ABYCjwEvp87EzO6QdDxwcmqS7anoWK5/A/+WNJ6nOYEnOYjRtLMVdWxASL/F7hPgBZqZglHCKzTxR+AePxVrcCqkJLcMSVcCOwAtZrZV9PxyADN7U9KHwLpdvL4TcFn0+quSXu1FGG3AndHz3QmJ7PlQUFFF2Ny2Biab2adR3LdFy+9XZvYqcLSk45jJN5jH8dzPNmxAOxtRxZoUT8IzYC7wOm1MpZF6WjGuppWrzcxv8T3IFVKSew04oOMfMztO0kigo/LJdCBFVwdYdNdWfxcYJ6kuwyWtm1L64UToL/z1MguX9s1iOf0mql7uAO6QNJqpHMQ7fJckm7AqzWxEHesCQ+OKsJeShMb9G7QwjSRJGjBuJcntwLM97R91xauQ9q4+ClRK+lHKa9Upz58ADgWImqPjgLeyfH1jYHznBZpZI3ANcJmk8mjclSV9N018jwAHSloxGm94tGPkOWAXSSMkJYBvZXh/i4G67j6EvjCzOWZ2iS2xLWljJT7gGB7hLi6ngT+ymHtYwsuETvp866ptAaYDj9LGX1jIBSS5lbd4gQtpYHtaGG0t9lMze8oTnEtVMJWcmVlUFf1R0i+BTwmHcp4SjfIn4CpJUwldzkdEe0Mzvf5n4P+iZuoU4H8ZFn06cC7wuqSmaJm/SRPf65JOJ/T5lRBqjePM7NloB8MzhB0PLwGlaZZzXRRnr3c89ISZLQBuA26TVMZCxjOF7XiTPWlnW6CGsbSwCrWMooRRwAj6v4lrhHQ/l9DYn0UDs2ljIVUkeJsk/6SNR4CnrNVvGOO6VzB7V93AkjQW2BaxCZVshbEhLYyhkiZG0MZQyhhCJXWUUkvYBVMNJAg/nWWEBnwJ4XZ9LYS0n0x5Xg8swlhAE5/TwkJEPRUYSRK8RSv/I8mLwFTCDp7Ggf4cXOHzJOeyJqkUWANYBxgNjCbBOMoYB6yMMRyjknbKMRJACe2UItoooZkSmhBLEA2E6+x+TDNv084MYGb0mAHM86t+uFzxJOecK2qFtOPBOed6zJOcc66oeZJzzhU1T3LOuaLmSc45V9Q8yTnniponOedcUfMk55wrap7knHNFzZOcc66oeZJzzhU1T3LOuaLmSc45V9Q8yTnniponOedcUfMk55wrap7knHNFzZOcc66oeZJzzhU1T3LOuaL2/0zqHt3iWM9fAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#Graph the predictions as a pie graph\n", "from matplotlib import pyplot as plt\n", "\n", "slices = [Predictions.count(0),Predictions.count(1)]\n", "labels = ['Bad Credit', 'Good Credit']\n", "colors = ['red', 'green']\n", "explode = [0, 0.1]\n", "plt.pie(slices, labels=labels, autopct='%1.1f%%', explode=explode, colors=colors, wedgeprops={'edgecolor': 'black'})\n", "plt.title(\"Credit Risk Ratio For a Multi Optimized Model - All\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "36" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#Now let us take a look at Foreign worker predictions only. the last element in the list represents Foreign workers (1 means foreign 2 means non-foreign)\n", "filtered_dataset=[]\n", "for i in range(len(test_dataset)-1):\n", " if test_dataset[i][-1] == 1:\n", " filtered_dataset.append(test_dataset[i])\n", "len(filtered_dataset)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "36" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(filtered_dataset)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "#Make Predictions on the test dataset\n", "Multi_F_Predictions = []\n", "for i in range(len(filtered_dataset)):\n", " Multi_F_Predictions.append(predictor.predict(filtered_dataset[i]))" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "#Turn each item into a single dimension element in the list \n", "for i in range(len(Multi_F_Predictions)):\n", " Multi_F_Predictions[i]=(Multi_F_Predictions[i][0][0])" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['0.24157475',\n", " '4.7756188e-05',\n", " '0.06342731',\n", " '0.93585235',\n", " '6.523058e-05',\n", " '0.5261048',\n", " '0.7142568',\n", " '0.014334942',\n", " '0.017389935',\n", " '9.352473e-05',\n", " '0.6630546',\n", " '0.4402347',\n", " '0.044109173',\n", " '0.13703965',\n", " '0.13513479',\n", " '0.09022986',\n", " '0.27975675',\n", " '0.7313284',\n", " '0.8185003',\n", " '0.123835415',\n", " '0.042360604',\n", " '0.000822239',\n", " '0.0046404405',\n", " '0.03956926',\n", " '0.932917',\n", " '0.76213884',\n", " '0.00038311264',\n", " '0.12971416',\n", " '0.90214473',\n", " '0.0013121088',\n", " '0.0013635819',\n", " '0.8794973',\n", " '0.026009575',\n", " '0.08722933',\n", " '0.85275924',\n", " '0.0009555685']" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Multi_F_Predictions" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "#Each prediction value is between 0 - 1. But the target variable \"Credit Risk\" is either bad (0) or good (1) there for we will round up/down to ensure every value is either 0 or 1\n", "for i in range(len(Multi_F_Predictions)):\n", " if (float(Multi_F_Predictions[i]) >= float(0.5)):\n", " Multi_F_Predictions[i] = 1 \n", " elif (float(Multi_F_Predictions[i]) < float(0.5)):\n", " Multi_F_Predictions[i] = 0" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "25\n", "11\n" ] } ], "source": [ "#Check how many predictions were 0 (Bad Credit) and 1(Good Credit)\n", "print(Multi_F_Predictions.count(0))\n", "print(Multi_F_Predictions.count(1))" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVcAAAD3CAYAAACpWhdCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAt60lEQVR4nO3dd5gb1dXH8e9vvV5vcwMbY7CpxnQwJfRO6L0TTC8JJSGQhFSSEJJAAknehJJA6Jgeek3o3TSDC81girEBG4zb9l3tnvePOwtikXa1tqSRtOfzPHpWK41mjqSZozt37pmRmeGccy67yuIOwDnnSpEnV+ecywFPrs45lwOeXJ1zLgc8uTrnXA54cnXOuRyIPblKWkWSSSqP/n9I0jFZnP8vJV2ZwXRPSjoxW8vtDUmXSfp1HMsuJJKOlfRsN89vK2l6nmNa4mVKWklSvaR+WY7pQ0nfzuY8s7XMrttzoZL0hqQdcrmMjJKrpCMkvRKtKJ9GCXCbXARkZnuY2XXRcrvd2KJpnpTUHMU2T9KdkkYmze88M8tq0oxWtKZomXMkXSupNsPXfuM9mdnJZvb7JYijc0WuT7pN6e18lpakHaI47uzy+IbR408u4XxN0pjO/83sGTNbs5vpJeksSe9G389Hkv4kaUCultkdM/vIzGrNrH1JXr8konXRJO3b5fG/R48fm69YlkQUf2uXdfqwbC/HzNY1syezPd9kPSZXST8C/g6cB4wAVgL+CeyXZvo4frG+b2a1wBigFvhLHpa5T7TMccBGwC/ysMx0hkQbca2ZbdjbF2fpO/sc2ErSskmPHQO8k4V5Z+oi4LvA0cBAYA9gJ+C2PMZQCN4hfPbAl9/vIcB7sUXUOxckrc+1ZnZrb15cKK3mbpOrpMHAucBpZnanmTWYWZuZ3WdmZ0XTnCPpdkk3SFoMHCtpsKSrolbux5L+0LlrJKmfpL9Ercz3gb26LPNJSSdKWhu4DNgy+vVa2NObMbOFwN2EhNc5v3Mk3RDdr4zi/ELSQkkvSxqR4n2PlDRV0k8yWOYc4H9dlvlzSe9JqpP0pqQDosdTvqfo1/oPSa8/SdIMSfMl3StphZ7iSPEeVoheOz+a10ldPpOvfWcpXr+XpNckLZY0S9I5PSyylfDZHx69vh9wKHBj0jy/scuoNN0xkp6O7k7pbL1ELeTZad7vGsCpwHgzm2hmCTN7AzgI2F3STtF01yp0wzwSfT9PSVo502Uq7LWcFa0fDdF6PkJhb65O0qOShnZ9v5I6v/POW7OkD6PpypLWmS8k3SZpmaRlHiVpZvTcr3r4HgDuA7bujAPYHZgKzEmaZ5mks6P5fibp+mh773GZPcWbK91tF9HnfJqkd4F3o8f2ljQ52tafl7RB0vRfdnNIqpJ0naQFkt6S9NMU3/lPou98kaRbJVX2FG9PLdctgUrgrh6m2w+4HRhC2JiuAxKEluRGwK5A5wZ0ErB39PimwMGpZmhmbwEnAxOjX68hPcSAQqvpQGBGmkmOAQYDo4Flo/k3dZnHKsBTwCVm1mMLWNIoQgspeZnvAdtGy/odcIOkkZm8pygJnE9ITCOBmcAtPcWRws3AbGAFwmd8nqSdk57v+p111UBoAQ4h/ACeImn/HpZ5ffQagN2AN4BPliB2zGy76O6GGbZedgZmm9lLXeYzC3gB2CXp4fHA74FhwGSi99+LZR4UzW8ssA/wEPDLaH5lwOkp3k/nd14LDI1iujl6+nRgf2B7wve1ALgUQNI6wL+Ao6LnlgVG9fBZNAP3Ev3QEb6T67tMc2x02xFYjbDHd0mGy0wbb65kuF3sD2wOrCNpY+Bq4HuE+C8H7lXqLqLfAqsQPoddgCNTTHMo4UdqVWADUjRIuuopuS4LzDOzRA/TTTSzu82sAxhESDZnRC3dz4D/46sv+lDg72Y2y8zmEz6wpXWRpEXAPMIK/oM007UR3tMYM2s3s0lmtjjp+XWAJ4Hfmtm/e1jm3ZLqgFnAZ4QvCAAz+4+ZfWJmHdEG+i6wWYbvZTxwtZm9amYthO6GLaOkn8686Nd5YfQLOxrYBviZmTWb2WTgSsLG0unL78zMmrrO0MyeNLNp0fNTCYlg++4CN7PngWUkrUnqDTqXhgGfpnnu0+j5Tg+Y2dPR5/srwuc7uhfLutjM5prZx8AzwItm9lo0v7sIDYfuXET48epsEX4P+JWZzY7mcQ5wcNTCPxi4PyneXwMdGcR4PXB01BrdnrBXkWw88Dcze9/M6gnr2eEZLrO7eLPhJ0nr87ykeHvaLs43s/nR+nwScLmZvRht69cBLcAWKZZ3KHCemS0ws9mE76eri6Jtej5hz2BcT2+ip+T6BTAsgw9tVtL9lYH+wKedHxDhV2O56PkVukw/s6cgM3C6mQ0m/KIMJf0v+wTCLvwtkj6RdIGk/knPjwc+JrToerK/mQ0EdgDWImnjlXR00u7IQmA9vr5xd2cFkj6TaMX/Alixm9cMM7Mh0e0v0Tzmm1ld0jQzu8xjFt2QtLmkJyR9Hv1wnZzhe5gAfJ/QIuppjyeb5hFaNKmMjJ7v9OV7jz7f+YTPLFNzk+43pfg/7cFNSd8jrDNHRI0RCNvMXUnry1tAO+EYx9e2FzNrIKwP3TKzZ4HhwNmERNn1B/Rr61l0vzzDZXYXb1qSxid1izzUzaR/SVqfO9e5TLaLrnnox0lJeiFhjzXV99w1J6XaNuYk3W+km++4U0/JdSJhF2P/HqZLPrXWLMIvRPIGP8jM1o2e/5TwJjutlOF8e2Rm04A/AJdKUorn28zsd2a2DrAVoXvi6KRJziFshDcpw+EzZvYUcC3RQTSF/rsrCAlm2WjX/3WgM56e3tMnhBWDaH41hNb2x5nEkzSPZSQNTHpspS7z6CmOmwi7lqOjH67L+Oo9dGcCoe/zQTNr7PJcQ/S3Oumx5TOYZyYeB0ZL+toeQtQi3QJ4LOnh0UnP1wLLsITdF70haVtCd8R+ZrYo6alZwB5J28sQM6uMWsZf214kVRPWh0zcAPyY1HsQX1vPCOtHgvBD0dMyu4s3LTO70b46SLVHhu8hZbxptouueeiPXWKsNrOb+aZP+XqDrDd7MWl1m1yjFeA3hGS1v6RqSf0l7SHpgjSv+RR4GPirpEFR5/fqkjp3KW8DTpc0Kupw/3k3IcwFRkmq6MV7uo7QSt636xOSdpS0fpQ4FxO6CZKHybQRjqrWABMkZToO+O/ALpLGRa81wtFzJB1HaLlm+p5uAo6TNC7qHzqPsNv5YYaxdPYzPg+cr3AQbwPgBFL3raYzkND6bY4S1hEZLvsDwm7oNw68mNnnhI3hSIUDm8cDq3czu7mEfrBMlvsO4QfgRklbRPNfF7gDeNTMHk2afE9J20Tfwe8Jn29nayXjZfZGlORvBY6OYk12GfBHfXVgbbikztE4twN7J8V7LpmPT7+I0If4dIrnbgbOlLRq9ANzHnBr1AXY0zK7izdXertdXAGcHO2BSVKNwkHagSmmvQ34haShklYkNIyWWo9fkpn9DfgRYffic8Ivwvf5Zh9OsqOBCuBNQmf37Xy1y3YFYdd8CvAqcGeqGUQeJxwUmZPU99JTvK2ElSrVoPzlo1gWE3ZlniL8und9/YGEBH11Jgk2ShrXA782szeBvxJa/XOB9YHnMn1PZvZYFPsdhF/U1fmqv7o3vkPopP+EsHv+WzN7pBevPxU4N+pX/g29GM5kZs+aWbqW4EnAWYRdunUJPwLpnANcF+3WHZrBor9P6Fu+AagH/kvoQz+oy3Q3EfrI5wObELqDlnSZmdqZaP1L2jV+I3ruH4S9hIejz/sFwoEZLIx4OC2K+VPC9pRyxERXUf/jY2YpT9p8NWEv42ngA8Ie6g8yXGbaeHOlt9uFmb1CWNcuIcQ/g/QHoc4lvL8PgEcJOaJlaWNW6s/dudIk6VrCqIKz447FFSZJpwCHm1m3B3B7Env5q3POxUlhXPvWURfmmoR+6qU+GFsQlQzOORejCsKIplWBhYTxs/9c2pl6t4BzzuWAdws451wOeHJ1zrkc8OTqnHM54MnVOedywJOrc87lgCdX55zLAU+uzjmXA55cnXMuBzy5OudcDnhydc65HPDk6pxzOeDJ1TnncsCTq3PO5YAnV+ecywFPrs45lwOeXJ1zLgc8uTrnXA54cnXOuRzw5OqcczngydU553LAk6tzzuWAJ1fnnMuB8rgDcK6TpCpgGLAsMAQYFN36AZ3XgLfo1gHUAfOTb2bWlt+onUtNZtbzVM5liaQVgHWB1SpgbC2sZzCmEUa2Q8VAaBkCiaFgg4HBUNYfBF9lVAPrABZBxzxgAZQthvIGGFAOrdUwtx/MqIepLfA2MAN4F5htvsK7PPHk6nJG0nBg03LYfBDs0AzjymDAWtCyFvRfG6pWB60OrAYMJcqiSyhKuHzIl9nU3oKmt6DtXahog0Q1TF4EjyfgReBlM5u3lG/TuZQ8ubqskVQNbF8N+5TDPm2w3PrQtD3UbA7lmwIrsXQJdEkZ8DHwMjAREk9Bw1So6g9fdMD9DXAP8KSZNcQQnitBnlzdUpG0chkcMgQOqYdx60LzgVC7O5RtTGEfMe0ApgL/hY47oH4qVNbCawvgNoN7zOy9uGN0xcuTq+s1ScMUEurJrTD2YLD9oWonwtGnYrUYeBy4F5rvACuDDxbBvwxuNbPP447PFRdProCkdmAaYY+1Hfi+mT3fi9efA9Sb2V9SPHc08NNo3gKuTjVdhstZBbjfzNaTtClwtJmdLmkHoLU3MS/BsvsDBwyB05ph8z0gcTzU7ApU5GqhMWoDHgWuhob7obwSXlwIlwF3mllLvNG5YuBDsYImMxsHIGk34Hxg+6WdqaQ9gDOAXc3sE0mVwFEppis3s0Rv5m1mrwCvRP/uANQDWU+ukoZXwCk1cMa6UH46DNwPqIUB2V5WIekP7BFuNfXAPbDdP2GjyfCvSuniFrjEzObGG6UrZIXcJRaXQcACAEm1kh6T9KqkaZL265xI0q8kTZf0KLBmmnn9AviJmX0CYGbNZnZF9PonJZ0n6Sngh5I2kfSUpEmS/idpZDTdJpKmSJoInJa0/B0k3R+1Zk8GzpQ0WdK22fgQJG00WLqlEj46DH7+HAx9EQaOB2qzsYAiUguMB56DgS/D4PHwk0r4cLB0i6RxMYfnCpWZ9fkboStgMmFM5CJgk+jxcmBQdH8YYYSPgE0I3QjVhGQ8g5BEu853PjA4zTKfBP4Z3e9PaHUOj/4/jNB9AOGYy/bR/QuB16P7OxC6CADOSbX8JfwsNhoMTywDDX+ExOdg5rdv3OaB/RESy0DD4PDdbRX3euy3wrp5yzVoMrNxZrYWsDtwvaTOPtLzJE0ldMGtCIwAtgXuMrNGM1sM3LuEy701+rsmsB7wiKTJwNnAKEmDgSFm9lQ03YQlXE6PJK03WHpoCDx3Lmz3MVT/EvoNy9UCi9yywC+h3xyovhC2WA4eGSI9I2mTuGNzhcGTaxdmNpHQSh1O2BscTmjJjgPmApWdk2YwuzcIrdx0OsdUCngjSvDjzGx9M9s1ejynRxwlrTpYumsQvHQ27PIxVJ0OZZU9v9QRdjlOAs2C6vNg6yHwzGDpPklrxB2bi5cn1y4krUWoZf8CGAx8ZmZtknYEVo4mexo4QFKVpIHAPmlmdz5wgaTlo3kPkHR6iummA8MlbRlN11/Suma2EFgkaZtouvFpllMHDOzl+6ysks6thjd+DHvPhqqzoF91b2bivlQBnAqKPsfda2BKtXRBVFjh+iBPrkFVdDBoMmFX/RgzawduBDaV9Aohsb0NYGavRtNNBu4Ankk1UzN7ELgUeFTSG8AkUozQMLNW4GDgz5KmRPPdKnr6OODS6IBWU5r47yMk+4wOaEnaoxbe3xl+9BZU/QbKe5WZXVo1wNlQPh2qdoPTauADSXvHHZfLPx/n2odIGj0IrqyFra+Cmt3jDqgPeBg4Hhrr4NnFcJKZfRR3TC4/vOXaB0hSuXRcFbx1Juz0nifWvNkVmAHVZ8KO1fBmuXR8dLDUlThvuZY4ScMHwY3LwVa3Q82GcQfUh00DDoCGz+CpOjjKzObHHZPLHW+5ljBJu1TD9BNg+9c9scZufeB1qDkGdq6GdyTtHHdMLne85VqCJJVVwx8q4Ye3QvW34w7IfcPDwOHQ1AqXNcBZ0QFUV0I8uZYYSYMGwp1jYYsHoGZE3AG5tOYB+0HjNJhUB/uZ2YK4Y3LZ490CJUTS2FqYdhhs/bwn1oI3DHgKqo+CzWpgWjTG2pUIT64lQtKe1TDprzDqCqgsxdMAlqJy4FIY8A8YWQ0vS9oz7phcdni3QAnoL51YAxc9AFVbxx2MW2LPA3tBUyP8rMXs4rjjcUvHk2uRq5J+Pgh+/QxUj407GLfUPgC2gcYF8Ocm+L35Blq0PLkWKYWi9QuHwynPQvWouANyWfMpsC00zIErG+BMT7DFyZNrEZLUrxauXgUOegJq/LSApWc+sCM0vA931MPxPlSr+HhyLTKSVAsT1oMDHoZqP+FK6aoHdg+XAP9vHRxqZh1xx+Qy56MFiogk1cDFY2D/Rzyxlrxa4BGoWRt2r4V/+zkJiosn1yJSBb9fEY59HGr62nWs+qoqQoIdDYdXh/MDuyLhybVIVEo/HgZnPg01Q+MOxuXVIOCp0Lf+gyrprLjjcZnxPtciUC4duSxc/jJUrxR3MC42s4BNoXE+fL/N7Jq443Hd8+Ra4CRtVgtPTITq9eIOxsVuOrApNNXDrmb2bNzxuPS8W6CASVqhGh660ROri6wJ/AeqquE+SaPjjsel58m1QEmqGggP/xwG7ht3MK6g7A78BmoHwsOSquKOx6XmybUARWNZr/82rHZ2uHqzc1/zUyjfDVYeCDf4EK3C5Mm1AJXDSSvAHjdAlW81LhUB10PVKNitAn4Ydzzum/yAVoGRtGY1vPoKVK8ddzCu4M0AxkFjA2xuZq/HHY/7irdcC4ikioFw94VQ6YnVZWIM8HeoqoW7JQ2IOx73FU+uBaQa/rQFrHSKfy+uF04AbQcja+DCuGNxX/FugQIhaYeh8MB0qB4edzCu6MwDxkLTAtjHzB6LOx7nLaSCIKmyFm683hOrW0LDgJuhqgZuCYNNXNw8uRaAKvj19jBk7zwvdyFwMLAWsDYwEZgCbAmsD+wDLO7m9e3ARkBy3D8DNgCOTnpsAvCPbAXt0toN2BtqavwELwXBk2vMJK0hOPNfUJ3vZf+QMCD9bUJSXRs4EfgTMA04gO478f4RvabTIsJ1oKYSEu80oAm4Fjg1u6G7NC6GqjI4QdK4uGPp6zy5xkiSBsG1v4OKfNcxLgaeBk6I/q8AhhBq17eLHtsFuCPN62cDDxCScacyoBUwQlLtT0jOp+OVEPkyHPgLVA6Ca724IF6eXON12HKw4RnQL98Lfp+wIR5H2LU/EWgA1gPujab5D+FMTKmcAVzA11eggcBB0fxWBQYDLwP7ZTd014MTQCvCGMGRccfSl3lyjYmkATVw0dVQUx7D8hPAq8ApwGtADaE74GrgUmAToI7Qou3qfmC5aJqufgpMBv4K/Bo4F7gSOBT4QzbfgEurH3A11FTBP8J1LF0cPLnGpBy+uxlUbRvT8kdFt82j/w8mJNu1gIeBScB3gNVTvPY5Qut2FeBw4HG+2UR6Lfo7FrgeuA14HXg3W2/AdWsLYEeo6A/fjzuWvsqTawwk1VTA7/8aLpMUi+WB0YQ+VoDHgHWAz6L/OwgtzZNTvPZ8Qp/rh8AtwE7ADV2m6Wy1thEObkFY2RqzEr3LxJ/DXtHZkvxyazHw5BqDCjhjF+i/UcxxXAyMJwydmgz8EriZ0NpcC1iB0CcL8AmwZ4bzvRv4VvT6IXw1tEvAhtkI3GVkXWAv6FcJP447lr7IK7TyTNKQKpg9GWrGxh2MK3kzgA2goQlGm9mCuOPpS7zlmmcD4EcHQpknVpcPY4BDoKwq1He4PPKWax5JqqqCuZNgoJ/1yuVL1Hqtb4IRZubd3nniLdc8Ehy1FcgTq8unMcBWgEIXu8sTb7nmSVSN9eFdsNJOcQfj+pxHgQNhZh2sar7R54W3XPNnp2VhmR3jjsL1STsDy8KyhJFzLg88uebJYPjZT6HGi71dHAT8HGoGhxF3Lg+8WyAPJA2rhNmfwQAfze3i0ggMh+ZGGGNmH8cdT6nzlmseCA7bExKeWF2cqoEDwMpC1bLLMU+ueTAYTjkhnBvFuVgdC1UD4Xtxx9EXeLdAjkkaMwimfgFVcZz9yrlk7cAwaFoIG5vZ23HHU8q85ZpjFXDMkVDmidUVgn7AUdBvwNevxONywFuuOTZY+uh/MHqLuANxLjIJ2AHm1sNIH/OaO95yzSFJqxgM3yzuQJxLsjEwIFw4wk9xkUOeXHNrjz2gwz9kV0gE7AVSuD6lyxHf7nNoKBy6fwxXdXWuJ/tA1dBw9R2XI97nmiOSBgyAhR9D5bJxB+NcFwuBEdDSCkPNrCnueEqRt1xzZ9s1oNUTqytEQ4C1oQWI6zJuJc+Ta470hx338cIBV8AOgNqqzK/e43rJk2uODISdtwzDCp0rSFuGKxTsEHccpcr7XHNAUtkAqJ8JVSPiDsa5NOYDI0O/a42Ztff4Atcr3nLNjTUGQYcnVlfIlgGWDVc/XyvuWEqRJ9fc2GwL8F0CV/CiykGvc8kBT645UAPbbucHs1wR2BZqa2GbuOMoRZ5cc6ASNtogFMI4V9A2Biq+bMC6bPLkmgMtsOoacQfhXAbGAE0wOu44SpGPFsgySZXl4Rrx/fw0g67QdQCVkGiDZcysLu54Som3XLNvtRHQ6InVFYMyYIVwea3V4o6l1Hhyzb411vCRAq6IjAl/Vo83itLjyTX7Vl8HKuMOwrlMrQdVfJljXbZ4cs2yATBqNFTEHYdzmVoV+td4cs06T65ZVgUj/UxYrpgsAwyA5eKOo9R4cs2yfjDCk6srJkMBwbC44yg1nlyzb1lPrq6YDAUsNGBdFnlyzbIELOPJ1RWTZYB2GBx3HKXGk2uWtcHAoXEH4VwvDAVaYVDccZQaT65ZZtDPhwq4YlILJHyES9Z5cs0ygzI/Y4srJuVAh+eCrPMqzRzwtTQ3bgfmxB1ECWonNArijqPUeHLNMgN5yzX7pgFHQ1053BZ3LKWoEt6JO4ZS48k1ywzkTYDsuxbaDC5fbHZW3LE4lwnPAzngZ23JrnbgGmhrhmvijsW5THlyzbL+0OQnxcyuJ4AO+NjM3ow7Fucy5ck1y/pD/cK4gygxV0BjHfwz7jic6w1PrllWBosWxh1ECWkA7oV+HXBT3LE41xt+QCvLBAsXxh1ECbkLqIQXm8w+S/W8pM2B4fmNqs+YbWaT4w6iWHlyzTKDLxbGHUQJuRzqFsK/Uj0naQ3KeYpRNOc5rNLXgZhNBeFE2m4JeHLNshaY80XcQZSIT4GXwzp6T8oJ+nEMGyH28pOOZF0L8Gda4g6jmHmfa5Y1wHsfQlvccZSCG6FjANxjZk1dn5Mk+nESG3lNfE60A6I97jCKmSfX7Jv5DnwjGbjeuwwaFsPlaZ7emiqqGZnXkPoOT65LzZNr9s18L+4ISsBUYE7YOX065QQVnMSmVOO1xrnRAYhE3GEUM0+u2TdjFlR6ldbSuRba2uEaM+vo+pykSjo4mA18/c2ZNqCM1rjDKGa+cmbf/A5onxd3FEWsnZBcuyl33YsRtPthrBxaBJTxSdxhFDNPrllmZlYNM9+KO5Ai9jjQEcZYpv4YKzmFTRmY36j6mEWA4T1cS8GTaw60wXOvxh1EEYvKXdONbR1Ggm1YO99R9TELMVrwNsJS8OSaA/Xw3LOhctP1Uj1wX/flroexBgkq8xlVHzSfRowP4w6jmHlyzY1JL4bjra6XonLXFyxNuSuVnMrG1OQ3qj5oPglgZtxhFDNPrrnx5lwYsDjuKIpQD+WuYzFWZbU8B9UXLaIf8FHcYRQzT645YGaJgTDjtbgDKTKfAJOgH3BvygnKOZYN6Ue/vIbV93QAjVQBs+IOpZh5cs2RZnjiGe8a6JUboaM/3J2m3LWMMk5knJe75lwdUE69mfkJcZaCJ9ccaYQH7gnHZ1yGLoOGOvh3mqe3pppKL3fNg0VAOZ/GHUax8+SaO09PhUq/5EtmpgBzQ7nrMyknqOAkNqHGy13zYCEAH8QbRPHz5JojZtZQC1OfjDuQInEttLbD1d2Uux7k5a558hEtNPNU3GEUO19Zc2gR3PEgfk7MnkTlru3dlLvuzfJe7po379GCeXJdWp5cc6gd/ncvtPlJXLr3WPjzkZm9nXKCSk5hEy93zYsmYDEDgElxh1LsPLnm1pRF0DIt7igK3JXQuCjN1V2jctetWSffUfVRHwEVTDYzP+H7UvLkmkNm1tEB19/gVyZIq7Pc1eCWNJMczhokGJDPqPqwD2ijhQfjDqMUeHLNsSa4/lrvGkjrTqASJnq5a4GYQRMdPBF3GKXAk2vuTWmB+RPjjqJA9VDuuiawipe75kkLMJ9K4OW4QykFnlxzzEKZy1UTfNTAN3wMvBrKXe9LOYGXu+bXR8AA3vDKrOzw5JoHrXDDTdDh2fXronLXu7opdz3By13z6EMStPBA3GGUCk+ueWBmM8pgym1xB1JAjB7LXbehmiqWz2dUfdy7NNLBk3GHUSo8uebJQjjvfKjzA1vBFOCz0FXybMoJ/Oqu+bUQmE8/4LmYIykZnlzz58FZ0PRi3FEUiGtCuetV3ZS7Hsj6vn7mzWQSlHGz97dmj6+8eWJm7U1wwV+gMe5Y4pYAruu+3HUfL3fNIwNeoYVWLo87lFLiyTWP2uGqB0B9/XrFjwGCmWY2PeUEfnXX/JoJtDEPL3nNKk+ueWRmC/vBdef18WFZV0JDN+Wuw0mwlV/dNY8m0UQrl5qZHxLIIk+uedYAv7saOmbHHUhM6oD7obyHctd2L3fNkxbgLcowJsQdSqnx5JpnZjanDK44t4+2XqNy1+fN7POUE1RyKptQnd+o+rC3gHImmtmcuEMpNZ5cY9AAf7gB2vvidYszKHddmVXzG1Of9hJ1NHNJ3GGUIk+uMTCzzwX//C30qWEvs4HXui93Pc7LXfNoAfAZIt334ZaKJ9eYNMKfboNEXzrXa1TuemeqsZRe7hqDSSQo4yYza407lFLkyTUmZvZFG/zseGjoC4doo3LXxh7KXQd4uWueNAEvkqCVC+IOpVR5co1RAi5/B2bfGHJPSZsMzAubdOryygq+y6Z+dde8eYEEZdxrZu/FHUqp8uQaIzNrXwzHnQ5Ni+MOJseugdY2uDJNuWsVHRzgV3fNk2ZgIm20cHbcoZSy8rgD6OvMbOIg6e6z4aCLKM3RnQngemhvgWvTTLIPI2lnUB6DgnDxnWsIl5/tANYBdiQUKN9OOJnJEOAQoCrF65uAe4HPAAH7AaOBR4B3geWBA6Npp0TTb5GLN9JLL5JAPGhm78YdSinzlkIBqIMzroKWUj2py6OA4EMzeyflBHFd3bUcOAY4BTgZmAHMIpyna1Xg9Ohv6vN2wX+BMcAPotcPI7QKZwGnEjp75hKS+GTgW7l5G70SOmbaaOFXcYdS6jy5FgAz+7wRjj8YGhviDiYHMih33TKWclfx1b5Ce3QTMB0YFz0+Dkh1we9mQk3+xtH/5YTWraL5GCGplhF6mTeHghhi9jStwH/SntfBZY0n1wJhZncsgvt/WGJjX+uAB7ordxXfYWyM5a4dhJKGC4HVgVGES9J2tqMHAql+8RYA1cDdwGXAPUArIVmvHT02FKgEPgHWytUb6IWFwMu008rP4w6lL/DkWkDq4Lu3QF0pXdf4DqASnjOzeSknGMCpbBxjuWsZoVvgR4SLes3N8HUdwKeEXf2TgQq+6j7YJprnbsDjhH7cScBtwFPZCnwJPEIj8Hcz+zTGKPoMT64FxMwWNcAh46EpdeF98bms+3LXtYCVCuLqrlXAKoR+11pCk5vob6oLew+KbqOi/9chJNtknf8vSzigdSjh4NcX2Qq6Fz4BppMgwfkxLL1P8uRaYMzsqVa47GBoaI87mKU0G5gSehrvTzlBOccxjn6xrYUNhAM8EPpH3ycclFqTcACK6O+aKV47EBgMdLbH3weGd5mms9Xa2QcLoU+2bakj750EcDsNtHOGmdX1OL3LCh+KVYAa4aevwVZnw8bnQ/+441lSN4Ry1zua0pW79uf4WMtd6wh9ph2E5LcuIZGOBv4DvEZIoIdE0y8mDL06Mvp/D0K/Rzuhf3X/pHm/BawIXw4vG0U4pDcC8l6F9gSt1DMRSzsUzuWA/Py4hUnSctXwxg0w7IC4g1kCBqwK9TNhTzN7puvzkrZnKPfxQ7/iQE59BExgMW2MNbNMe5RdFni3QIEys88aYfejoHFq3MEsgdeAL8Jw/NTlrgP4Lpuk7M102dIK/IdG2jjeE2v+eXItYGY2qRFO2BUai+1MxtdAS7flru3s7+WuOfY/mmnhITO7I+5Q+iJfuQtch9kti+HC7aBhYdzBZCgBTICObspd942l3LUveQ+YSgOtnBh3KH2VJ9ci0AS/+wQmfBsaiuG63I8Agg/S1q5Xcqpf3TWHmoA7aKSNI8xsYdzh9FWeXIuAmVkDnDYdHtoPGvM9kqe3eih3XY4EmxdExVKpeoAmEtxiZg/HHUpf5sm1SJhZRz0c8SK8eCQ0faMjs0AsBh4M5a63ppxAfIc1/equOTMZYzoLaOX0uEPp6zy5FhEza6uDvR+CaYdDUyLugFKIyl2f7bbcdSO/umtOvA88QD1t7BJ2dlycPLkWGTNrrIMd/wsv7QuNhXZ97h7KXdcGRhVEuWupmQvcQhNt7GNmb8YdjvPkWpSiBLvrs/DELgV0msJZwNSey137+1qXZYuB62iilRPMLM5Tw7gkvpoXKTNrrYP9X4N7C2WYVlTueruZfaNBLakMcTzjirectyA1A9fSQCt/NLOb4w7HfcWTaxEzs0Q9jJ8OEzaChhlxxgJcHq7uekWaSbajlgq/umsWtQM30Ug9t5LgvLjDcV/nybXImVlHvdkpH8PPNoamx2OK41UyKHfd1Mtds8aAu2lmDi/QyvfMTxJScDy5lohWs0vrYK99YPHF0J7vLS26uusVqTZySdUk2J/1fX3LmidpYzrv08q+ZlaIA0f6PF/ZS4iZPdEIG/8SZp0Azfm6XkxU7treAtelmWRfViTh5a5ZYITE+jzzaGUnH3JVuDy5lhgze68eNrwdHlsfGt7IwzIfBsrg/W7KXeO5umup6QDuo5nn+YA2NvYzXRU2T64lyMwW18E+H8APN4PGS6Ajl90EPZS7jiDB5rFc3bWUtAE308jrvEYr3zKzYjtRWp/jybVEmZklzK5qhI1+Ce/sBo25uC7XYuChTMpd47veQPFrBK6mgZk8TCs7mNniuENyPfPkWuLM7J062PA5+Pca0DQBsnpY+XagEp4xs9SX3RvAKbFe3bXYLQL+TQPzuIZWDjKz1rhDcpnx5NoHmFlrg9mZi2CH02DGVtAwPUvz7qHcdR1gFKtmaWF9zVzgchqp41xrtR+kOvG4K1yeXPsQM3upDtZ5FX6zETSeDW1LM6LgI2BaWIceSDlBOcexkZe7LpEPgatoopGTLGEXxB2O6z1f7fsYM0u0mP2tCda8GB5dFRpvJByI7q0J0F7efbnrcV7u2ksdwPO0cwP1tLKfmd0Ud0huyXhy7aPMbPYisz3nwN6nwhtjoeF+wjDKjF4P/Bua6tOXu27PQCoYkZ14+4R64HoaeZI3SbCBmT0Sd0huyXly7ePM7InFsP57MP4I+HBjqH86g9dNAuZDA/B8ygm83LV33gcuoYmP+RetbGJmH8Qdkls6nlwdFtxTB2OmwKl7wdwNof4/hOqrVKJy1yu7KXfdz8tdM9AKPEgLN7OAZva1VvuJmRX6lXxcBnzld18ys/YOswn1sOJUOPYkmLICNP4NOpIHVrYBN4Ry12vTzGo/ViThNVk9+Ai4hEYm8xBtrGFmj8Ydksse+cl0XHckbTYIfpWAXceDvgsD5gJHwusLzNZP+ZoqPcOebMMGeQ62WLQBj9HKJJpo43gzuzPukFz2ecvVdcvMXlpktl8jjL0e/rQjzD0Q2nsod/2WX901hXbgVYz/o5HX+G/UWu02sUoaIekmSe9LmiRpoqQDshGOpCclbZri8f6S/iTpXUmvS3pJ0h5LsZxjJV0S3T9Z0tFJj6+w5O+gsJXHHYArDmY2CzhH0rnAukDq6zSJI1jLy12/xoC3gf/SQDPTaeEHZpb6QGASSQLuBq4zsyOix1YG9s1luMDvgZHAembWImkEsH2K+PqZWXtvZmxmlyX9eyzwOvDJUsRasDy5ul6JqoSmpZ1gAKf41V2TfAg8RD0LmBtd7vqhXpzYeiegNTkhmdlM4GIASZWE6rhNCccef2RmT3TzeBVwDbAO8BZQ1XWBkqqBk4BVO8cvR2ffui16vh74G7Ab8GNJqwCnAxXAi8CpZtYu6TjgF8CnwDtAS/T6cwiDzj6M4rtRUhOwpZk1Zfi5FAXvFnBZI2ldxIpe7grMIVzb6iY+Yy4n08pYM3uwl1cMWJdwkYd0TgOw0Pf9HeC6KLGme/wUoNHMNgD+CGySYp5jgI+6OTlMDfC6mW0OfAEcBmxtZuMIHR/jJY0EfgdsDexCSOZfY2a3A68A481sXKklVvCWq8um8ugChH35J/sL4DEaeZcECX6NcVm2TrYi6VJgG0Jr9lvR/YsBzOxtSTOBsd08vh1wUfT4VElTlyCMduCO6P7OhAT9cujBoAr4DNgceNLMPo/ivjVafp/iydVlhaQy+nNsnyx3bSP0qb5IHXMQ8HcSXGBmdUs55zeAgzr/MbPTJA0jtPgAlOZ16R6HnovwZgArSRqYJv7mpH5WEfqDf/G1hUv7Z7CckteX2xguu3ZgIP37VLnrHOB+WriQZh7gBWZzEgmGWZv9OguJFeBxoFLSKUmPJfdnPw2MB5A0FlgJmJ7h4+vBNwfLmVkjcBVwkaSKaNqRko5MEd9jwMGSloumWyY64PYisIOkZSX1Bw5J8/7qoHRHQ3vL1WXHAI5lFLW0QkmPFGgGpmG8SD2LaMO4nARX5KJc1cwsagX+n6SfAp8TSo5/Fk3yT+AySdMIB66OjY7up3v8X8A1UXfAZOClNIs+G/gD8Kak5miZv0kR35uSzgYellRGaMOfZmYvRAeuJhIOaL0K9EuxnGujOEvygJYXEbiskLQtlfyJBJuwOgnGUcMYKIlOgmZgJjCNJt6mjHKeoJl/AI/0diiS6zs8ubqskjQcOJBKTiTB+oyihdUZyEqIFSiOZNtKKE19nwTv0Mh8KhnAFJq5FWOCmX0Wd4iu8HlydTkT9cVtS392pB/fppXVGEYTq1PFyvRnNBTEebPagNnAB7TzDg18ThUVvEUL99LBY8ALZpavK5W7EuHJ1eWNpBpgM8rYjgHsTivjqKGNVShnJFUMgi9vtaTupVsabcB8wnCpeRhzaeQzEsyniv68T4IHSPAw8JyZNWR56a6P8eTqYiOpH7A+sA39WZdyVgdWIsEIEgxiAM0MpI3BiKFUMoQKqgiDfIxw1v509zuAxTSzgFYWAfVU0EZ/KphLGe/QxmskeINwFH2amS3K+wfgSponV1eQoiE8ywOjvryVsyrljATaMdqBBEbiy79Ge/S3LbrNIezwz4r+fu4X+XP54snVOedywIsInHMuBzy5OudcDnhydc65HPDk6pxzOeDJ1TnncsCTq3PO5YAnV+ecywFPrs45lwOeXJ1zLgc8uTrnXA54cnXOuRzw5OqcczngydU553LAk6tzzuWAJ1fnnMsBT67OOZcDnlydcy4HPLk651wOeHJ1zrkc8OTqnHM58P/M/SyW40BPlwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#Pie Graph\n", "slices = [Multi_F_Predictions.count(0),Multi_F_Predictions.count(1)]\n", "labels = ['Bad Credit', 'Good Credit']\n", "colors = ['red', 'green']\n", "explode = [0, 0.1]\n", "plt.pie(slices, labels=labels, autopct='%1.1f%%', explode=explode, colors=colors, wedgeprops={'edgecolor': 'black'})\n", "plt.title(\"Credit Risk Ratio For a Multi Optimized Model - Foreign\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "46\n", "138\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD3CAYAAAAT+Z8iAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqp0lEQVR4nO3deXwddfX/8dc7e9KkLQW6Qwtlbwtl+5aWrSKrKIsiCFUExK8gAj9BQeWL1BVkUURAXED2TRAQZLFlKYjsUKCUHYoFCpQW2mbfzu+Pz0Sml5vkJrnJ3Htzno9HHrmZO3fm3MzM+37mM3NnZGY455wrXEVJF+Ccc65/edA751yB86B3zrkC50HvnHMFzoPeOecKnAe9c84VuMSDXtJESSapJPr7Lklfz+L0fyTpzxmM94Cko7M1356QdImk05OY90DI1jJNXVdyQV/eW38sd0lHSPpXNqeZzXlKulzSz/u7pnwiaWdJL/fnPDIKekmHSXpSUq2kpdHKvVN/FGRm+5jZFdF8u12BooBujGr7UNLfJI2JTe+XZpbVAJe0WFJDNM/3opW3OsPXfuo9mdkxZvazXtTREXy1sZ9nezqdbIg+UN+Manhb0g0dz8WXaZIkbSHp75JWSlot6X5JM3vw+jmSro4P68t76+1y763Y+vJ0yvB1JDVLWjxQtfRGrP5/pAy/WtKcfpqnSaqLbV8fZ3seZvaQmW2a7enGdRv0kk4Czgd+CYwC1gcuBvbvZPwkWlvfMbNqYCOgGjh3AOb5hWie04CtgR8OwDw7M9zMqqOfrXr64r4us6hF+zVg9+h/sh1wb1+mmW2SJgEPA88DGwBjgVuAf0qakWRtCRgiaUrs78OAN5Mqphd2kLTjAM5vq9j2NbynL86FPdAug17SMOCnwHFm9jczqzOzFjO73cy+H40zR9JN0afqKuAIScMkXRq1/t+R9HNJxdH4xZLOjVrfbwD7pszzAUlHS9ocuASYkeknqZl9DNxKCN+O6f23FSapIqpzuaSPJT0haVSa9z1G0nOSvpfBPN8D7kmZ5w8kvR61GhdJOjAanvY9pe7OSvqmpNckrYhaoGO7qyPNexgbvXZFNK1vpvxP1lhmaV6/r6RnJK2StKSbFtP2wD1m9nrH/8TM/hib1n+7xTr2aKJ14KNoL2Cf2LgbSHow+t/Nk3RRais6Nm6n61kac4BHzOw0M1thZqvN7ALgKuBX0fQ6Woz/K+ndaLonR8/tDfwIOESxPac07+1hSb+J1q83JM2Mhi+R9IFi3Tzx5S7pdq25Z9Yu6Yjouc0kzY2W5cuSDo5NY+1oOa+S9DgwqYvl1OEqIN7ddDhwZcr/dvPovX0s6QVJ+2U6z67qzZKzgU67f7rafqLle4ykV6P17yJJ6mkB0bp3paRlkt6S9H+SiqLn4uvBCmCOpPJonf+PpPcVuu0qo/FnSXo7Nu1tom1vtaS/Srohtp7MUthjPjlan5ZKOrK7ertr0c8AKggtn67sD9wEDAeuAa4AWgkt7K2BPYGO7pNvAp+Phm8HHJRugmb2InAMYePM6JNU0trAF4HXOhnl68AwYD1g7Wj6DSnTmAjMBy40s273DCSNB/ZJmefrwM7RvH4CXC1pTCbvSdJuwJnAwcAY4C3g+u7qSOM64G1Cy/Ug4JeSPht7PnWZpaojBMBwwofxsZIO6GRejwKHS/q+pO26CNsO04GXgXUIG+2lsY3tWuBxwvKZQ9hT6ExX61mqPYC/phl+I7CjpKrYsM8AG0fT+4Gk3c3sbsJe7Q3d7DlNB56L6r+WsOy2j2r8KnCh0nTzmdkXOlqNhOX1HnCvpCHA3GhaI4FDgYslTY5eehHQSFhXjop+unM18BWFRtfmQA3wWMeTkkqB24F/RvM8HrhGUkf3QqfzzKDebLgI2ETS7qlPZLj9fJ6wTLaKxturFzX8jrB9bwjsSthW4oE7HXiD8D/4BaExsQmhQbgRMA74cZr6ywh5ezkwgrAdH5gy2uho3uOAbwAXSVqry2rNrNMfYDbwXjfjzAEejP09CmgCKmPDDgXujx7fBxwTe25PwICS6O8HgKOjx0cA/+pm/g8A9cDKaDoLgPVT6rs6enwU8G9gy06m82tgMXBoN/NcDNQCq6N53kvoPuls/AXA/p29p2ih/jx6fClwduy5aqAFmJhmuhOj+X8c+/ke4YOsDaiJjXsmcHm6ZZbJD6H77jfdrCvzCB8Qy4EfpPxv48v0tdhzVdF7GE3oFmwFqmLPXx1bfh3vt6S79SxNfa3A3mmGbxZNc1xs+pvFnj8buDR1Xerivb0ae25qNL1RsWHLgWmpyz32/CbAB8DO0d+HAA+ljPMH4AygOFo34vX+MnX9SrO+lETLai/gLOA0YHdgcTTezoQPmqLYa6+L3n+X8+yq3s7ecw/WwXj93wYeja0jczLZfqLX7xR7/kZi62qaeRqwik+2rwui/0ETsEVsvG8BD8TWg//EnhNhu5gUGzYDeDN6PAt4O3q8C/AOoNi4/+KTfJhFaJyWxJ7/ANihq/9ddy365cA66r6PaUns8QSgFFga7fZ9TFjQI6Pnx6aM/1Y3087ECWY2DNgSWAsY38l4VxG6Wa6Pds3PjlovHWYT/sk3ZTDPA8yshvCP34zQOgVA0uGSFsTe/5T4890YS+x/Yma1hOUwrovXrGNmw6Ofc6NprDCz1bFx3kqZxhK6IGm6wsHKZZJWEvZEOn0PZnaNme1O2AM4BvippM5aSu/FXlcfPayO1V0fG7ezOrtbz1J9SGjhpRoDtAMfdTLPt6K6MvV+7HEDgJmlDkt74F6hq/Q24HQzeygaPAGY3vEeo/c5m/DBuC4h9HqzPV1JCKRDCUEZNxZYYmbtKdMdl8E8u6q3S1EXUUfX1c7djP4nYJSkL6Spvbvt573Y43qi5dHF/LeJbV8nELaDspT33dX2tS6hQfNU7H9ydzQ81VjgHYsSPM20AJabWWu699CZ7oL+EcIu2gHdjJdaVBNrhs9QM+vYdVtKaHF2WD/D6XbLzJ4n9N2l7XezcHzhJ2a2BTCTsAt3eGyUOYRAuDaD7oeOac4ntFLOBZA0gbASfgdY20L3zELCp3om7+ldwsZCNL0hhG6AdzKpJzaNEZJqYsPWT5lGd3VcC/wdWC/6EL2ET95Dp6L/8V8J3RdTuhs/xVJC3fFulPU6Gbe79SzVPODLaYYfTOhKi3+4pK6f70aP++1Sr1H/7rWEPZI/xJ5aAsyPvcfhFrp4jgWWEfZUMt2e4m4mdMm9YWapHw7vAut19DnHpvtOBvPsqt4umdlk++Sg50PdjNtC6Bb9GWuul73efnow/w8JewkTYsO62r4+JHzAT479T4ZZ6KZLtRQYl5JfnW0DGesy6M1sJaEf6SJJB0iqklQqaR9JZ3fymqWEvr3zJA2VVCRpkqRdo1FuBE6QND7qV/pBFyW8D4yP+q0ydQWhVbdf6hOSPiNpahTiqwgLqy02SgshDIYAV6Ws6F05H9hD0rTotUbYIIgOlMQDr7v3dC1wpKRpksoJu8WPmdniDGvBzJYQuqjOVDgAvSWhLy9dX3xnagit60ZJ/0M4MyOt6ODTvpJqouW9DzCZWL9vhnW/BTxJOHhVpnA2TGqLrWPc7tazVD8BZkr6haQRUa3HEz7oT00Z9/RoXZ9M6HftOFX0fWBiD9aLnvgFYd05MWX4HYT+6K9F216ppO0lbW5mbcDfCP+vKklbsOZB1k6ZWR2wG+mPaTxG6Go4JZrfLMJyuD6DeXZab4b/h564CigH9o4N6/P2053of3Aj8ItoPZoAnMSn94w6xm8nNP5+I2kkgKRxnezxPkLIpO9IKpG0P/A/fa252xXWzH5NeBP/RwivJYTW6q1dvOxwwq7NIsIu8U18stv8J0L3ybPA04SVpjP3AS8A70n6sLtao3qbCf1o6b6IMjqqZRXwIuGga+p50c2EA7ojgcsy2ajNbBlhV/h0M1sEnEdYYO8T+mkfzvQ9mdm9Ue03Ez7dJwFf6a6GNA4l9Gm+Szi4c4aZze3B679N6H5ZTfiwv7GLcVcRzkj5D6Ef82zgWDPrzRd3ZhP6L5cT9s5uILTc0+lqPVuDmb0K7EQ4ALeY8L/9ErCXmT2cMvp8wsH1e4Fzzeyf0fCOg7nLlXIuehYcCuwAfBTrPpgddb/tSVgH3iV0O/yKEHAQtsXqaPjlwF8ynaGZPWnRmVIpw5sJDaV9CK3Ri4HDzeyl7uaZQb1ZEwXuGYSDlh3DsrX9dOd4wofhG4Q+9GuBy7oY/1TCOvWowplu84BPnTsfy59vELalrxI+PDvbBjKiNbuCnMstCl+8esnMzhiAeU0knE9emtIH6lxiJD0GXGJmGX+Ip0r8EgjOxUW7+ZOirpi9CaeB3ppwWc4NGEm7Shoddd18nXCSyd19mWbi39hyLsVoQnfe2oTvARxrZs8kW5JzA2pTQldpNeE7OQdFx6R6zbtunHOuwHnXjXPOFTgPeuecK3Ae9M45V+A86J1zrsB50DvnXIHzoHfOuQLnQe+ccwXOg9455wqcB71zzhU4D3rnnCtwHvTOOVfgPOidc67AedA751yB86B3zrkC50HvnHMFzoPeOecKnAe9c84VOA9655wrcB70zjlX4DzonXOuwHnQO+dcgfOgd865AleSdAHODSRJAg4GqoBWoCX6WQYsBd41s7rkKnQu+2RmSdfg3ICRNKUKntofWprAWsGawN4LP8UroELQVgnLS+B9g3fq4fVGeA14GlhgZvVJvw/nesKD3g0qkqZOhH+9CUPTPW/ASqKmfez3Imh4BFrehKqqMOixlfAQ8BQe/i7HedC7QaW7oO9OE7CQkO6PhPBvfgOGVMHbzXBrA9wMPGJmbVks27k+8aB3g0pfgz6dJmABcDu0Xg/170BRGdy5Cm4A/mlmtdmal3O94UHvBpX+CPpUbwF/B7seVj8F5UPgiRVwDXCjma3or/k61xkPepf3JJUDa0c/Q4Di6OcFM1ueMm6/B33cSuAe4DqouxuKS+GW1XCemT01EPN3DjzoXY6TVAJMADYBNqGMqRQzBWNd2hlGKzUYJZTRSAWtlGIUAU2U0sRca7ADUqY3oEEf9wHwZ2g7H5qa4a2V8CtCK79hoGtxg4sHvcsZUahvCcyggj0wtqOZUVTSxAhaGUU561LBCKAaqCScDV8GKGVii4DbmWf1tkfKPBIL+g5twF3AeVD7GCC4tB5+Z2avJ1WTK2z+hSmXGEllwC4Usztl7EExk6mmmQmUMJFKxhI6Y0oLaz0tBj4ffqpfBy6Gb/8R/neodPtqOMXM3kq4RFdgCmoDcrlP0ghgHyo4lGJ2YwQtbMYQ1qeY8UAl5UnXOJAmAedB6Y+h9Cw48AL4QrV0WR3MMbMPk67PFQYPetfvJK0NHEYFR1LMZCbQxBRq2BiooTLp+nLBMOBMKD0xhP7RV8ORldK5jXCOn57p+sqD3vULSUXA7pRzPCXswca0MY0qNgRKKUu6vlw1GvgjlJ8KnALfuxtOLJVOb4U/mFlz0vW5/ORB77JK0noU87+UcgxDKWc61UxF3m7vmUnAzVC1ADgRznwaTpL0ZTN7MuHSXB7yoHdZIWkzyphDCfszDbEt5YxJuqr8Nw14AIZcC1XHwoNDpAvr4XQza0q4NJdH/Hr0rk8kbacK3U05TzOTL3EyFXzeQz6bBMwGvQKVu8Bx1fCSpO2SrsvlDw961yuSZqhCj1DFfD7DHpxMJbMo8S6a/jMauBOqLoEJNfBglXR29K3gPpPUJmmBpGclPS1pZg9fP0fS9zp57nBJCyW9IGlRZ+NlOJ+JkhZGj7eTdEH0eFZPax5MPOhdj0iaqArdRiXz2IvpnEQVO1Dkh1cHRrx1v+snrfutszDpBjObZmZbAT8EzszCNJG0D/D/gD3NbDKwDeHKEKnj9bgb2cyeNLMToj9nAR70nfCgdxmRNFSlOpdSFrED+/BdqtgG+VGeZHS07n8PE4bAv4qkw7I4+aHARwCSqiXdG7Xyn5e0f8dIkk6T9LKkecCmnUzrh8D3zOxdADNrNLM/Ra9/QNIvJc0HTpS0raT5kp6SdI+kMdF420Z7Go8Ax8XmP0vSHZImAscA3432SnbO4v+iIPhm6rokqQjxDUo5h00pY08qk7t4gIsT8FXQVlC1J/ypWtq2LnyztjfXwq+UtACoAMYAu0XDG4EDzWyVpHWARyX9ndAy/wqwNSFHniZcpj/VlE6GdxhuZrtKKgXmA/ub2TJJhwC/AI4C/gIcb2bzJZ2TOgEzWyzpEqDWzM7t8TsfBDzoXackbUw51zGczdiPIYxLuiKXzlTgeaj6PByzCLaWdICZrerhZBrMbBqE4y/AlZKmED5PfilpF6AdGAeMAnYGbum4s1YU/r1xQ/R7U8KHwtxwW1+KgaWShhE+DOZH410F7NPLeQ1aHvTuUyQVU8z3KOUMZlHGdIq9ky+3rQM8BFXHwczr4DlJu5vZa72Zlpk9ErXe1wU+F/3e1sxaJC0mtPoh3HmxOy8A2wL3dfJ8x43YRbis9Iz4k5KGZzgf1wXffN0aJG1IGU8witM5lkpmeMjni1LCt2rPgfWq4GlJu/dmOpI2I7SolxOuzvBBFPKfIVwyGuBB4EBJlZJqgC90MrkzgbMljY6mXS7phDTjvQysG+1NIKlU0mQz+xhYKWmnaLzZncxnNVDTozc6iHiL3v2XivQ1Svk9s6hgBw/4fHUMFG0BNfvBbaXSd1rM/pLByzr66CG0rr9uZm2SrgFul/Qk4Y6JLwGY2dOSboiGvUW4UfqnmNmdkkYB8xT6ZAy4LM14zZIOAi6IumtKgPMJewRHApdJqifcxyWd24GbooPFx5tZ2noGK78evQuXCy7jIso5jNlUMTrpirIgh69HP1BeBnaEhlXwvWazi5OuxyXHW/SDnKTxlHEn6zGJg6jyLzwVjk2Bx6ByJpxTIVU2mp2XdE0uGb5zPohJmkUpz7MTmzPbQ74QTQKegKq14aeV0ilJ1+OS4UE/SKlEx1HOnXyF4exCia8JhWt94FGoGg5nlEvHJ12PG3i+eQ8ykqQynUkVZ/MtKpmUdEVuIKwH/BuqhsKvSqVvJl2PG1ge9IOIpBLKuJzhHM+3qGJE0hW5gbQB8DBU1sBvi8MZLm6Q8KAfJCRVUMYdjOYgjmYI1UlX5JKwCXAvVFbAFVm6GJrLAx70g4CkGsp4iEnswuFUDa7bb7tUWwOXQWUV3CNpZNL1uP7nQV/gJFVRxn1swVS+TKWfUOsADgGdCMNr4C5JfpHpAudBX8AklVPGXWzMZPaj3Je2i/s5lO4Im1fDpdG3Vl2B8k2/QEUHXm9hAtvzRSp9SbtURcCNUDkSvlgGftplAfPNvwBJKqKM6xjLrhxCJcVJV+RyVQ0wF6oq4CxJn026Htc/POgLURkXsi77cBhV3ifvurMhcAtUVsLN0eWJXYHxoC8wKtbRVPB1vsoQv4+ry9RuwNFQMRT+nHQtLvs86AuIpB0p4QIO9+vWuJ47C8qHwh6Svph0LS67POgLhKT1KeUOvkwlvvPteqEKuB6qqsK1330tKiAe9AVA0hDKmMssqtk46WpcPtsR+IZ34RQcD/pCUMZf2Jj1mOmHXl3feRdO4fGgz3OSDqGCfdmfSvwrLy4LvAun8HjQ5zFJ61HCnzmEKj/DxmXTjsCR4UqXv0m6Ftd3HvR5SlIR5dzEzlQwLulqXCH6GZQZfEnSZknX4vrGgz5fFfN9RjCZnb1f3vWPtYAfQdkw+HXStbi+8aDPQ5KmUMQZHMwQX4KuP50IxcWwq6Ttk67F9Z7HRJ6RJMq5gj0oZ62kq3FxS4DPAJsDk4HfRsPnAOOAadHPnZ28/m5gU2Aj4KzY8FOBLYHDY8Ouik2/P1UBZ0LlMLjQr3CZvzzo889satiU7XzZ5ZoS4DzgReBR4CJgUfTcd4EF0c/n0ry2DTgOuCt6zXXR75XAv4HnonGeBxqAy4Fv98ebSOMoUE347NpjgGbpsszDIo9IGkYpv+MA77LJRWOAbaLHNYSW/TsZvvZxQkt+Q6AM+ApwG2EDbQaMEPClwDnACdHjgVAC/AaGDIXfSfI1Lw/5QssnZZzJFlQwPulCXHcWA88A06O/LyR0vxwFfJRm/HeA9WJ/j4+G1QBfItz+bwNgGPAEsH9/FN2FL4WaxgJ+U/E85EGfJyRNBY5gTyqSrsV1rZYQjOcDQ4FjgdcJ3TZjgJPTvMbSDOvoED8leu15wOnATwnXJzgY+HnWqu6agDOhejj8eIBm6bLIgz5flHMhn6WcIUkX4rrSQgj52UDH9QNGAcWEje2bhG6aVOMJB3M7vE1oPsc9E/3eBLgSuBFYCLyajcIzsC9QChtI2qbbkV1O8aDPA5J2oJjt2NaXVy4z4BuEvvmTYsOXxh7fAkxJ89rtCYH9JqFP/npgv5RxOlrzLYQDsxA24Pq+Fp6hYuCEcB2ck7od2eUUD458UM457EalfzUqtz1MOO3xPtY8lfIUYCqhj/5+PrmmwLt8cgZOCaEffy/CB8XBhNNcOtxK+DAYCwwHZkTTFLBVf7yZTnwTipvDt2WHD+BsXR/JLF3voMsVkmYwhHl8128L2COLgNuZZ/W2ximBkqZOhH+9GbrPXS98Cepvg9Nazc5PuhaXGW/R57pyzvPWvMsl3w1Xtvy+f4Eqf3jQ5zBJMylhK7byCxC73LEjsG7YI9ot6VpcZjzoc1k5Z7Crt+ZdbhFwMgwZnv5MUZeDPOhzlKQJtLOLt+ZdLjoEVA+7SfLb0OcBD/pcVcqJbEMR5UkX4tynrQ1MhSa8+yYveNDnIEnlGEfzP37fKJe7vgI1NeFMUJfjPOhz0wGMJjSbnMtR+4HaYD+/0Fnu8wWUiyo4mR2oSboM57qyCTAifNfLL4mQ4zzoc4yksbSyJX6XTpcHDobycjgw6Tpc1zzoc88X2ZQ2P6XS5YMDobQSDk26Dtc1D/pcU8FRTKUq6TKcy8QMoBXGSFo/6Vpc5zzoc4ik0bSyBZOSrsS5zBQDO4YLas5MuhbXOQ/63HIgG9M6YPeIcy4LdoLqCtgh6Tpc5zzoc0kFR7Gl31rE5ZftQVWwS9J1uM550OcISdU0s5V327h8sy1QC5v7+fS5yxdM7pjJSOr9u7Au36wDDA03vfJmSo7yoM8VxXyWjalOugznemNbaA+/XC7yoM8VpezLBhQnXYZzvbGzH5DNaf61nBwgqZoiNmF80pU41zvb+QHZnOYt+twwk5E0eP+8y1dTgQbYKOk6XHoe9LlA7MgkP63S5a9RQDMMkeTfAslBHvS5oILpjPb+eZe/ioEaaCRkvssxHvS5oJ2pjEy6COf6ZmS4FMKYpOtwn+ZBnzBJFbQw0m8y4vLduPBrbLJVuHQ86JO3GTXU+/lPLt+tD2V4iz4nedAnbyqjUdJFONdXE6CiyFv0OcmDPmnFTGGsfyPW5b+xoBq/DEJO8qBPWimTGOotepf/xgAl4DcgyUEe9EkT6/ttwF0hiL4I4ndHy0Ee9EkzRnvHjSsE0Tel/LSCHORBn7RW1vYWvSsEJYB50OckXygJklSOqKAy6UoGl1VQdmHSRRSgxeGXZ0oO8oWSrNFU0ECRX+dmAL3aBBf/CIYlXUghaoV7k67BfZoHfbKGUkZb0kUMJmbWCJycdB3ODSTvo09WGcVY0kU45wqbB32yyvyalc65/uZBn6wy7zxzzvU3j5lklXuLvh+1sbakPZIuo0A9Y2YfJl2Ey4wHfbK8Rd9fxgCj2Yh2/pp0KQWnjlIauAM4JOlSXGY8ZpLlXWf9ZS3gKP8qWr9YANyD3zIwj3jQJKuW5qRLcK6H2gGjNekyXOY86JO1mqakS3Cuh0LQ+5qbRzzok1VLsy8Dl2cagBbeT7oMlzkPmWTV0uLn3bg8U0cL7XyQdBkucx70yVpNqx8Qd3kmHFlannQZLnMe9MmqpY1S2pMuw7keqKMND/q84kGfIDNrpYTV1CZdiXM9UI/hQZ9XPOiTVsJSPk66COd6oIEiPOjzigd98t5gZdIlOJehdqCWSmBJ0qW4zHnQJ62ZF71F7/LGKqCYOjNbnXQpLnMe9Elr43VW0JB0Gc5lZDlQwptJl+F6xoM+eYtZ7hdCcHliOdDO80mX4XrGgz55r/Khn0vv8sSHNNPEs0mX4XrGgz55r9NIiXfeuLzwPg3AK0mX4XrGgz5hZtZOOa/5lUNczjPgfUqBRUmX4nrGgz4XtPII7yZdhHPdWAm00ga8kXQprmc86HNBCw/xln8/1uW4JUApT5iZJV2K6xkP+tzwOG8nXYJz3fgPzTRwT9JluJ7zoM8Nr9BIEauSLsO5LrxJI/DvpMtwPedBnwPMrJ1S5vN60pU414lmYAWVwFNJl+J6zoM+VzTyN172fnqXo94GynjNzPxE4DzkQZ877uENSvza9C4nvUgzzdyQdBmudzzoc4SZLUEsY2nSlTiXwoBFtNDOrUmX4nrHgz6XtPF3XqMt6TKcW8MyoJlG4LmkS3G940GfS1q5nReoS7oM59bwEu3ALX7+fP7yoM8t97GCIlYkXYZzMQuppYW/Jl2G6z0P+hxiZi2I63neu29cjlgFLKcUmJ90Ka73POhzTQuX8hQN+E6yywULaKOYm8ysKelSXO950Oeex2ik1i9y5hLXDjxOI81clHQprm886HOMmRltXMqzeAvKJes/QAvLgceTLsX1jQd9LmrjCp6lndakC3GD2hPU08wFfrZN/vNb2OUgM3tVFXqGhcxk2gDO+ENY49yKj4DPAI3A00BVNPyzwCZpXv8qcDdhl38bYOdo+NzoudHAF6NhzwINwA7ZK99lUSPwMsUYVyZdius7b9HnqiZ+yoPUDuhB2XWAY6OfbwGlwObRczvEnksX8u3AncBs4DhgIfABITCWAN+m4w5F0AIsALbvn7fhsuBZjGLuN7NlSZfi+s6DPnf9k1pW8GZCc38DGAEMz3D8d6LxRxD2E6cALwMC2ggh30JY4x4GpgPFWa3YZUsb8CANNDEn6VJcdnjQ5ygzM5r5GQ8ldEXLhYSw7vA4cDFwK6S9kfkqYGjs76HRsHLCXsElwFpABfAusFnWK3bZ8jzQygtm9ljSpbjs8KDPbdewBGOgd55bCa3xydHf2wMnAscANZDxPYYU/d6J0OWzF3Afod//KeBG/Gs4uaYduJ9amjgt6VJc9njQ57Do2t8XMD9tG7r/vAaMAaqjv6sJa0oR4SDrO2le09GC77CK8KEQ13FlzrUJB2MPJvTjL89K1S4bXgEaeQeYl3QpLns86HNdK+fwEq18MIDzfB6YGvt7dezxS8DINK8ZSwjsjwh7BAuBTVPG6WjNd/TZQ2j1t/S9ZJcFBtzHapo4zU+pLCwe9DnOzFbSzk+5Z4CuatlMOBC7eWzYXEL//MXAm8De0fBVwNXR42Lgc8BVwEWEbp/4B8KLwDhCy78SGB9NT4TTLl3yXgE+5iPw684XGvkHd+6TVEEpb/M11mb9pKtxBakVuIA6VnGwmd2ZdDkuu7xFnwfMrJEWvs9dA3xevRs8nqCdJhYAdyVdiss+D/r8cSXLWc4rSZfhCk49cD9NNHGM980XJg/6PGFmbTRzDLdTT3PS1biCcj9NwPVmtjDpUlz/8KDPI2Z2Ny3M5QGPepcly4BnaKWZU5MuxfUfD/p808S3eJxm3ku6EJf32oGbqaOdH/k1bQqbB32eMbP3aeO7/I062pOuxuW1J2jnI16lnQuTLsX1Lw/6fGRcykpe5HGPetdLK4B5NNHEV8zM16MC50Gfh8zMaOKr3EsjHyZdjcs77cBN1NHOHDN7OelyXP/zoM9TZvYybZzMddT5JQRcjzxCGx/yEm2cl3QpbmB40Oezdv5ALfdxt99f1mVoCfAADTTzZTNrS7ocNzA86PNY1IXzNZ7jIxYlXY3LebXAtdTTwmFmltQtbVwCPOjznJmtpIX9uJUGPkq6Gpez2oDrqaOV35nZ7UmX4waWB30BMLMnaON0rqHOv0rl0ppHM8t4hha/ochg5EFfKNr4Nau4gxuo95Mu3RoWAU+ykiYO8H75wcmDvkBE95g9nCU8z11+cNZF3gZuoZ4WPmdmfi+vQcqDvoCYWTPN7MMC3udRvOU22C0DrqKBFg42syeTLsclx4O+wJjZR7Qwi3tZjX8VZvBaCVxOPc1828z+kXQ5Llke9AXIzN6khb25iXoWJ12NG3ANhJBv5OfWbpcnXY5Lngd9gTKzx2hhP66lniVJV+MGTDNwJXXU8hfaOCvpclxu8HvGFjhJe1PG3zicSsYnXY3rVw2EkF/ObTTzNb9YmevgLfoCZ2Z308xBXEk9byVdjes3dcCl1LOcqz3kXSpv0Q8SkvaglFs5lCo2TLoal1WrgMuop46LaOFUv++rSzUoWvSSRkm6VtIbkp6S9IikA7M07QckbZdmeKmksyS9KmmhpMcl7dOH+Rwh6cLo8TGSDo8NH9vd681sLi18jutYzQI8CArFx8CfqKeWs6zZTvGQd+mUJF1Af5Mk4FbgCjM7LBo2Adivn2f9M2AMMMXMmiSNAnZNU19xT7+taGaXxP48AlgIvJvB6+ZLms4/uJ/ljGA3SlFP5uxyytvANdTTzGnWaucnXY7LXQXfdSPps8CPzexTIRs9XwH8HtgOaAVOMrP7uxheCfwF2AJ4EZgIHBf/QoqkKsIFYTcws1Vp5lkL/BrYCzg5msYJQBnwGPBtM2uTdCTwQ2Ap8ArQZGbfkTSHcC3CxcDlwDuEQ3EzzKwhg//JSMqYx0ZsxBepLPyP+wL0LMYd/70S5d+TLsfltsHQdTMZeLqL548DMLOpwKHAFVHIdzb8WKDezLYEfgFsm2aaGwH/SRfykSHAQjObDiwHDgF2NLNphOsMzpY0BvgJsCOwB+GDZQ1mdhPwJDDbzKZlEvLR6z6gmem8xv1cRh11mbzK5YR24B6auYP3aWG6h7zLxGAI+jVIukjSs5KeiAbtBFwFYGYvAW8Bm3QxfBfg6mj4c8BzvSijDbg5evxZwofFE5IWRH9vCEwHHjCzZWbWDNzQi/l0yswaaOYLLOMPXEiDn5GTBxqBq6jnKZ6hhSlm9kLSJbn8MBiC/gVgm44/zOw4QpiuGw3qrJe6q97r7vq7XgPWl1TTyfONsX55EY4fTIt+NjWzORnOp0/MrN2a7WQaOIirWcV8WvzKlzlqCXAR9bzDtTSzs1+gzPXEYAj6+4AKScfGhlXFHj8IzAaQtAmwPvByhsOnAFumztDM6oFLgQsklUXjjpH01TT13QscJGlkNN6I6GDxY8AsSWtLKgW+3Mn7Ww109oGSETO7kxYm8zDPcjl1rO7L1FxWtQH308IVrGY1h1uTfdPM/C7BrkcKPuij080OAHaV9Kakx4ErgFOjUS4GiiU9T+geOcLMmroY/nugWtJzwCnA453M+v8I1w9cJGkh4cyfZWnqWxSN+89omnOBMWa2FJgDPALMo/PjDJcDl0haEB0o7hUze5tmZvAuv+UiGnilt1NyWbMC+CN1PMrjtLK5md3c7WucS6Pgz7pxPSfpM5RyHRtRw75UUZ10RYOMAU9j3E0j7ZxGG7/1b7q6vvCgd2lJqqKUXyC+xd5UsDXyc+4HwHvAbdSxgv/QxCFm9nzSJbn850HvuiRpGmVcy7qsz4EMYZ2kKypQDcB9NLGAZlr5Psaf/bZ/Lls86F23JBVTzAmInzONYmZR7t05WWKELz/dRSPGX2nmJD+jxmWbB73LmKSRlPIT4AhmUMJMSqhIuqo8ZYRzuOZSy2qW0MwRZtbZgX3n+sSD3vWYpA0o52xgX2ZRzvYU+WUUMmSEi1nMpZZVvE8z3wdu84Otrj950Ltek7Ql5ZyP2IEZlLEdxQxJuqocZYSv0f2TWlayLAr4Wzzg3UDwoHd9Jmkq5fyINg5gKjCTiv9+73iwayb0wf+LOhpZRhOnAjd7wLuB5EHvskbSKEo4ATie8YgdqGYjBsHFsNN4H3iCJp7FKOZBGjkTmO/Xi3dJ8KB3WRd9Q/cwKjiedjZlCjCNCtaj6ysI5btVwIsYT7OaFbRh/J5W/mhmfsk4lygPetevJE2kiK9SyrcoYS22oZzNKWE0hXEBjpXAIowFrOZDSijhHzRxBXCPmbUmXZ5z4EHvBkh0p6+tKeVIijgQGMEk2tmUIUyCvDkvv4VwJcnFtPESdaygmGJup4mrgHnRJaWdyyke9C4RkiYCe1LBl2hhZ4bRwsZUsh6ljAOGkxvdPI2EmzQupo1XqGUZVZTyCi3cSRtzCfcM8KtJupzmQe8SJ6kEmI7YlQp2o5VtERWsSxPjqWI0paxFCP8aoLgfimgh3OvrA+A9WnmXOj6ghEbKKOPVWLD/28xq+6EC5/qNB73LSZLGAlshplHOdMQkWhlHK0OppJFhtDKCYoZQRgWllCHKgHLCnXeLCddyb41+t8X+Xk0rH9PIKtpYjainnDZKKGMpRTxHI49hPEe46fobfiqky3ce9C6vRDdyGU+4ofoEOtr5JaxFMWtRxHBgKFCOaMRoApowGjCaaKeWFt4m3HA9/rPCT310hcqD3jnnClwhnODmnHOuCx70zjlX4DzonXOuwHnQO+dcgfOgd865AudB75xzBc6D3jnnCpwHvXPOFTgPeuecK3Ae9M45V+A86J1zrsB50DvnXIHzoHfOuQLnQe+ccwXOg9455wqcB71zzhU4D3rnnCtwHvTOOVfgPOidc67AedA751yB86B3zrkC9/8B+LNine2zYcMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#TESTING - Non Foreign Workers\n", "filtered_dataset=[]\n", "for i in range(len(test_dataset)-1):\n", " if test_dataset[i][-1] == 2:\n", " filtered_dataset.append(test_dataset[i])\n", "len(filtered_dataset)\n", "\n", "Single_F_Predictions = []\n", "for i in range(len(filtered_dataset)):\n", " Single_F_Predictions.append(predictor.predict(filtered_dataset[i]))\n", " \n", "for i in range(len(Single_F_Predictions)):\n", " Single_F_Predictions[i]=(Single_F_Predictions[i][0][0])\n", " \n", "for i in range(len(Single_F_Predictions)):\n", " if (float(Single_F_Predictions[i]) >= float(0.5)):\n", " Single_F_Predictions[i] = 1 \n", " elif (float(Single_F_Predictions[i]) < float(0.5)):\n", " Single_F_Predictions[i] = 0\n", "\n", "print(Single_F_Predictions.count(0))\n", "print(Single_F_Predictions.count(1))\n", "\n", "#Pie Graph\n", "slices = [Single_F_Predictions.count(0),Single_F_Predictions.count(1)]\n", "labels = ['Bad Credit', 'Good Credit']\n", "colors = ['red', 'green']\n", "explode = [0, 0.1]\n", "plt.pie(slices, labels=labels, autopct='%1.1f%%', explode=explode, colors=colors, wedgeprops={'edgecolor': 'black'})\n", "plt.title(\"Credit Risk Ratio For a Single Optimized Model - Non-Foreign\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "# Generate Pareto Frontier \n", "def get_pareto_frontier(Xs, Ys, maxX=False, maxY=True):\n", " \"\"\"plot Pareto front\n", "\n", " in AMT this is impossible, since there is only one merged metric, But I\n", " show here just from a scientific viewpoint\n", "\n", " \"\"\"\n", " sorted_list = sorted([[Xs[i], Ys[i]] for i in range(len(Xs))], reverse=maxX)\n", " pareto_front = [sorted_list[0]]\n", " for pair in sorted_list[1:]:\n", " if maxY:\n", " if pair[1] >= pareto_front[-1][1]:\n", " pareto_front.append(pair)\n", " else:\n", " if pair[1] <= pareto_front[-1][1]:\n", " pareto_front.append(pair)\n", "\n", " pf_X = [pair[0] for pair in pareto_front]\n", " pf_Y = [pair[1] for pair in pareto_front]\n", " return pf_X, pf_Y" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "s3.ObjectSummary(bucket_name='sagemaker-us-east-2-921553072635', key='sagemaker/sagemaker-amt-credit-risk-model/data/output/tmp/02397abf-output.txt')" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# S3 bucket's tmp folder is populated by results from training jobs run by HPO (Automatic Model Tuning)\n", "import boto3\n", "import pandas as pd\n", "import io\n", "\n", "s3 = boto3.resource('s3')\n", "bucket = s3.Bucket(session.default_bucket())\n", "prefix_objs = bucket.objects.filter(Prefix=\"sagemaker/sagemaker-amt-credit-risk-model/data/output/tmp/\")\n", "objects = iter(prefix_objs)\n", "next(objects)" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "prefix_df = []\n", "\n", "from io import BytesIO\n", "\n", "for obj in objects:\n", " key = obj.key\n", " body = obj.get()['Body'].read()\n", " tmp = str(body, 'UTF-8').strip()\n", " prefix_df.append(tmp)\n", " \n", "# prefix_df" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "df = pd.DataFrame(prefix_df)\n", "df['models_list'] = df[0].str.split(',')\n", "df_models = pd.DataFrame(df['models_list'].tolist()).fillna('').add_prefix('model_')\n", "df_models.rename(columns={'model_0': 'sd', 'model_1': 'auc'}, inplace=True)\n", "df_models.to_csv('sd_auc_pareto_frontier.csv', index=False, header=True)\n", "sd_str_list = df_models.iloc[:, 0].tolist()\n", "sd_float_list = [float(x) for x in sd_str_list]\n", "auc_str_list = df_models.iloc[:, -1].tolist()\n", "auc_float_list = [float(x) for x in auc_str_list]" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "pf_X, pf_Y = get_pareto_frontier(Xs=sd_float_list,Ys=auc_float_list)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [], "source": [ "#Eliminate all 0's in pf_X and corresponding points in pf_Y \n", "i = 0 \n", "while i < len(pf_X):\n", " if pf_X[i] != 0:\n", " i += 1\n", " else:\n", " del(pf_X[i])\n", " del(pf_Y[i])\n", "#Eliminate all 0's in sd_float_list and corresponding points in auc_float_listY \n", "i = 0 \n", "while i < len(sd_float_list):\n", " if sd_float_list[i] != 0:\n", " i += 1\n", " else:\n", " del(sd_float_list[i])\n", " del(auc_float_list[i])" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Baseline Statistical Disparity: 0.75\n", "Baseline Area Under Curve: 0.74 \n", "\n", "Optimized Statistical Disparity: 0.43\n", "Optimized Area Under Curve: 0.72 \n", "\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEWCAYAAABIVsEJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABEVklEQVR4nO3deXxU1fn48c+TjTVhB4GAoAgoCGETlCIgXzVuuGEVRQtWxaXaUrVqLfq11VardWnRn1WruFCoYFG/FHCpBCtqZRVBXBAQAqIQBBIIS5Ln98e9GSeT2TKZPc/79cqLzJ17zz33ZJjnnuWeI6qKMcYYU1cZic6AMcaY1GQBxBhjTEQsgBhjjImIBRBjjDERsQBijDEmIhZAjDHGRMQCiDFJSkQmish7ic6HMYFYADFpR0Q2iUi5iJSJyLci8pyINI/BeaaLyL31TON0EXlXREpFZIeILBaRsdHKo8+5RolIcSzSNg2TBRCTrs5R1ebAQGAI8Ju6HCyOmP7/EJFxwGzgBSAf6ADcBZwTg3NlRTtNYyyAmLSmqluBBUBfEWklIvPcO/3v3d/zq/cVkSIRuU9ElgD7gaNEpLeIvCUiu0TkcxH5sbvvNcBlwK/cms7/uduPddPZLSJrA9UmRESAh4HfqeozqrpHVatUdbGqXu2z70NufjeKyBle2yeJyDq39rJBRCZ7vTdKRIpF5DYR2Q7MdMuhk5vfMhHpFJVCNg2WBRCT1kSkC3AmsBLn8/4ccCTQFSgHpvkccjlwDZAL7ADeAv4OtAfGA0+ISB9VfQqYAfxRVZur6jkikg38H/Cmu/+NwAwR6eUna72ALsCcEJcwFPgcaAv8EfibG3wAvgPOBvKAScAjIjLQ69gjgNbu9V4BnAFsc/PbXFW3hTi3MUFZADHp6lUR2Q28BywGfq+qJar6iqruV9VS4D5gpM9x01V1rapWAIXAJlV9TlUrVHUF8AowLsA5hwHNgftV9ZCqvgPMwwk8vtq4/34T4jq+VtWnVbUSeB7oiNPUhar+S1W/UsdinMA1wuvYKuBuVT2oquUhzmNMnVm7qElX56nq294bRKQp8AhOYGjlbs4VkUz3Cxpgi9chRwJD3UBULQt4McA5OwFbVLXKa9vXQGc/+5a4/3YENga5ju3Vv6jqfrfy0dy9njOAu4GeODeDTYFPvI7doaoHgqRtTL1YDcQ0JDfjNB0NVdU84GR3u3jt4z099RZgsaq29PpprqrX+dkXYBvQxafzvSuw1U9ePnfTvzCSCxGRRji1oYeADqraEpgf5Fr8vTamXiyAmIYkF6ffY7eItMa5ew9mHtBTRC4XkWz3Z4iIHOu+/y1wlNf+/wX24XSsZ4vIKJwRVbN8E1ZnHYVfAlPdzvA8EckQkR+JyFNhXEsO0Ainn6bCrY2cFuKYb4E2ItIijPSNCckCiGlIHgWaADuBD4GFwXZ2+0lOAy7BqV1sBx7A+eIG+BtwnDvi6lVVPQSMxems3gk8AVyhqp8FSH8OcDFwpZv+t8C9wGuhLsTN203Ay8D3wKXA6yGO+QxnNNYGN882CsvUi9iCUsYYYyJhNRBjjDERsQBijDEmIhZAjDHGRMQCiDHGmIik1YOEbdu21W7duiU6G3G3b98+mjVrluhsJA0rj5qsPGqy8qhp+fLlO1W1XSTHplUA6datG8uWLUt0NuKuqKiIUaNGJTobScPKoyYrj5qsPGoSka8jPdaasIwxxkTEAogxxpiIWAAxxhgTEQsgxpjY+eoruP56yMuDjAzn3+uvd7ablGcBxBgTGwsWQL9+8MwzUFoKqs6/zzzjbF+wINE5NPUU6zWfC91lQNeLyO1+3r9VRFa5P2tEpNKdJbX6/UwRWSki82KZT2NMlH31FYwbB/v3w+HDNd87fNjZPm6c1URSXMwCiIhkAo/jzEx6HDBeRI7z3kdVH1TVAlUtAO7AWXthl9cuPwfWxSqPxpgY+dOfagcOX4cPwyOPxCc/JiZiWQM5AVivqhvcaa5nAecG2X88zlTTAIhIPnAW8EwM82iMiYWXXgovgLwYaHFHkwpi+SBhZ2ouD1oMDPW3o7vUaCHwM6/NjwK/wlkEKCARuQa4BqBDhw4UFRVFnOFUVVZW1iCvOxArj5oSUR4jy8pqLI0YiJaWsjjOebPPR/TEMoD4+/wEWnzkHGBJdfOViJwNfKeqy91V3QJS1aeApwAGDx6sDfEJ01g+Wfvp6pn8e+F9HChfR+MmxzKm8E6O6zc+JueKFnvSuKaElEfz5k6HeQiSmxv3vNnnI3piGUCKgS5er/NxVl3z5xK8mq+A4cBYETkTaAzkichLqjohJjk1tahWsfKjJ5n36h9ZuGwGjTr15OC2L9i9+3KApA8iJsEmTHBGWwVrxsrOhssvj1+eTNTFMoAsBY4Rke7AVpwgcanvTu76zCMBT3BQ1TtwOtVxayC3WPCIvkOHyti96yv2fL+BPbs3stv9d8/3G9jz/Ub27c9m4bKFnH5HBzKzdlNZ0YEFf3iRli0nWwAxwd18Mzz/fOgAMmVK/PJkoi5mAURVK0TkZ8AbQCbwrKquFZFr3fefdHc9H3hTVffFKi/pyLtpCelO+9a/q/WlXll5mNK9W9jz/QZUlW5Hn1rj/RUfPsZ7i34T8ByNcg7RqFNPMrN2k5kFoDTq1NM5pzHBHH00zJnjDNU9fLhmIMnOdn7mzHH2MykrprPxqup8YL7Ptid9Xk8HpgdJowgoinrmUtinq2fyyqw7WbD0RU/TUvn+S+n/2as0bpzrqU2U7tmCaiUA7Tr0p9vRq2qk06JV96DnOXw4l4PbvqCyogOgVFYIB7d9QeM+x8boykxaOeMMWL3aGar74otQVub0jVx+uVPzsOCR8tJqOveG4t8L72PB0hdrNC0t/MPfycgopHGjMr/H7Nm9EVVF5IexDS1bH0Prtr1p0bI7LVodRYuW3WnZ6ijn91bd+erzf1Gpl7PgDz8EqjOGXM6Ywvvidakm1R19NEyb5vyYtGMBJAUdKF/nt2mpUU7tVsDmuZ3cgHAUlZWHyMpq5HmvY+chXHlD4Oao6iaxli0nO6Ow+hzLmML7rP/DGANYAEkJqsqmr96kc5eTyGmUS+Mmx/ptWuKojpxyxh2eWkReiyPJzm5Sr3Mf12+8BQxjjF8WQJKYqvL1hrdYUnQ33xR/yI9OuY9hI37NmMI72b27dtPSWec/ZF/2xpi4sQCShFSVzRv/zZKiu9m25X3P9mXvP8SAE35Wq2mJo7tz1nnWtGSMiS8LIElEVdmyaRFLiu5m6+b3aryXmZlD7+MvparSGQ7p3bRUVFTEcf1GxTu7xpgGzgJIkti8qYj3i+6m+Ot3a2zPyMjm+IFXMfRHd5DXokuAo40xJv4sgCSBea+M57M1s2psy8jI5vgBVzJ0xK/Ja9E1QTkzxpjALIAkgY6dh3oCSEZGFn0LnMDRouWRCc6ZMcYEZgEkznbu+JS27Wqsq0W/QZNZ9uEjdDvqNIadfCctWnZLTOaMMaYOLIDEydYt7/N+0d18veFtJly9lCM6Dfa8l53dhCtv+Kzez2wYY0w8xXRNdAPbij9kzkunM/PZ4Xy94W0AlhTdXWs/Cx7GmFRjNZAY+WbrRywpuptN6xfW2C6SQdOm7aisPExmZnaCchc/xaXlbN5TTtcWTcjPjW6QTNW0jUkXFkCibPu2ZSwpupuNX9aYhBiRDI49/jJOPHkqrdock6DcxVdxaTkz126lskrJzBDG9+kctS/jVE3bmHRiASRKKioO8Prsi9jwxbwa20Uy6N13PCeePJXWbXslKHeJsXlPOZVVigKVVcrmPeUhv4jDvfOPJO1Y5tuYhsgCSJRkZTWmqqrCa4vQu+8lnDjyLtq07Z2wfCVS1xZNyMwQz5181xahg0e4d/51TTuW+TamobIAEqGDB/bQqHGLGtuGj7qHTevfoHffixl28tRaw3UbmvzcJozv0znsvoS63PnXNe1Y5tuYhsoCSBi8l4/NzulB69Yt2L9vG1fdtJ6srMae/Tp2PoGrf77RHgD0kp8b/hdwXe/865J2XcUybWPShQWQEPwtH3tqwYU0a/otq5c/zcChN9bYP5mCR6qNJLI7f2NSiwWQEPwtH/vGH17hzCGF7Nr5WaKzF1CqjiSyO39jUoc9SBjCD8vHKplZkJlVvXzsfv7nrMcTnb2A/PUnpLqV23cza+1WVm7fneisGGOwGkhImZlH+l0+tnGfYxOdtaDSbSTRyu27WbhhBwAb9+wHYMARLROYI2OMBZAQWrdpwakFF/LGH16psXzsmML7Ep21oNKtP+Hzkn21XlsAMSaxLIAEUVa6jZIdq2nWtIozhxTSKGc/jfscy5jC1Fg+Nl79CfHorO/Vppmn5lH92hiTWBZAgvh09UuoVpGRAT16DuKSiUWJzlLSiVdnfXVt4/OSffRq08xqH8YkAQsgQRw6WEpWVhMqKsrpWzAp0dlJSvGc9mPAES0tcBiTRCyABPGjU37HkOG38vnal+l53IWJzk5SSrfO+nCk2vM1xsRKgwwgA/46gFXbV4Xcr+CIAlZOXkm/gVfFPlMpKt0660NJ1edrjImFBvkcyIn5J5KTmRN0n5zMHE7KPylOOTKpIh2frzEmUjENICJSKCKfi8h6Ebndz/u3isgq92eNiFSKSGsR6SIii0RknYisFZGfRzNfU0+eSoYEv/RMyWTqyKnRPG1aqr4jf3dzCTPXbqW4ND2+UItLy3m/eFet66lushNoME12xgQSswAiIpnA48AZwHHAeBGpMT2tqj6oqgWqWgDcASxW1V1ABXCzqh4LDANu8D22PjrmdmRSwaSAtZAsMjilxZFUlHyJqkbrtGkpHe/IgwXF6ia7k7u2seYr0+DFsgZyArBeVTeo6iFgFnBukP3HAzMBVPUbVV3h/l4KrAM6RzNzwWohQhUDSz5j1vSRlJVujeZp00463pGHCor5uU04Kb+1BQ/T4MWyE70zsMXrdTEw1N+OItIUKAR+5ue9bsAA4L8Bjr0GuAagQ4cOFBUVhZ3B09qdxvzt86nQHxaCyiKDwVSRB+S1GsTyFeuB9WGnmQhlZWV1uu5o6y057JUc8vQQ65cXJ7y06lsepZKDZLVBEQSl5KvPKFp/KHoZjLNEfz6SjZVH9MQygIifbYHag84BlrjNVz8kINIceAX4haru9Xegqj4FPAUwePBgHTVqVNgZ7DWoF0f9+SgqKn4IIILyP+7vI0ZP4djjw08vUYqKiqjLdUN6D0WNpDx8DahRPqm9FHE0yiOdWHlET9hNWCLSSkT6iMhRIiF6oB3FQBev1/nAtgD7XoLbfOV1vmyc4DFDVf8Zbj7rwrcvJCcjm8EoeUCjRi3o0fv8WJw24dK14zuarJnKmNCCBgIRaSEivxaRT4APgb8CLwNfi8hsERkd5PClwDEi0l1EcnCCxOv+zgGMBF7z2ibA34B1qvpwXS+qLmr0hWiVp/bRu+8lZGen55dHOnZ8G2PiL1RNYg5OP8YIVe2lqj9S1cGq2gW4HzhXRH7q70BVrcDp03gDpxP8ZVVdKyLXisi1XrueD7ypqt7TrQ4HLgdO8Rrme2ZklxhcdS0kQzIYgpDnbu9TMDEWp0sK6djxHUqgYbnGmMgF7QNR1VODvLccWB7i+PnAfJ9tT/q8ng5M99n2Hv77UGJi6slT+Wjjvzml5AsAWrftTcfOfvv704I9PW7Db42JhqABREROB3JVdY7P9suA71T1rVhmLl6+31jEuO92kZkhHDjUjHbtB+C0oqWvhrR0bDwnfDSmIQk1CusenBFSvv4NzAVSPoB8unomr8z8NQuWvepZMEp1Akf3mpkSa37ESqxGaSVi9FdDnPDRmHgIFUCaquoO342qul1E0mJFn38vvI8Fy17i9Ds6kJm1m8qKDiz4w0u0bDW5wQaQWDX5JKopqaE12RkTL6E60RuLSK0g4w6xTYv/hQfK19GoU08ys5TMLMjMUhp16smB8nWJzlrCxGqUViJHf/kblmsd68bUT6gA8k/gae/ahvv7k+57Ka9xk2M5uO0LKiuEygqorBAObvuCxk2OTXTWEiZWo7SSafSXPQtjTP2FasL6DXAvznMfX+OMjOqC84xGWkxVO6bwTnbvvpwFf3jR0wdyxpDLGVN4X6KzljCxavJJpqYk61g3pv5CDeOtAG4XkXuAHu7m9aqaNrdr1f0cLVtO5kD5Ohr3OZYxhfc12P6Par6jtKLV+Z0so7+sY92Y+gs1jPcCn00KtBSRVe4suWnhuH7jG3zACCaRz1HEatRWMtWGjElVoZqw/A3hbQ30E5Gfquo7MciTSTKJau6JdeBKltqQMakqVBPWJH/bReRInDmx0vdxbeORqOYe66cwJrlFNJ27qn7tDuU1DUCimnusn8KY5BZRABGRXsDBKOfFJLFENPdYP4UxyS1UJ/r/UXsRqNZAR5zZco2JKeunMCZ5haqBPOTzWoES4Et3nXNjjDENVKhO9MX+tovIcBG5VFVviE220ks6Lx9rjGm4wu4DEZEC4FLgx8BG0mQqk1iztSjqZuX23Xxeso9ebZox4IiWUUs3lkHcbhBMQxWqD6QnzlK043Garv4BiKoGW8rWeLGhqOFbuX03Czc4kz9v3LMfICpBJJZB3F/agAUU0yCEqoF8BvwHOEdV1wOIyJSY5yqNNMShqJHekX9esq/Wa+8AEmm6sQzivml/8t1e1uwotRqnaRBCBZALcWogi0RkITCLOC41mw4a2lDU+tzt92rTzFPzqH4djXTrG8SDBS7ftAGrcZoGI1Qn+lxgrjuF+3nAFKCDiPw/YK6qvhn7LKa+hjQUtT53+9W1DX99IPVJtz5BPFTg8k0bqFEDaQg1TtNwhdWJrqr7gBnADBFpDVwE3A5YADE11Pduf8ARLf32e9Q3XX9BPJwmsXACl2/aDanGaRq2UJ3ozVW1zHubqu4C/ur++N3HNFypspZIuE1ikQSuhlTjNA1bqBrIayKyCngNWO7WRBCRo4BRwMXA08CcGObRpJhYfYFGM91wm8QaWh+WMXURqg9kjIicCUwGhotIK6AC+Bz4F/ATVd0e+2waE111qVlYjcIY/0L2gajqfGB+HPJiTNxYzcKY+otoNl6TemL1hHcqi1XNwp5MNw2FBZAGIFZPeJvabOoa05BkJDoDJvb8PeFtYsNf57wx6SrsACIiPxKRSe7v7USke+yyZaKhuLSc94t30b5ZTo3t3k94m+iq7pwXsAcJTdoLqwlLRO4GBgO9gOeAbOAlYHiI4wqBx4BM4BlVvd/n/VuBy7zycizQTlV3hTrWBOfblDK0U0u+23fI+kBizDrnTUMSbh/I+cAAYAWAqm4TkdxgB4hIJvA4cCpQDCwVkddV9dPqfVT1QeBBd/9zgClu8Ah5rAnOtymlcVYml7gzxaayWHVQRzNdG/ZrGopwA8ghVVURUQB3bqxQTgDWq+oG95hZwLlAoCAwHpgZ4bHGRzrOAhyrDmrr+DYmMuEGkJdF5K9ASxG5GrgS5wn0YDoDW7xeFwND/e0oIk2BQuBnERx7DXANQIcOHSgqKgqRrfRTVlbm97p7Sw57JYc8PcT65cWsj3/WomprRnMqMvNAhIrKKhavXEvnqtqz6AQqj/qmm6rqWh7pzsojesKdTPEhETkV2IvTD3KXqr4V4jB/075rgH3PAZa482zV6VhVfQp4CmDw4ME6atSoENmKTDI3nRQVFeHvutPteYQaNYXMDEb26+P3ugKVR33TTVV1LY90Z+URPeF2ok8BZocRNLwVA128XucD2wLsewk/NF/V9diYS8Wmk1Bpp2JwSZWJGo1pKMJtwsoD3hCRXTiLSs1R1W9DHLMUOMYd7rsVJ0hc6ruTiLQARgIT6npsvMRqRbt4rpTnnXYqt/mnwkSNxjQUYT0Hoqr3qGof4AagE7BYRN4OcUwFTp/GG8A64GVVXSsi14rItV67ng+8WT3Tb7Bj63BdUVWfsf3Vz2IUl9Z+oCyWzwwESzuRD7sFKw9jTGqp61Qm3wHbgRKgfaid/U3EqKpP+ryeDkwP59hEibSJo66r2UX7DrhvO2ek9fHt82qknagRWqlc8zHG1BZuH8h1OGt/tMNZ++PqhvZMRiRNHJGsZhcNvl/Ux7fPq3XORLT5x7LJzhgTf+HWQLoCv1DVVTHMS9pJ1J1+ogJXKNEuj1QcCGBMOgkZQEQkAzhHVe+IQ37SSqLu9JP1IcJoloc1hxmTeOEsKFUlIh+LSFdV3RyPTKWTeN7pe9+RJ+uw1GiVhzWHGZN44TZhdQTWishHgPdoqbExyZWpM3935Cflt47bua2WZUzDE24AuSemuTD1lqg78kQ1JaXKw3/WT2PSWbhTmSyOdUZM/SRzh32sJPvDf9ZPY9JduMN4S/lhLqocnPVA9qlqXuCjTDxZh33ysX4ak+7CrYHUWPtDRM7DmXLdJJFE3JGnSlNSIlhwNemurk+iA6Cqr4rI7dHOjElNyd6UlCipFFytr8ZEItwmrAu8XmbgLG8baGp2Y4wrFYKr9dWYSIVbAznH6/cKYBPOCoFpw+7ATENlfTUmUuH2gUyKdUYSKV3vwFZu383nJfvo1aYZA45omejsmAjE48bG+mpMpIIGEBH5I7DBdwZdd4GpI1T1tlhmLl7S8Q5s5fbdLNywA4CNe/YDWBBJMfG6sUmlvhqTXEKtB3I27nKxPh4Dzop+dhIjlutyJMrnJfuCvk5FDW0tkXiu25Kf24ST8ltb8DB1EqoJS1W1ys/GKhHxt255SkrHO7BebZp5ah7Vr1NZOHfj1c09pZJT73Mlw2fBmpZMsgsVQPaLyDGq+qX3RhE5Bkir28BUGC1TF9XNVenSBxKqmdE7wEhWGwaURtYMmUz9Yel4Y2PSS6gAchewQETuBZa72wYDdwC/iGG+TBQMOKJlygeO6tpAk6yMoHfj3gFGEb8BJpwv4mTrD0u3GxuTXoIGEFVd4D51fitwo7t5DXChqn4S47yZBs63NvA/3dpSXlHlNwh4N/cIWiPA1KVWYc1GxoQvnPVA1gA/iUNeTBhScWhupH0KvrWB8oqqgFPUezf3lHz1Gfm5vQKmE6xWYc1GxoQvoqlMGoJk6Uj1lopDc+vTp1DX2kB1c0/R+kNRScf7GpLts2BMMrAA4kcydaR68zc0N9kDSH36FKJVG6hPOon8LFjgMsku1HMgiEim++BggxFq/H2inkfwHYqbCkNzk+UZm0ifc4jnsxjeqgPXu5tLmLl2a4N59sWklnD6QCpF5FzgkTjkJykEa/JI5B1pu2aNOKZVU8oOV9K/fV5Uax+xutuN5t1/sE70+gh27Q1xoS5jwhVuE9YSEZkG/IOaa6KviEmuEizYl16yLB3brlmjmKUd7aAY6VBU77KuqFLe3LgDVaKax1DXbgt1GRNYuAHkJPff33ptU+CU6GYneQT60kvHO9JwHtKL1RdouHf/IlDlLiAQzesPp1xtoS5j/At3Nt7Rsc5IqkjHO9JENdnV5e6/SVYGb2/aGbXrD/cBxUSyhwhNsgt3QakOwO+BTqp6hogcB5yoqn+Lae6SVCKGecYycCWqya6ud//tmjWKyvXHq2/FmHQXbhPWdOA54E739Rc4/SENMoB4i+eXUSzvSBPRZFff5zMiVZcHFI0xgYUbQNqq6ssicgeAqlaISGWog0SkEGfq90zgGVW9388+o4BHgWxgp6qOdLdPAa7C6Wv5BJikqgfCzG/c+H4ZxaKjN5ESVfOJJeugNiY6wg0g+0SkDe466CIyDNgT7AARyQQeB04FioGlIvK6qn7qtU9L4AmgUFU3i0h7d3tn4CbgOFUtF5GXgUtwakJJJR4dvYmWiJpPLFkHtTHREW4A+SXwOnC0iCwB2gHjQhxzArBeVTcAiMgsnHXUP/Xa51Lgn6q6GUBVv/PJWxMROQw0BbaFmde4imVHr4mddOigtifVTaKFOwprhYiMBHoBAnyuqodDHNYZ2OL1uhgY6rNPTyBbRIqAXOAxVX1BVbeKyEPAZpx1R95U1Tf9nURErgGuAejQoQNFRUXhXFJMHAJ6Sw57JYc8PcT65cWsj8N5y8rKEnrdyaYhlEep5PBpVhuqEDJQjqsoIVcP+d23IZRHXVh5RE+oNdEvCPBWTxFBVf8Z7HA/29TP+QcBY4AmwAci8iGwA6e20h3YDcwWkQmq+lKtBFWfwl12d/DgwTpq1KggWUpPRUVFxPq6U+luNx7lkWjvF+9CN5cAzvonbY7uHXAgQEMoj7qw8oieUDWQc9x/2+M8TPiO+3o0UAQECyDFQBev1/nUboYqxuk434fTz/Iu0N99b6Oq7gAQkX+6568VQEzsJevkkomQLIHUBgKYZBBqQalJACIyD6dD+xv3dUecDvJglgLHiEh3YCtOJ/ilPvu8BkwTkSwgB6eJ6xGgGTBMRJriNGGNAZbV4bpMFNm8TI5kCqQ2EMAkg3A70btVBw/Xtzj9FwG5Q31/BryBM4z3WVVdKyLXuu8/qarrRGQhsBqowhnquwZAROYAK4AKYCVuM5WJv2S92413bSDZAmk6DAQwqS3cAFIkIm8AM3H6MS4BFoU6SFXnA/N9tj3p8/pB4EE/x94N3B1m/kwMJePdbiJqA8kaSI1JlHBHYf3M7VAf4W56SlXnxi5bJtkk291uImoDyRhIjUmksFckdEdcBes0NyZuElUbSLZAakwihTuZ4gXAAzijscT9UVXNi2HeTIwly4iiSDS02kAq/61M+gq3BvJH4BxVXRfLzJj4SaYRRZFqKLWBdPhbmfQUck1017cWPNJLotb6Nj8oLi3n/eJdIdc7t7+VSVbh1kCWicg/gFeBg9UbQzyJbpKYjShKrLrUKuxvZZJVuAEkD9gPnOa1TbFO9ZTV0PoQkk1dRpHZ38okq3CH8U6KdUZM/DWUPoRkEulSuva3Msko1GSKf6HmBIgK7AQWqep7scyYSSwb9RN9tpSuSTehaiD+5p9qDTwoIv9Q1Uejn6WGJRm/qG3UT2zYUrom3YSaTPF5f9tF5EngfZylaE2EkvWLOtnmfEoX1hlu0k3YT6J7c5eZjXZe0ka4tYpk/aK2L7rYsM5wk27qHEDcqdcvx1nLw/hIh+GZ9kUXO9YZbtJJqE70UmqvIlgOLAYmxypTqSxdhmfaF116Sca+NpP6QvWB5MYrI+mirrUK+6I2sVYqOUnZ12ZSX0R9ICawZK5VmIZpr+QkZV+bSX0WQGLAahUmmeTpoaTsazOpzwKIMWkuVw9ZrdjERJ0CiIi0BxpXv1bVzVHPkTEm6qxWbGIhrOncRWSsiHwJbMQZgbUJWBDDfBljjEly4a4H8jtgGPCFqnYHxgBLYpYrY/x4Z9MOnly+iXc27Uh0VowxhB9ADqtqCZAhIhmquggoiF22jKnpnU07+O+23Xx/8DD/3bbbgogxSSDcALJbRJoD/wFmiMhjQEXssmVMTV+U7Av62hgTf+EGkHNxFpT6BbAQ+Ao4J0Z5MqaWnm2aBX1tjIm/cBeU2iciRwLHqOrzItIUyIxt1oz5wSnd2gFOzaNnm2ae18nOphAx6SysACIiVwPX4KwFcjTQGXgSpzPdmLg4pVu7lAkckLzT9RsTLeE2Yd0ADAf2Aqjql0D7WGUq0YpLy3m/eBfFpeWJzopJsPp8FvxNrGlMOgn3QcKDqnqoeg0Qd0p331l604LdNZpq9f0sJOt0/cZES7gBZLGI/BpoIiKnAtcD/xe7bCVOsi7yZOKvvp8Fm1jTpLtwm7BuA3YAn+CsAzIf+E2og0SkUEQ+F5H1InJ7gH1GicgqEVkrIou9trcUkTki8pmIrBORE8PMa71U3zUK2F1jAxeNz0J+bhNOym9twcOkpZA1EBHJAFaral/g6XATFpFM4HHgVJzVC5eKyOuq+qnXPi2BJ4BCVd3szrVV7TFgoaqOE5EcoGm4564Pu2s01YJ9Fmx0lTFhBBBVrRKRj0Wkax0nTzwBWK+qGwBEZBbO8ySfeu1zKfDP6nRV9Tt33zzgZGCiu/0QcKgO564Xm3jOVPP3WbB+MmMc4faBdATWishHgOcRYFUdG+SYzsAWr9fFwFCffXoC2SJSBOQCj6nqC8BROE1mz4lIf2A58HNVrfX4sYhcgzPEmA4dOlBUVBTmJaWPsrKyBnndgcS6PLZmNKciMw9EqKisYvHKtXSuKovZ+erLPh81WXlET7gB5J4I0hY/23xHbmUBg3CeJ2kCfCAiH7rbBwI3qup/3alTbgem1kpQ9SngKYDBgwfrqFGjIshqaisqKqIhXncgsS6PGjWQzAxG9uuT1DUQ+3zUZOURPeE+ib7Y+7WIDMdpflrs/wjAqXF08XqdD2zzs89Ot2axT0TeBfrjzLlVrKr/dfebgxNAjEk46yczxhHuKCxEpEBE/igim4B7gXUhDlkKHCMi3d1O8EuA1332eQ0YISJZ7vQoQ4F1qrod2CIivdz9xlCz78SYhLLRVcaEqIGISE+cL/7xQAnwD0BUdXSohFW1QkR+BryBM2/Ws6q6VkSudd9/UlXXichCYDVQBTyjqmvcJG7Emfk3B9gATIroCo0xxsREqCasz3Cak85R1fUAIjIl3MRVdT7OMyPe2570ef0g8KCfY1cBg8M9lzHGmPgK1YR1IbAdWCQiT4vIGPx3jhtjjGlgggYQVZ2rqhcDvYEiYArQQUT+n4icFof8GWOMSVJhdaKr6j5VnaGqZ+OMplqFjYoyxpgGLexRWNVUdZeq/lVVT4lFhowxxqSGOgcQY4wxBiyAGGOMiZAFEGOMMRGxAGKMMSYiFkCMMcZExAKIMcaYiFgAMcYYExELIMYYYyIS7oJSKevw4cMUFxdz4MCBRGclZlq0aMG6daFm1284YlUejRs3Jj8/n+zs7KinbUwqSvsAUlxcTG5uLt26dUMkPeeBLC0tJTc3N9HZSBqxKA9VpaSkhOLiYrp37x7VtI1JVWnfhHXgwAHatGmTtsHDxIeI0KZNm7SuyRpTV2kfQIDwg8dXX8H110NeHmRkOP9ef72z3TR4dhNiTE0NIoCEZcEC6NcPnnkGSktB1fn3mWec7QsWJDqHxhiTVCyAgFPDGDcO9u+Hw4drvnf4sLN93LiIayKZmZkUFBTQv39/Bg4cyPvvvx+FTP/g2muvZc6cOQBcddVVfPppdJaPX7lyJVdddRXg9AHcdNNN9OjRg379+rFixQq/x4wYMYKCggIKCgro1KkT5513HgBFRUW0aNHC895vf/vbkOffuHEjQ4cO5ZhjjuHiiy/m0KFDtfZZtGiRJ82CggIaN27MvHnzauxz44030rx5c8/refPmcffdd4dbDMaYQFQ1bX4GDRqkvj799NNa22q57jrV7GxVp97h/yc7W/WGG0Kn5UezZs08vy9cuFBPPvnkiNIJ5NJLL9XZs2dHNU1V1XHjxumqVatUVfVf//qXFhYWalVVlX7wwQd6wgknhDz+ggsu0Oeff15VVRctWqRnnXVWnc5/0UUX6cyZM1VVdfLkyfrEE08E3b+kpERbtWql27dv92xbunSpTpgwocbfoKqqSgsKCnTfvn11yo9qmJ+nJLNo0aJEZyGpWHnUBCzTCL9zrQYC8NJLtWsevg4fhhdfrPep9u7dS6tWrQAoKytjzJgxDBw4kOOPP57XXnsNgH379nHWWWfRv39/+vbtyz/+8Q8Ali9fzsiRIxk0aBCnn34633zzTa30R40axbJlywBo3rw5d955J/3792fYsGF8++23AOzYsYMLL7yQIUOGMGTIEJYsWVIrndLSUlavXk3//v0BeO2117jiiisQEYYNG8bu3bv9nt/7+HfeecdTA6krVeWdd95h3LhxAPzkJz/h1VdfDXrMnDlzOOOMM2jatCkAlZWV3Hrrrfzxj3+ssZ+IMGrUqFo1FWPqo7i0nPeLd1FcWp7orMSNBRCAsrLo7uejvLycgoICevfuzVVXXcXUqVMB57mCuXPnsmLFChYtWsTNN9+MqrJw4UI6derExx9/zJo1aygsLOTw4cPceOONzJkzh+XLl3PllVdy5513Bj3vvn37GDZsGB9//DEnn3wyTz/9NAA///nPmTJlCkuXLuWVV17xNFN5W7ZsGX379vW83rp1K126dPG8zs/PZ+vWrQHPPXfuXMaMGUNeXp5n2wcffED//v0544wzWLt2bdC8l5SU0LJlS7KyssI6H8CsWbMYP3685/W0adMYO3YsHTt2rLXv4MGD+c9//hM0PWPCVVxazsy1W3l3cwkz125tMEEk7Z8DCUvz5k6HeTj7RaBJkyasWrUKcL5Er7jiCtasWYOq8utf/5p3332XjIwMtm7dyrfffsvxxx/PLbfcwm233cbZZ5/NiBEjWLNmDWvWrOHUU08FnLtrf1+M3nJycjj77LMBGDRoEG+99RYAb7/9do1+kr1799Z6duKbb76hXbt2ntdOTbemYKOSZs6cWSMwDRw4kK+//prmzZszf/58zjvvPL788suAx9f1fN988w2ffPIJp59+OgcOHGDbtm3Mnj2boqIiv/u3b9+ebdu2BUzPmLrYvKecyipFgcoqZfOecvJzm8Tl3MWl5WzeU07XFk3ids5qFkAAJkxwRlsFa8bKzobLL6/3qU488UR27tzJjh07mD9/Pjt27GD58uVkZ2fTrVs3Dhw4QM+ePVm+fDnz58/njjvu4LTTTuP888+nT58+fPDBB2GfKzs72/Olm5mZSUVFBQBVVVV88MEHNGkS+MPWpEmTGs885Ofns2XLFs/r4uJiOnXq5PfYkpISPvroI+bOnevZ5l0TOfPMM7n++uvZuXMnbdu29ZtG27Zt2b17NxUVFWRlZQU9H8DLL7/M+eefT3Z2NgcOHGDlypWsX7+eHj16ALB//3569OjB+vXrAef5oGDXb0xddG3RhMwMobJKycwQuraIX/CYuXar57zj+3SOaxCxJiyAm292AkQw2dkwZUq9T/XZZ59RWVlJmzZt2LNnD+3btyc7O5tFixbx9ddfA7Bt2zaaNm3KhAkTuOWWW1ixYgW9evVix44dngBy+PDhkM1AgZx22mlMmzbN87q6duTt2GOP9XzZAowdO5YXXngBVeU/S94nNy+PNu07+E1/9uzZnH322TRu3Nizbfv27Z5axUcffURVVRVt2rQBYMyYMbWap0SE0aNHe0aXPf/885x77rkBr2nmzJk1mq/OOusstm/fzqZNm9i0aRNNmzatcT1ffPFFjSY6Y+ojP7cJ4/t05uSubeL6Je6v5hNPFkAAjj4a5syBpk1rB5LsbGf7nDnOfhGo7gMpKCjg4osv5vnnnyczM5PLLruMZcuWMXjwYGbMmEHv3r0B+OSTTzjhhBMoKCjgvvvu4ze/+Q05OTnMmTOH2267jf79+1NQUBDxcOA///nPLFu2jH79+nHcccfx5JNP1tqnd+/e7Nmzh1K3ae/MM8/kqKOO4ugePbjmmmu490+Psqv8EIcqqzjzzDNrNAf59kWA08Hdt29f+vfvz0033cSsWbMQEaqqqli/fj2tW7eulYcHHniAhx9+mB49elBSUsJPf/pTwOmf8W4e27RpE1u2bGHkyJFhl8GiRYs466yzwt7fmFDyc5twUn7ruNYAqms+AnGt+XhEOnwrGX8iHsZbbf16Z6huXp5qRobz7w03ONuT2N69e2OS7sMPP6xPP/10jW2lBw/rttJyz0/pwcP1Oscnn3yiU6ZMqVcavkKVx/bt2/WUU06JKG0bxpv60q08tuzdr0u2lOiWvfsjOp56DOO1PhBvRx8N06Y5P4brrruO2bNn19iWk5mBAAqI+7o++vbty8MPP1yvNOpq8+bN/OlPf4rrOY2Jlfzc+HeeV7MAYgJq3Lgxl/sMHMjJzKB1kxwOVVaRk5lR7wCSCEOGDEl0FoxJCxZATJ2lauAwxkRXTL8FRKRQRD4XkfUicnuAfUaJyCoRWSsii33eyxSRlSJijwwbY0ySiVkNREQygceBU4FiYKmIvK6qn3rt0xJ4AihU1c0i0t4nmZ8D64A8YmjAXwewavuqkPsVHFHAyskrY5kVY4xJGbGsgZwArFfVDap6CJgF+A7kvxT4p6puBlDV76rfEJF84CzgmRjmEYAT808kJzMn6D45mTmclH9SrLNijDEpI5Z9IJ2BLV6vi4GhPvv0BLJFpAjIBR5T1Rfc9x4FfuVuD0hErgGuAejQoUOtqStatGjheZYhkCkDp/DcyueC7pMhGUwZNCVkWv5s3bqVm2++mc8++4yqqioKCwu59957yckJHLR2797N7NmzufrqqwFnqo5f/epXvOhnQsfKykq/+br33nsZPnw4o0ePrnOevXXs2NHvxInl5eVccMEFzJs3j8zMTGbMmMGDDz4IwK233spll10WMM1XX32VK664gqKiIgYOHMjmzZuZMGEClZWVHD58mMmTJ3ue+wjk4MGDTJ48mZUrV9K6dWumT5/OkUceWaM8SktLKSws9ByzdetWLr74Yh544AGmTZvG888/T1ZWFm3btuXxxx+na9eu7Ny5k6uvvrrGk/TVDhw4EHB6lGRVVlaWcnmOJSuPKIp0/G+oH+Ai4Bmv15cDf/HZZxrwIdAMaAt8iRNUzgaecPcZBcwL55z1eQ7kunnXac7vcpT/pdZPzu9y9Pp514eVjq+qqiodMmSIPvvss6qqWlFRoVdeeaXecsstQY/buHGj9unTJ6xzxOo5kGreU6F7mzZtmj766KOq6kyl3r17dy0pKdFdu3Zp9+7dddeuXX6P27t3r44YMUKHDh2qS5cuVVXVgwcP6oEDB1RVtbS0VI888kjdunVr0Hw9/vjjOnnyZFVVnTlzpv74xz/2pB/IwIEDdfHixaqq+s4773imdH/iiSc8x6uqTpw4Ud97771ax9tzIKnPyqMmknQ692Kgi9frfMB39rpiYKGq7lPVncC7QH9gODBWRDbhNH2dIiIvxTCvTD15KhnivzgyJZOpI6dGlO4777xD48aNmTRpkpNWZiaPPPIIzz77LPv372f69Omce+65FBYW0qtXL+655x4Abr/9dr766isKCgq49dZb2bRpk2fqjenTp3Peeedxzjnn0L17d/7617/y8MMPM2DAAIYNG8auXbsAmDhxInPmzGHZsmWeJ+GPP/54z/xYX331FYWFhQwaNIgRI0bw2WefAc5CTieeeCJDhgzxzBzsz4wZMzzTi7zxxhuceuqptG7dmlatWnHqqaeycOFCv8dNnTqVX/3qVzWmOsnJyaFRo0aAU7OoqqoKWbavvfYaP/nJTwAYN24c//73v/1Owljtyy+/5LvvvmPEiBEAjB492jP1+7BhwyguLvbse9555zFjxoyQeTCmIYtlAFkKHCMi3UUkB7gEeN1nn9eAESKSJSJNcZq41qnqHaqar6rd3OPeUdUJMcwrHXM7MqlgUq2+kJzMHCYVTOKI5kdElO7atWsZNGhQjW15eXl07drVMzfTRx99xIwZM1i1ahWzZ89m2bJl3H///Rx99NGsWrXK0yzkbc2aNfz973/no48+4ne/+x1NmzZl5cqVnHjiibzwwgs19h08eDCrVq1i1apVFBYWcssttwBwzTXX8Je//IXly5fz0EMPcf311wPOdO/XXXcdS5cu5Ygj/F/3oUOH2LBhA926dQPCn+595cqVbNmyxTNLsLctW7bQr18/unTpwm233RZ08kTfc2ZlZdGiRQtKSkoC7j9z5kwuvvhiv7P6/u1vf+OMM87wvLbp3o0JLWYBRFUrgJ8Bb+CMpHpZVdeKyLUicq27zzpgIbAa+AinyWtNrPIUir9aSH1qH+A0Efr7wvLefuqpp9KmTRuaNGnCBRdcwHvvvRcy3dGjR5Obm0u7du3Iy8vjnHPOAeD4449n06ZNfo95+eWXWbFiBffffz9lZWW8//77XHTRRRQUFDB58mRPP8eSJUs8c1n5PkhYbefOnbRs2bLG9fjyve6qqiqmTJkS8CnwLl26sHr1atavX8/zzz/vWQArkHDO6c3fHF0AL730EsuWLePWW2/1bLPp3o0JLabPgajqfFXtqapHq+p97rYnVfVJr30eVNXjVLWvqj7qJ40iVa19uxoDvrWQ+tY+APr06eNZIbDa3r172bJlC0e7kzP6fukF+xKsVt3cA5CRkeF5nZGR4Zm23dvatWu5++67mTVrFpmZmVRVVdGyZUtPzWTVqlWsW7cu7DxEMt17aWkpa9asYdSoUXTr1o0PP/yQsWPH1iqfTp060adPn5A1AO9zVlRUsGfPHr+TMgJ8/PHHVFRU1KoNvv3229x33328/vrrNcrUpns3JjR7nNiHdy2kvrUPcKYq379/v6dZqbKykptvvpmJEyd62t/feustdu3aRXl5Oa+++irDhw8nNzc3ohFf/uzZs4dLLrmEF154wbNIVF5eHt27d/fMdaWqfPzxxwAMHz6cWbNmAQTsB2jVqhWVlZWeIHL66afz5ptv8v333/P999/z5ptvcvrpp9c4pkWLFuzcudMzxfqwYcN4/fXXGTx4MMXFxZSXO1NRf//99yxZsoRevXoBcMcdd3hGRB2qrKLsUAWHKqsYO3Yszz//PODM9nvKKacEDHy+072D05w2efJkXn/9ddq3r/kIkk33bkxoFkB8VNdCMiSj3rUPcO7k586dy+zZsznmmGPo2bMnjRs35ve//71nnx/96EdcfvnlFBQUcOGFFzJ48GDatGnD8OHD6du3b42mlUi8+uqrfP3111x99dWeznRwgsPf/vY3+vfvT58+fTxrsj/22GM8/vjjDBkyhD179gRM97TTTvM0t7Vu3ZqpU6d61lm/6667PLWBu+66i9df9+3+qmndunUMHTqU/v37M3LkSG655RaOP/54wJne/ogjjuBQZRW7yg9ReqiCXeWHuHziJEpKSujRowcPP/ww999/vye96mus9vLLL9cKILfeeitlZWWeZryxY8d63rPp3o0JQ6TDt5Lxp97Tubu27d2mJz93sn5T+k2dj62r5557Tm+44YZ6pRHrYbyBrFixQidMmBDz85x22mmqGv5U8tEojxEjRvgdhmzDeFOflUdN2HTu0dUxtyOLJy4OvWMDN2DAAEaPHk1lZSWZmZkxO88bb7wBRH8q+UB27NjBL3/5S1q1ahWT9I1JFxZAEmzixIlMnDgx0dmI2JVXXhm3c8VrKvl27dpx3nnnxSRtY9JJgwggGmAorUk9iZxKXoM8pGhMQ5T2neiNGzempKTE/vObelFVSkpKajw9b0xDl/Y1kPz8fIqLi9mxY0eisxIzBw4csC82L7Eqj8aNG5Ofnx/1dI1JVWkfQLKzs+nevXuisxFTRUVFDBgwINHZSBpWHsbER9o3YRljjIkNCyDGGGMiYgHEGGNMRCSdRieJyA7g60TnIwHaAjsTnYkkYuVRk5VHTVYeNfVS1aArvwaSVp3oqtou0XlIBBFZpqqDE52PZGHlUZOVR01WHjWJyLLQe/lnTVjGGGMiYgHEGGNMRCyApIenEp2BJGPlUZOVR01WHjVFXB5p1YlujDEmfqwGYowxJiIWQIwxxkTEAkgKEZFCEflcRNaLyO1+3h8lIntEZJX7c1ci8hkvocrD3WeUWxZrRSStVwkL4/Nxq9dnY42IVIpI60TkNR7CKI8WIvJ/IvKx+/mYlIh8xkMYZdFKROaKyGoR+UhE+oaVcKRLGdpPfH+ATOAr4CggB/gYOM5nn1HAvETnNYnKoyXwKdDVfd0+0flOZHn47H8O8E6i853gz8evgQfc39sBu4CcROc9QWXxIHC3+3tv4N/hpG01kNRxArBeVTeo6iFgFnBugvOUSOGUx6XAP1V1M4CqfhfnPMZTXT8f44GZcclZYoRTHgrkirPaXHOcAFIR32zGRThlcRzwbwBV/QzoJiIdQiVsASR1dAa2eL0udrf5OtGtki8QkT7xyVpChFMePYFWIlIkIstF5Iq45S7+wv18ICJNgULglTjkK1HCKY9pwLHANuAT4OeqWhWf7MVVOGXxMXABgIicABwJhFz8Jq2mMklz/tbk9R2DvQI4UlXLRORM4FXgmFhnLEHCKY8sYBAwBmgCfCAiH6rqF7HOXAKEUx7VzgGWqOquGOYn0cIpj9OBVcApwNHAWyLyH1XdG+O8xVs4ZXE/8JiIrMIJpisJozZmNZDUUQx08Xqdj3Pn5KGqe1W1zP19PpAtIm3jl8W4Clke7j4LVXWfqu4E3gX6xyl/8RZOeVS7hPRuvoLwymMSThOnqup6YCNO+3+6Cfe7Y5KqFgBX4PQJbQyVsAWQ1LEUOEZEuotIDs6XwOveO4jIEW57bnU1NAMoiXtO4yNkeQCvASNEJMttthkKrItzPuMlnPJARFoAI3HKJp2FUx6bcWqnuO39vYANcc1lfITz3dHSfQ/gKuDdcGpi1oSVIlS1QkR+BryBM6riWVVdKyLXuu8/CYwDrhORCqAcuETdYRXpJpzyUNV1IrIQWA1UAc+o6prE5Tp2wvx8AJwPvKmq+xKU1bgIszx+B0wXkU9wmnluc2uqaSXMsjgWeEFEKnFGLv40nLRtKhNjjDERsSYsY4wxEbEAYowxJiIWQIwxxkTEAogxxpiIWAAxxhgTEQsgfrizlFbP4PqxiPxSRDLc9waLyJ/d3xuJyNvuvheLyAj3mFUi0iSxV+GfiMwXkZZ12P9/RWSr1wyuYyM4529F5H/c33/hPpMRUX6CnGOTiHzi/r3eFJEj6ptmHc8fk+sKcC7PZ7AOx0StfNzPxC3u756/bYB9C9xZEep6jiIRGRzu9gBpjBKRedE4b5D9f1E9RY6ITBeRjW4ZfyEiL4hIZ699/f4NQmxv63O+s0XknrpcUyxZAPGvXFULVLUPcCpwJnA3gKouU9Wb3P0GANnuvv8ALgMecl+XhzqJOOL6N1DVM1V1dx0Pe8R9QvUi4Nm65llV71LVt92XvwCaer0XSX4CGa2q/YFlODOtesSyrEUkk9heVw0+n8G6iHr5+Pxt/SnA+f+TdkQkC7gS+LvX5lvdMu6FMx3IIq8H9CDw3yDg38bHv4Cx3jcriWQBJAR3BtdrgJ+5/8lGicg8EWkPvAQUuHfnk4EfA3eJyAzwrL+wVJw59u9xt3UTkXUi8gTO3FVdQuz3tFurebO6ViMiPdyaz8ciskJEjg50Pl/VdzXB0g9SFutw5sdpKyLj3bumNSLygJt2pnsXtsZ9b4q7fbqIjBORm4BOOP+pFvnk5wERud4rn/8rIjeHe10+3gV6BCjrB73yd7Gb/igReVec9RA+FZEn5YcaZ63rdLeXiXP3/V/gzkDX5f7+S/f4NSLyizD+vje5+VgtIrP8/A09d9ZuOT0rzp3zBreM61M+fstaRO4UZz2Jt3G+HKu3TxeRce7vQ0Tkffdz+ZE4T73/FrhYfqilN3Pzu1REVorIue6xTURklnvef+DMXRYW9zr+4/5fWCEiJ3m9nRfg73qaiHzg7j9bRJr7pOn3s+zjFGCFqtaaM8qdHuURYDtwhp9j3wV61GG7J12gCDg70D5xFYv551P9Byjzs+17oANea27gs/4GMB0Y5/5+Gs5i9YITqOcBJwPdcJ6KHhbGfhVAgbvfy8AE9/f/Aue7vzfGufP1m46f69gEtA2Wvs/+/wvc4v4+FGcOnc4400C0w5nN4B3gPJyJC9/yOraln3LZBLT1k58BwGKv7Z8CXet6Xe7v04AH/JT1hcBbOE/jdnCvoaP7dzyAs15CprvPOJygUOs63bQU+LG/8/tc1yCcyema4UwZvta91mB/321AI+8y9LnWUfzwGfxf4H2gkXu+EpxacSTlE+izWH0NTYE8YD0/fCamu2WVgzMNyBB3e55bZhOBaV75+L3XdbYEvnDL5pc4T0gD9HPLZrCf6yjy3e7mq7H7+zHAMq9y8vd3bYvzRd3M3e824C7v9AnwWfY57z3Ajf7+/3ttexTnCXe/f4Nwt/ukeRnwl0R/T6qqTWVSB/5mtAzmNPdnpfu6Oc6HezPwtap+GMZ+G1V1lbt9Oc4c/blAZ1WdC6CqB8C5owqQzrtB8lgr/QD7TRGRCUApcDHOf7AiVd3hnnsGzhfN74CjROQvOFXtN4OcuwZVXSki7UWkE84X9vequtm9ow73uhaJMxXDauA3OF9Q3mX9I2CmqlYC34qzQuEQYC/wkapucK9nprvv4QDX+SpQSXjTof8ImKvu1CEi8k9gBM5cRIHKfzUwQ0Redc8Vyr9U9SBwUES+wwmOxX72C1U+gT5Due417HevodYcWzi1km9UdSk4k/O5+/rudxpOE8wt7uvGODcKJwN/do9dLSKrw7juatnANBEpwPm79PR6z9/f9QDO+hdL3PzlAB/4pLmB0J/ljoSeW823AHz/BqG2+/Mdzs1NwlkACYOIHIXzwfwOZ86YsA4D/qCqf/VJqxuwL8z9DnptqsSp1gcKZH7TCcFf+v48oqoPeeXtPH87qer3ItIfZ5rsG3Ca9K6sQ37m4NwhHoGz6A3U7bpGq9dcRuJ0YvuWdSC+c/poiP0PuIEolGBpBCr/s3C+UMcCU0Wkj/ppJgmSTqD/1+GUj7/P4i8IPDW897HhzIskwIWq+rnPOQjzeH+mAN/izLScgRMgqgX6u76lquMDJRjmZ7kcJwAGMwB3oSZXjb9BGNv9aeyeO+GsDyQEEWkHPIlTDa/LB/wN4MrqtlUR6SxOv0mk+wGeO7vi6i9xcUaCNa1rOvX0X2CkOH0XmTir2y122/wzVPUVYCow0M+xpTh3tP7MwpkpdBxOMIHoXte7OO3xme7f9WTgI/e9E8SZrTQDp5b1XqDrDJB2oOt6FzhPRJqKSDOcyQz/EyiD7vm7qOoi4Fc4tYTmgfaPskBl/S5wvttPkYuznoivz4BOIjLEPTZXnE5m33J5A7hRxDNr9AB3+7s4TTOIsx53vzrkuwVO7acKuBynuaqav7/rh8BwEenhnq+piHjXWgjzs7yOAP0V4rgJp5aysA7XEo6eQFJMCmo1EP+aiLOwSjZOW+yLwMN1SUBV3xSRY3EWMQIoAybg3CHWeT8flwN/FZHf4jSzXBQknagv46qq34jIHcAinLu5+ar6mnvH9pz8MJrnDj+HPwUsEJFvVHW0T7pr3S+orar6jbstmtc1FzgRZ/U1BX6lqttFpDdOE8b9wPE4X2ZzVbXK33UGSNvvdanqChGZzg+B6hm3ua5bgHQygZfE6YAWnNrf7giutc4ClbV7Df/AWXzpa/wEQFU9JM6ghL+IMxigHPgfnLK73f3/9AecZs5HgdVuENmE0yH8/3A+O6vd83xEYP8SkcPu7x/gjFp6RUQucs/nXasK9HedCMwUkUbufr/B6Y+p1pnQn+UFON8N3h4Ukak4/TIf4tQsDgW5llBWi0j1Kokvq+ovgdEB8hN3NhuvafBEZBROp3ByjGwxKUNE5uLciHwZp/N1AP6uqmPicb5QrAnLGGMidztOM1W8dAVujuP5grIaiDHGmIhYDcQYY0xELIAYY4yJiAUQY4wxEbEAYowxJiIWQIwxxkTk/wOWvLD806iRTAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "from matplotlib import pyplot as plt\n", "\n", "#Worst one wihtout optimization - You can get thess baseline value from the values you found in Notebook 2\n", "baseline_sd = 0.75 \n", "baseline_auc = 0.74\n", "\n", "#REAL ONE After optimization - follow steps from Notebook 2 for the training job created in this notebook to find these values. \n", "optimized_sd = 0.43\n", "optimized_auc = 0.72\n", "#combined = 0.75\n", "ax = plt.subplots()\n", "#Add grid \n", "plt.grid()\n", "\n", "#Graph pareto front chart \n", "plt.title(\"Pareto Chart\")\n", "plt.xlabel(\"Difference in Positive Proportions in Predicted Labels (DPPL)\")\n", "plt.ylabel(\"Area Under Curve (AUC)\")\n", "\n", "#Plot co ordinates \n", "plt.plot(pf_X, pf_Y, linestyle='--', linewidth=3, marker='o', c='#8B8000', mfc='b')\n", "#Plot additional points\n", "plt.plot(sd_float_list, auc_float_list, linestyle='None', marker='o', markersize=3, color='#86C5DA')\n", "#Show baseline SD & AUC \n", "plt.plot(baseline_sd, baseline_auc, \"ro\", markersize='10', label=(\"Baseline (\"+str(baseline_sd)+\", \"+str(baseline_auc)+\")\"))\n", "#Show optimized SD & AUC \n", "plt.plot(optimized_sd, optimized_auc, \"gv\", markersize='10', label=(\"Optimized (\"+str(optimized_sd)+\", \"+str(optimized_auc)+\")\"))\n", "#Legend\n", "plt.legend(loc=\"lower left\")\n", "\n", "#Print Baseline Values \n", "print(\"Baseline Statistical Disparity: \",baseline_sd)\n", "print(\"Baseline Area Under Curve: \",baseline_auc,\"\\n\")\n", "\n", "#Print Optimized Values\n", "print(\"Optimized Statistical Disparity: \",optimized_sd)\n", "print(\"Optimized Area Under Curve: \",optimized_auc,\"\\n\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The cell below provides a range of visualization for the tuning job above. This visualization and its libraries were provided by the following repo:\n", "\n", "https://github.com/aws-samples/amazon-sagemaker-amt-visualize/blob/main/1_tuning_of_builtin_xgboost.ipynb " ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Setting altair renderer to default.\n", "Tuning job xgb-tuner-27-00-32-14 status: Completed\n", "\n", "Number of training jobs with valid objective: 100\n", "Lowest: 0.5139889717102051 Highest 0.6838629841804504\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
etagammamax_depthmin_child_weightTrainingJobNameTrainingJobStatusTrainingStartTimeTrainingEndTimeTrainingElapsedTimeSecondsTuningJobNamevalidation:auc
480.5486120.0124214.00.000066xgb-tuner-27-00-32-14-052-be514283Completed2023-01-27 00:39:38+00:002023-01-27 00:40:47+00:0069.0xgb-tuner-27-00-32-140.683863
680.5444510.0009044.00.000078xgb-tuner-27-00-32-14-032-c6499eb4Completed2023-01-27 00:37:02+00:002023-01-27 00:37:40+00:0038.0xgb-tuner-27-00-32-140.678610
710.6438510.0000884.00.000003xgb-tuner-27-00-32-14-029-43d2f514Completed2023-01-27 00:36:36+00:002023-01-27 00:37:08+00:0032.0xgb-tuner-27-00-32-140.677474
860.6483770.0000984.00.000018xgb-tuner-27-00-32-14-014-7c090c65Completed2023-01-27 00:35:18+00:002023-01-27 00:35:50+00:0032.0xgb-tuner-27-00-32-140.669266
810.7185540.0549624.00.000013xgb-tuner-27-00-32-14-019-b6885f1dCompleted2023-01-27 00:35:35+00:002023-01-27 00:36:12+00:0037.0xgb-tuner-27-00-32-140.667242
350.5248440.0667383.00.000008xgb-tuner-27-00-32-14-065-f1f06d11Completed2023-01-27 00:40:21+00:002023-01-27 00:40:58+00:0037.0xgb-tuner-27-00-32-140.665081
870.6117080.0000914.00.004199xgb-tuner-27-00-32-14-013-e0d8f46aCompleted2023-01-27 00:35:16+00:002023-01-27 00:35:53+00:0037.0xgb-tuner-27-00-32-140.663159
440.7305830.0026274.00.000487xgb-tuner-27-00-32-14-056-250a354cCompleted2023-01-27 00:39:24+00:002023-01-27 00:40:01+00:0037.0xgb-tuner-27-00-32-140.659589
880.4742160.0000163.00.000453xgb-tuner-27-00-32-14-012-33b1a582Completed2023-01-27 00:35:14+00:002023-01-27 00:35:51+00:0037.0xgb-tuner-27-00-32-140.658692
400.4987170.0002305.00.000013xgb-tuner-27-00-32-14-060-c903e078Completed2023-01-27 00:39:45+00:002023-01-27 00:40:22+00:0037.0xgb-tuner-27-00-32-140.657385
\n", "
" ], "text/plain": [ " eta gamma max_depth min_child_weight TrainingJobName TrainingJobStatus TrainingStartTime TrainingEndTime TrainingElapsedTimeSeconds TuningJobName validation:auc\n", "48 0.548612 0.012421 4.0 0.000066 xgb-tuner-27-00-32-14-052-be514283 Completed 2023-01-27 00:39:38+00:00 2023-01-27 00:40:47+00:00 69.0 xgb-tuner-27-00-32-14 0.683863\n", "68 0.544451 0.000904 4.0 0.000078 xgb-tuner-27-00-32-14-032-c6499eb4 Completed 2023-01-27 00:37:02+00:00 2023-01-27 00:37:40+00:00 38.0 xgb-tuner-27-00-32-14 0.678610\n", "71 0.643851 0.000088 4.0 0.000003 xgb-tuner-27-00-32-14-029-43d2f514 Completed 2023-01-27 00:36:36+00:00 2023-01-27 00:37:08+00:00 32.0 xgb-tuner-27-00-32-14 0.677474\n", "86 0.648377 0.000098 4.0 0.000018 xgb-tuner-27-00-32-14-014-7c090c65 Completed 2023-01-27 00:35:18+00:00 2023-01-27 00:35:50+00:00 32.0 xgb-tuner-27-00-32-14 0.669266\n", "81 0.718554 0.054962 4.0 0.000013 xgb-tuner-27-00-32-14-019-b6885f1d Completed 2023-01-27 00:35:35+00:00 2023-01-27 00:36:12+00:00 37.0 xgb-tuner-27-00-32-14 0.667242\n", "35 0.524844 0.066738 3.0 0.000008 xgb-tuner-27-00-32-14-065-f1f06d11 Completed 2023-01-27 00:40:21+00:00 2023-01-27 00:40:58+00:00 37.0 xgb-tuner-27-00-32-14 0.665081\n", "87 0.611708 0.000091 4.0 0.004199 xgb-tuner-27-00-32-14-013-e0d8f46a Completed 2023-01-27 00:35:16+00:00 2023-01-27 00:35:53+00:00 37.0 xgb-tuner-27-00-32-14 0.663159\n", "44 0.730583 0.002627 4.0 0.000487 xgb-tuner-27-00-32-14-056-250a354c Completed 2023-01-27 00:39:24+00:00 2023-01-27 00:40:01+00:00 37.0 xgb-tuner-27-00-32-14 0.659589\n", "88 0.474216 0.000016 3.0 0.000453 xgb-tuner-27-00-32-14-012-33b1a582 Completed 2023-01-27 00:35:14+00:00 2023-01-27 00:35:51+00:00 37.0 xgb-tuner-27-00-32-14 0.658692\n", "40 0.498717 0.000230 5.0 0.000013 xgb-tuner-27-00-32-14-060-c903e078 Completed 2023-01-27 00:39:45+00:00 2023-01-27 00:40:22+00:00 37.0 xgb-tuner-27-00-32-14 0.657385" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "
\n", "" ], "text/plain": [ "alt.VConcatChart(...)" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#!pip install altair\n", "from amtviz import visualize_tuning_job\n", "visualize_tuning_job(tuner, trials_only=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Clean-up\n", "\n", "Remove any resources to save on AWS cost." ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "#CLEAN UP - Uncomment code in this cell to Spin Down Endpoint\n", "\n", "# Create a low-level SageMaker service client.\n", "sagemaker_client = boto3.client('sagemaker', region_name=\"us-east-2\")\n", "\n", "# Delete endpoint\n", "sagemaker_client.delete_endpoint(EndpointName=endpoint_name)\n", "\n", "#Clean-up temporary files in s3 bucket.Removing these temp files will impact re-creating the 'pareto frontier' chart\n", "\n", "import boto3\n", "s3_client = boto3.client('s3')\n", "session = Session()\n", "s3_bucket = session.default_bucket()\n", "s3_bucket_prefix = \"sagemaker/sagemaker-amt-credit-risk-model/data/output/tmp/\"\n", "response = s3_client.list_objects_v2(Bucket=s3_bucket, Prefix=s3_bucket_prefix)\n", "for object in response['Contents']:\n", " s3_client.delete_object(Bucket=s3_bucket, Key=object['Key'])" ] } ], "metadata": { "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Data Science 2.0)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-2:429704687514:image/sagemaker-data-science-38" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.13" } }, "nbformat": 4, "nbformat_minor": 4 }