{ "cells": [ { "cell_type": "markdown", "id": "df276921", "metadata": {}, "source": [ "# Churn Prediction with Text and Interpretability" ] }, { "cell_type": "markdown", "id": "1c1f7791", "metadata": {}, "source": [ "This notebook runs the entire churn prediction pipeline from data preparation to model evaluation and interpretation.\n", "\n", "Alternatively, everything can be run from the terminal as well (see README.md).\n", "\n", "Prerequisite: Dataset has been created (see README.md)." ] }, { "cell_type": "markdown", "id": "032bb6ce", "metadata": {}, "source": [ "### Setup" ] }, { "cell_type": "code", "execution_count": 1, "id": "a32ade35", "metadata": {}, "outputs": [], "source": [ "import os\n", "import pandas as pd\n", "from matplotlib import pyplot as plt\n", "\n", "os.chdir(\"../scripts\")\n", "\n", "import preprocess\n", "import train\n", "import interpret" ] }, { "cell_type": "code", "execution_count": 3, "id": "9d500e76", "metadata": {}, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "markdown", "id": "4b8b1336", "metadata": {}, "source": [ "### Load and Prepare the Data" ] }, { "cell_type": "code", "execution_count": 4, "id": "94c30e10", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
churnchat_logstateaccount_lengtharea_codeinternational_planvoice_mail_plannumber_vmail_messagestotal_day_minutestotal_day_calls...total_eve_minutestotal_eve_callstotal_eve_chargetotal_night_minutestotal_night_callstotal_night_chargetotal_intl_minutestotal_intl_callstotal_intl_chargenumber_customer_service_calls
0noCustomer: Well, the only thing that I'm consid...CT134area_code_408nono0177.291...228.710519.44194.31138.748.932.402
1yesCustomer: Well, I just want to be able to canc...WV78area_code_408nono0226.388...306.28126.03200.91209.047.8112.111
2noCustomer: I would like data.\\nTelCom Agent: Ok...IN88area_code_415nono0183.593...170.58014.49193.8888.728.352.243
\n", "

3 rows × 21 columns

\n", "
" ], "text/plain": [ " churn chat_log state \\\n", "0 no Customer: Well, the only thing that I'm consid... CT \n", "1 yes Customer: Well, I just want to be able to canc... WV \n", "2 no Customer: I would like data.\\nTelCom Agent: Ok... IN \n", "\n", " account_length area_code international_plan voice_mail_plan \\\n", "0 134 area_code_408 no no \n", "1 78 area_code_408 no no \n", "2 88 area_code_415 no no \n", "\n", " number_vmail_messages total_day_minutes total_day_calls ... \\\n", "0 0 177.2 91 ... \n", "1 0 226.3 88 ... \n", "2 0 183.5 93 ... \n", "\n", " total_eve_minutes total_eve_calls total_eve_charge total_night_minutes \\\n", "0 228.7 105 19.44 194.3 \n", "1 306.2 81 26.03 200.9 \n", "2 170.5 80 14.49 193.8 \n", "\n", " total_night_calls total_night_charge total_intl_minutes \\\n", "0 113 8.74 8.9 \n", "1 120 9.04 7.8 \n", "2 88 8.72 8.3 \n", "\n", " total_intl_calls total_intl_charge number_customer_service_calls \n", "0 3 2.40 2 \n", "1 11 2.11 1 \n", "2 5 2.24 3 \n", "\n", "[3 rows x 21 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.read_csv('../data/churn_dataset.csv')\n", "df.head(3)" ] }, { "cell_type": "code", "execution_count": null, "id": "ede78f6b", "metadata": {}, "outputs": [], "source": [ "X_train, X_test, y_train, y_test = preprocess.prep_data(df, use_existing=False, test_size=0.33)" ] }, { "cell_type": "code", "execution_count": 27, "id": "5d0f6d4f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "((2233, 841), (1100, 841), (2233, 1), (1100, 1))" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train.shape, X_test.shape, y_train.shape, y_test.shape" ] }, { "cell_type": "markdown", "id": "c477f751", "metadata": {}, "source": [ "### Train and Evaluate the Model" ] }, { "cell_type": "code", "execution_count": 28, "id": "0d68bf05", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/ec2-user/SageMaker/churn_test/scripts\n" ] } ], "source": [ "!pwd" ] }, { "cell_type": "code", "execution_count": 29, "id": "eb2e2878", "metadata": { "collapsed": true, "jupyter": { "outputs_hidden": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "starting epoch: 1\n", "Train Epoch: 1, train-auc-score: 0.9561\n", "test_auc_score: 0.9422\n", "starting epoch: 2\n", "Train Epoch: 2, train-auc-score: 0.9571\n", "test_auc_score: 0.9453\n", "starting epoch: 3\n", "Train Epoch: 3, train-auc-score: 0.9602\n", "test_auc_score: 0.9480\n", "starting epoch: 4\n", "Train Epoch: 4, train-auc-score: 0.9594\n", "test_auc_score: 0.9467\n", "starting epoch: 5\n", "Train Epoch: 5, train-auc-score: 0.9628\n", "test_auc_score: 0.9529\n", "starting epoch: 6\n", "Train Epoch: 6, train-auc-score: 0.9711\n", "test_auc_score: 0.9555\n", "starting epoch: 7\n", "Train Epoch: 7, train-auc-score: 0.9756\n", "test_auc_score: 0.9586\n", "starting epoch: 8\n", "Train Epoch: 8, train-auc-score: 0.9804\n", "test_auc_score: 0.9598\n", "starting epoch: 9\n", "Train Epoch: 9, train-auc-score: 0.9810\n", "test_auc_score: 0.9618\n", "starting epoch: 10\n", "Train Epoch: 10, train-auc-score: 0.9644\n", "test_auc_score: 0.9552\n", "saving scores\n", "saving model\n" ] } ], "source": [ "# train the model\n", "train.train(\n", " X=X_train,\n", " y=y_train,\n", " X_test=X_test,\n", " y_test=y_test\n", ")" ] }, { "cell_type": "code", "execution_count": 30, "id": "294bd939", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABBqElEQVR4nO3deXxU5fX48c/JThJCIAlhCZCwyo6sKoggFcSVpVVwRxF3qba2aq1+tVZtq/7UiiBaVBTBFaVKhWpFQJAthB0lYU3YEraQQPbz++MOYZJMIEAmk+W8X695MXPvc+89mZfeM8997n2OqCrGGGNMaX6+DsAYY0z1ZAnCGGOMR5YgjDHGeGQJwhhjjEeWIIwxxnhkCcIYY4xHliCMMcZ4ZAnC1EkiskBEDolIsIfl40stGyQiqW6fRUQeFJH1IpItIqki8omIdC3nWJ1FZL7reIdFZJWIXOGdv8yYymMJwtQ5IhIPXAwocM1Z7OJVYCLwINAIaA98AVxZTvt/A/8FYoHGru0yz+K45RKRgMrcnzFgCcLUTbcAPwHvAreeyYYi0g64Dxirqv9T1VxVPaaqM1T1BQ/to4EE4C1VzXO9flTVxW5trhWRJBHJFJEUEbnctbyZiMwRkYMikiwid7pt838i8qmIfCAimcBtItJARP4lIntEJE1EnhUR/7P4fowBLEGYuukWYIbrNUxEYs9g2yFAqqour2D7A0Ay8IGIjCh9LBHpC0wHHgEigYHAdtfqmUAq0Az4NfCciAxx2/xa4FPXdjOA94ACoC1wPjAUKHG5zJgzYQnC1CkiMgBoBXysqquAFOCGM9hFFLCnoo3VmexsMM5J/yVgj4gsdPVEAO4Apqnqf1W1SFXTVHWziLQABgB/VNUcVU0C3gZudtv9UlX9QlWLgAhgOPBbVc1W1f3A/wPGnMHfZkwJliBMXXMrMF9VM1yfP6TkZaYCILDUNoFAvuv9AaDpmRxQVVNV9X5VbYOTnLJxeg0ALXCSVGnNgIOqetRt2Q6gudvnXW7vW7ni3OMaCD8MvIkz5mHMWbGBLVNniEg94DrAX0T2uhYHA5Ei0l1V1wA7gfhSmybgnJwBvgMmiUhvVV15pjGo6i4RmYRz+Qick3wbD013A41EpL5bkmgJpLnvzu39LiAXiFbVgjONyxhPrAdh6pIRQCHQCejhenUEFuGMSwB8BIwTkb6u21nbAw8BswBUdQvwBjDTdftrkIiEiMgYEXm09AFFpKGIPC0ibUXEzzVofTvOIDnAv1zHG+Ja31xEzlPVXcAS4HnX/rvhXI6a4ekPU9U9wHzgJRGJcO2rjYhccm5fmanLLEGYuuRW4B1V3amqe0+8gNeBG0UkQFXnAY8C7wBHgLk4g79T3fbzoGubScBhnEtEI3FuZy0tD6dH8i3Ora3rcX7p3wbgGuwehzNecAT4AedyEcBY17a7gdnAU6r631P8fbcAQcBG4BDOAPYZXQ4zxp1YwSBjjDGeWA/CGGOMR5YgjDHGeGQJwhhjjEeWIIwxxnhUq56DiI6O1vj4eF+HYYwxNcaqVasyVDXG07palSDi4+NZufKMn10yxpg6S0R2lLfOLjEZY4zxyBKEMcYYjyxBGGOM8ahWjUF4kp+fT2pqKjk5Ob4Opc4JCQkhLi6OwMDSk6MaY2qCWp8gUlNTqV+/PvHx8YiIr8OpM1SVAwcOkJqaSkJCgq/DMcachVp/iSknJ4eoqChLDlVMRIiKirKemzE1mFcThIhcLiI/u+rpljcV8mwRWSsiy0Wki9u6h0Rkg4isF5GZIhJyDnGc7abmHNj3bkzN5rVLTK5i6ZOAy3Dq6q4QkTmqutGt2eNAkqqOFJHzXO2HiEhznCmVO6nqcRH5GKd04rveitcYY04lt6CQ9KO57MvMZX9mDnszcxDguj4tCA2qnVfrvflX9QWSVXUrgIjMwimy7p4gOgHPA7jq8Ma7FXUPAOqJSD4QijMnvjHGVKrCIuVAlnPi35uZw77MHPZn5rAvM5d9R3PYeySH/UdzOZid53H7txdv49kRXRjUofZVd/VmgmhOyZq5qUC/Um3WAKOAxSLSF6dQSpyqrhKRF3HKPx7HqSE839NBRGQCMAGgZcuWlfsXVILDhw/z4Ycfcu+9957RdldccQUffvghkZGR3gnMmFpOVTl8LN/tpJ/Lvswc10k/l/1HneXpR3MpKlUWx08gOjyY2IgQ4hrWo1erhsRGhBAbEUzjiBBi6zvvk/dn8fjsddz2zgqu7t6MJ6/qREz9YN/8wV7gzQTh6QJ06epELwCvikgSsA5YDRSISEOc3kYCTsWuT0TkJlX9oMwOVafiqvbVu3fvalf96PDhw7zxxhtlEkRhYSH+/v7lbjd37lxvh1Yhp4vTGF/Iyi1wftm7Tvj7Tpz8M0++35+ZS15hUZltG4UF0bi+c/I/r0l9YiNCaBwRQhNXAoiNCCEqLIgA/9MP0UaFBzN34sVMXpDCG9+n8MPP+3n8io5c17sFfn41fwzOmwkiFWjh9jmOUpeJVDUTp9wi4oxobnO9hgHbVDXdte5z4CKgTII4E0//ewMbd2eeyy7K6NQsgqeu7lzu+kcffZSUlBR69OhBYGAg4eHhNG3alKSkJDZu3MiIESPYtWsXOTk5TJw4kQkTJgAn55XKyspi+PDhDBgwgCVLltC8eXO+/PJL6tWr5/F4r732GlOmTCEgIIBOnToxa9YssrKyeOCBB1i5ciUiwlNPPcXo0aOZOXMmzz33HKrKlVdeyd/+9jcAwsPDefjhh5k3bx4vvfQS27dv57XXXiMvL49+/frxxhtvAHDHHXcU7/P222/noYceqtTv1tRtRUXKnswcUvZnkZLueu3PJjk9i/SjuWXa1w8OoLHrBN83vpHzS9/1OTYimMb1Q2gcEUxwQOX+4AkO8Oe3v2rPVd2a8fjsdTz6+To+X53GcyO70rZxeKUeq6p5M0GsANqJSAKQhjPIfIN7AxGJBI6pah4wHlioqpkishO4QERCcS4xDQFq5Cx8L7zwAuvXrycpKYkFCxZw5ZVXsn79+uJnA6ZNm0ajRo04fvw4ffr0YfTo0URFRZXYx5YtW5g5cyZvvfUW1113HZ999hk33XRTucfbtm0bwcHBHD58GIC//OUvNGjQgHXr1gFw6NAhdu/ezR//+EdWrVpFw4YNGTp0KF988QUjRowgOzubLl268Mwzz7Bp0yb+9re/8eOPPxIYGMi9997LjBkz6Ny5M2lpaaxfvx6g+FjGnKmc/EK2H8gmZX/2yUTgSgbH8wuL20WEBNC2cTiDO8SQEB1Os8gQ18k/hMb1gwkL9u1AcdvG4cy68wI+WbWL5+Zu5opXF3HPoDbcO7hNpSelquK1b1RVC0TkfmAe4A9MU9UNInK3a/0UoCMwXUQKcQav73CtWyYinwKJQAHOpaepHg5zRk71S7+q9O3bt8SDY6+99hqzZ88GYNeuXWzZsqVMgkhISKBHjx4A9OrVi+3bt5e7/27dunHjjTcyYsQIRowYAcC3337LrFmzits0bNiQhQsXMmjQIGJinFl+b7zxRhYuXMiIESPw9/dn9OjRAHz33XesWrWKPn36AHD8+HEaN27M1VdfzdatW3nggQe48sorGTp06Dl9L6b2O5id5zrxn0gCTkLYdfBY8RiACDSPrEebmHD69YuiTUw4bWLCaNM4nKiwoGp/67Sfn3B9n5Zcel4sf/lqI69+t4Wv1u7muZFd6dc66vQ7qGa8mnJVdS4wt9SyKW7vlwLtytn2KeApb8bnC2FhYcXvFyxYwLfffsvSpUsJDQ1l0KBBHh8sCw4+Oejl7+/P8ePHy93/119/zcKFC5kzZw5/+ctf2LBhA6pa5n8s1fKHa0JCQorHHVSVW2+9leeff75MuzVr1jBv3jwmTZrExx9/zLRp08r/w02dUFikpB46VtwDONkjyC5xF1BwgB+tY8Lp2rwBI3o0p23jcNrEhJMQHUa9oJr5a9tdTP1gXht7PqN6NueJL9Zz/dSfGNOnBY8N70iD0Joz9UztvHm3Gqlfvz5Hjx71uO7IkSM0bNiQ0NBQNm/ezE8//XROxyoqKmLXrl0MHjyYAQMG8OGHH5KVlcXQoUN5/fXXeeWVVwDnElO/fv2YOHEiGRkZNGzYkJkzZ/LAAw+U2eeQIUO49tpreeihh2jcuDEHDx7k6NGjhIWFERQUxOjRo2nTpg233XbbOcVuapZjeQVsdfUAnB6B835rRjZ5BScHhqPDg2gdE86wzk1oExNWnAiaR9arFYO4pzOoQ2PmPzSQV7/dwtuLt/Htpn08eXVnru7WtNr3hsAShNdFRUXRv39/unTpQr169YiNjS1ed/nllzNlyhS6detGhw4duOCCC87pWIWFhdx0000cOXIEVeWhhx4iMjKSJ554gvvuu48uXbrg7+/PU089xahRo3j++ecZPHgwqsoVV1zBtddeW2afnTp14tlnn2Xo0KEUFRURGBjIpEmTqFevHuPGjaOoyDkZeOphmNqjqEj54Zd0ZizbyaY9maQdPtmL9RNoFRVGm5gwLmkf41wWauxcGooMDfJh1NVDaFAAj13RkWt6NOOxz9fx4MzVfLYqlWdHdKFFo1Bfh3dKcqpLDTVN7969tXRFuU2bNtGxY0cfRWTs+6/ZsnML+DwxlXd+3M7WjGxiI4K5qE20My4QE07bxuG0jAqtsYOwVa2wSJm+dDsvzvuZQlUevqw9t/dPqNAttd4iIqtUtbenddaDMMaUkXb4ONOXbGfm8p1k5hTQPa4Br47pwRVdmxLow5NZTefvJ4zrn8Cwzk148sv1PDd3M1+s3s3zo7rSvUWkr8MrwxJEDXXffffx448/llg2ceJExo0b56OITE2nqiTuPMS0xdv5ZsNeAC7v0oTb+yfQs2VkjbhmXlM0i6zHW7f05pv1e3lqzgZGvvEjt14Uz++GdiDcx7fruqs+kZgzMmnSJF+HYGqJ/MIi5q7bw7TF21iTeoSIkADGX5zALRfG0zzS8wOZ5tyJCMO7NqV/u2j+/s1m3l2ynXnr9/LMtV34VafY0++gCliCMKaOOpSdx4fLdzJ96Xb2ZebSOjqMv1zbmdG94mrt7KQVVpALxw6Ueh10/s3OOLksNxP8Akq+/APBLxD8/N3eB4D/iTaBJd5H+AfybJQ/d/fP59/r9/PDjDnsa96Iq3u0ICIstNR+S7937SsgBBpX/lhfHf+vwJi6Z8u+o0z7cTuzV6eSk1/Exe2ieWFUNy5pH1M7bz0tKoTjhzyc8A9Atodlxw5Cnudb0wEIiYTQqJOvokIoKnBe+cdPvi/Md73Pd9oU5nt4X1C82zjgHoBAYD/gcXrScoTFwCPJZ/PtnJIlCGPqgKIiZeGWdP61eBuLtmQQHODHqJ7Nue2iBDo0qe/r8M5MYT4cSfVwss8o+Uv/xOv4IcrOE+oSGOY60Tdy/o1uV/JziVc01Gvo/GKvLKpuCSbflTgK2ZlxhBf/s4E1OzPo3iyMh4e0Jj4yyNU23y35uF7inbvILEF42dlO9w3wyiuvMGHCBEJDq/e90qb6OpZXwOeJabzz4zZS0rNpXD+YR4Z1YGzfljQKq8bPKBTkweEdcHBrydeBFDi8E7Sw7DZ+gSdP5mFR0KSLh5N8I+dEf+J9oI/HWESchOMfAJwsmtkyPIZX727D54lpPPv1Rv4zI4O7Brbh/kvbEhJYdbcU23MQXrZ9+3auuuqq4kntzsSJGV2jo6O9EFlZ3pja29fff121+/Bxpi/dwczlOzlyPJ9ucQ24vX8CV3RtSlBANblNNT+nZBI4kHLy/ZFdoG5TdQfVh6jW0Kg1NGoDDeMhPLbkr/3g+s4Jt5Y5mJ3Hs19v5PPENOKjQnluZFcualt55wR7DsKH3Kf7vuyyy2jcuDEff/wxubm5jBw5kqeffprs7Gyuu+46UlNTKSws5M9//jP79u1j9+7dDB48mOjoaL7//vsy+y4sLPQ45XZycjJ333036enp+Pv788knn9C6dWv+8Ic/8J///AcR4YknnuD6669nwYIFPP3008VTkK9bt45HH32UBQsWkJuby3333cddd93Fnj17uP7668nMzKSgoIDJkydz8cUX++AbNafi3Ka6jf+s34uqFt+m2qtVQ9/cppp/HA5tL3nyP7gVDm5zkoD7pZ+QBs7JP64PdLveSQZRbZx/Q6Nq5cm/IhqFBfHydT0YdX4cf/piHTe8vYzRPeP405Udvd4LrFsJ4j+Pwt51lbvPJl1h+Avlrnaf7nv+/Pl8+umnLF++HFXlmmuuYeHChaSnp9OsWTO+/vprwJmjqUGDBrz88st8//335fYgkpKSPE65feONN/Loo48ycuRIcnJyKCoq4vPPPycpKYk1a9aQkZFBnz59GDhwIADLly8vnoJ86tSpNGjQgBUrVpCbm0v//v0ZOnQon3/+OcOGDeNPf/oThYWFHDt2rBK/RHMu8guL+Gb9Xv61eBtJuw5TPySAOwYkcMuFrYhrWAWXJ/OynRN+8ck/5eTnzLSSbes1ck74LS+AqBtdPQLXK7SR92OtwQa0i2bebwfyz/9t4c0ftvL9z/t54sqOjDy/udeSf91KED42f/585s+fz/nnnw9AVlYWW7Zs4eKLL+b3v/89f/zjH7nqqqsq/Mu8devWZabcPnr0KGlpaYwcORJwZmYFWLx4MWPHjsXf35/Y2FguueQSVqxYQURERIkpyOfPn8/atWv59NNPASdZbdmyhT59+nD77beTn5/PiBEjiqcfN75z+FgeM5fvYvrS7ew5kkNCdBjPXNuZ0T3jKrc2QmEBZO2FzN3O4PChbSd7AQdSnHXuwmKcE37CwJIJoFGCM8hrzlpIoD+PDDuPq7s78zo9/PEaPk9M468ju9AqKuz0OzhDdStBnOKXflVQVR577DHuuuuuMutWrVrF3Llzeeyxxxg6dChPPvnkaffXsGHDMlNun5ix1dOxy+M+Bbmq8s9//pNhw4aVabdw4UK+/vprbr75Zh555BFuueWW08ZoKl/y/ize+XEbnyU6t6n2bxvFX0d2YVD7xmd+m2pBHhzd45z8M9Nc/5547/qcta/keAA41/8btYa2v3JO/O5JIKRB5f2xxqPzmkTw2d0XMWPZDv7+zc+MmPQjPz56aaU/v1K3EoQPuE/3PWzYMP785z9z4403Eh4eTlpaGoGBgRQUFNCoUSNuuukmwsPDeffdd0tsW94lpoyMjDJTbkdERBAXF1dcHS43N5fCwkIGDhzIm2++ya233srBgwdZuHAh//jHP9i8eXOJfQ4bNozJkydz6aWXEhgYyC+//ELz5s3JyMigefPm3HnnnWRnZ5OYmGgJogqpKgu3ZDBt8TZ++CWdoAA/RvZozrgB8ZzXJMLzRgW5bif8Uif94pP/fsrcAhoUDhHNIaIZtOkIDVzvTyyLbAXBNbuUZm3g5yfcfGE8Qzs3YW3qEa883GgJwsvcp/sePnw4N9xwAxdeeCHg1H7+4IMPSE5O5pFHHsHPz4/AwEAmT54MwIQJExg+fDhNmzb1OEidlpbmccrt999/n7vuuosnn3ySwMBAPvnkE0aOHMnSpUvp3r07IsLf//53mjRpUiZBjB8/nu3bt9OzZ09UlZiYGL744gsWLFjAP/7xj+K62tOnT/fm12Zc8guLmJO0mzcXpvDLvixi6gfzu8vac0PPGKKKDkDmGliT5uHX/27ITi+7w+AGzkm+QXNn/OzESb84ATSHkHISjqmWYiNCuKxTyOkbngW7zdV4lX3/Z+d4XiEfrdjJW4u2sf/wUR6P/JbLG+wglgP4Ze6G4wfLbhQSCQ3iSp3w3U78EU2dW0GNceOz21xF5HLgVZya1G+r6gul1jcEpgFtgBzgdlVdLyIdgI/cmrYGnlTVV7wZrzG+duRYPtOXbuedJds5mJ3HNXHZPFvvVSIOrYeIzhDZAlr0dTvpNzt58g+q/EFKU7d5LUGIiD8wCbgMSAVWiMgcVd3o1uxxIElVR4rIea72Q1T1Z6CH237SgNneirUm6NevH7m5uSWWvf/++3Tt2tVHEZnKtC8zh7cXbeXDZTvJzivk0g4x/Ln5KhJWPAMBwXDd+9DpGl+HaeoYb/Yg+gLJqroVQERmAdcC7gmiE/A8gKpuFpF4EYlV1X1ubYYAKaq6w4uxVnvLli3zdQjGC7ZlZPPmDyl8nphGoSpXd2vKPf0a0WH5E7BkjnOr6Mg3nZ6CMVXMmwmiObDL7XMq0K9UmzXAKGCxiPQFWuFMauieIMYAM88lEFW1Yic+UJvGtyrb+rQjTF6Qwtz1ewj09+P6Pi248+LWtMxcCZ/f4AwwX/YMXPgA+FWTqTFMnePNBOHpjFz6jPEC8KqIJAHrgNVA8fy3IhIEXAM8Vu5BRCYAEwBatmxZZn1ISAgHDhwgKirKkkQVUlUOHDhQ/KCecb6TpVsPMHlBCou2ZFA/OIB7LmnDuP4JxNQT+P6v8OOrzvQSY/8Lzc73dcimjvNmgkgFWrh9jgN2uzdQ1UxgHIA4Z+9trtcJw4HEUpecSlDVqcBUcO5iKr0+Li6O1NRU0tM93PJnvCokJIS4uDhfh+FzRUXKfzft440FKazZdZjo8GD+ePl53HhBSyJCAiEjGT68A/YkQc9b4fLnbcDZVAveTBArgHYikoAzyDwGuMG9gYhEAsdUNQ8YDyx0JY0TxnKOl5cCAwOLp5EwNUhBHhQcr9FP5eYVFPFlUhpTfkghJT2blo1C+evILozuGedM2awKq96Dbx51BqKv/wA6Xu3rsI0p5rUEoaoFInI/MA/nNtdpqrpBRO52rZ8CdASmi0ghzuD1HSe2F5FQnDugys5LYWqf7AOwa5nrtRx2JzqFUHrcAAMedqZwqCGO5RUwa/ku3l60ld1HcujYNILXxp7PFV2aEODvGk84dhD+/SBs+rcNRJtqq9Y/KGeqoaIiOLAFdv7kJINdP8EBV7lEv0Bo2p0j0T3Ze/AwbdO+wE8Lodv1yMDfO9fnq6nDx/J4b8kO3l2yjUPH8ukb34h7BrdhUPuYkuNfW3+A2Xc7A9FD/mwD0canrB6E8a28Y5C26mTvYNcyyDnsrKvXCFr0g/Nvcv5tdj6z1x/gT7PXcyyvkMZcwl0BX3Fj0icErplFYsQQfm5/F9HxXWjbOJxWUWEE+vv25LrnyHH+tWgbHy7fybG8Qn7VsTF3X9KG3vGlpq8uyIPvn4UfX3MNRH8LzXr4JGZjKsIShKl8R9LcksFPTg2OE8XZozs4D3y16Oe8otoWF4I5llfAU19s4JNVqfSNb8RfR3bh0LF8kvcPZFLaDjpue4/BmXPoteJbvlp2AfcUjGSbtKBVVCjtGtenbePw4lfrmDCvTF7mLiU9izd/SGH26jSKFK7p3oy7LmntefK8jC3w2XhnILrXbTDsORuINtWeXWIy56awAPatP5kMdi13VQoDAupBXG9naogW/ZxKYeUUhfll31Hum5FIcnoW9w9uy8Qh7U5er3eXnUHe4tfwX/k2/vnZbG50KR/VG8MPmbHsOHCMwqKT/z03j6xHu9hw2saEl0gekaHnVoVrbephJi9I4ZsNewlye4ahRSMPxXlUIfE9+OYxZyD6mn/aQLSpVk51ickShDkzxw9D6kpXMlgGqasgP9tZV78ZtOx3snfQpCv4B55yd6rKJ6tSefLL9YQHB/LK9T0Y0K4C9XaPHYSf3oBlb0JuJpx3Ffn9f8+2oLYk788qfm3Zn8XW9CxyC07WM4gODzqZMGLCaevqfcRGBJf7rIyqsiTFeYZhcXIG9UMCuPXCeG7rH090eHD5Mc55ADZ/BQmXwMgpNhBtqh1LEObsqDqVw9x7B/s3AQriD026QIsLTvYQIlucdpfusnMLeOKL9cxencZFbaJ4ZUwPGtc/wwfrjh+Cn6bAT5Mh9wi0vxwG/gHiehU3KSxS0g4dJzn9qJM09mWRnO4kkKM5xc9lUj84gDZuPY0TPY/NezOZvCCFNalHiKkfzPgBCdzQryX1Q06R/EoMRD8JF95vA9GmWrIEYSquMB9WvA3bFzs9hBM1BYIbnEwELfpC817nVDRm4+5M7v8wke0Hsvntr9pz3+C2+J9pNTR3OUdg2VRY+rozAN72V3DJH51Yy6GqpB/NLe5pFPc80rNIP1pyYsRWUaHcNbANo3o2d55hKE9BHvzvL7Dkn874yui3bSDaVGuWIEzFLZ0E8x53ykee6B20vMAZXK6EX8CqyofLd/L0vzcSWS+QV8ecz4VtoiohcJecTCfBLfmnUzOh9SAnUbS66Ix2c+RYvquXcZQG9YK4rFPs6RNYxhb47A7YswZ6jYNhf7WBaFPtWYIwFaMKb1zgFJUZ/22l7/5oTj6Pfr6Or9fuYWD7GF6+rnv51+/PVW4WrJwGS15zekHxF8Mlf3D+rew5ucoMRL8OHa+q3GMY4yX2HISpmNQVkL7ZudOmkq1PO8J9HyaSeug4f7i8A3cPbIPfuVxSOp3gcOj/IPQZD6vehR9fgfeuhpYXOYmi9aDKSRRlBqLfdIr3GFMLWIIwJyW+B4Fh0Hlkpe1SVXlvyXaem7uZqPAgZk24gD6lHyDzpqBQuPBe6D0OEt+Hxf8P3h8BcX2dS09th5x9oti6wDUQnQGX/cUGok2tYwnCOHKPwvrZ0GVUpdUtPnIsnz98toZ5G/Yx5LzGvPib7jQMO7dnEM5aYD3oNwF63QqrP3ASxYzR0KynkyjaD6t4oig9ED12lg1Em1rJEoRxrP/ceZ6h562VsrvVOw/xwMzV7D2SwxNXduSOAQnVox5HQDD0uQPOvxnWzIRFL8LM66FJNydRdLji1L2A9F/g8/FuA9HPOb0UY2ohSxDGkTgdYs5znnw+B6rKvxZv44X/bCY2IoRP7r6Q81s2rKQgK1FAkNOb6HEDrP0IFr4IH90IsV1g4CPQ8ZqSiULVGcv45jGnN3L9DBuINrWeJQgD+zZC2krn1/A5/Mo/lJ3H7z9Zw3eb9zO0Uyz/+HV3GoSe+klqn/MPdCYK7DYG1n8KC/8Bn9wKMR1h4O+d8ZicIycHolsPghFTbCDa1AmWIAysft+ZZrvbmLPexcrtB3lw5moysvL4v6s7cetF8dXjklJF+QdA9zHQ9TewYTb88HfnmYYf/uaMz2RnwNBn4YL7bCDa1BmWIOq6glznWvx5V0LYmT+wVlSkTFmYwkvzf6F5ZD0+u+ciusbV3Cpw+PlD119D51Gw6UtY9JJzSemGj6Bpd19HZ0yVsgRR123+2pnPqOctZ7zpgaxcHv54DT/8ks6VXZvy/OiuTo3l2sDPz7m8VIm3/BpT01iCqOsSp0ODFtB68Blt9tPWA0yctZpDx/J5dkQXbuzXsmZdUjLGnJYliLrs0A7Y+j0MeqzC19ULi5RJ3yfzyre/EB8Vxju39aVTMw8FcowxNZ5XR9tE5HIR+VlEkkXkUQ/rG4rIbBFZKyLLRaSL27pIEflURDaLyCYRudCbsdZJSTMAgR43Vqj5/qM53DJtGS//9xeu6d6MOQ8MsORgTC3mtR6EiPgDk4DLgFRghYjMUdWNbs0eB5JUdaSInOdqP8S17lXgG1X9tYgEAfY0UmUqKnSeKG5zaYXqOCzeksFvP0oiKzefv43uynW9W9glJWNqOW/2IPoCyaq6VVXzgFnAtaXadAK+A1DVzUC8iMSKSAQwEPiXa12eqh72Yqx1T8r/IDPttIPTBYVFvDT/Z26etozI0EC+vG8A1/ex8QZj6gJvjkE0B3a5fU4F+pVqswYYBSwWkb5AKyAOKATSgXdEpDuwCpioqtlejLduSZwOoVHO1BLl2HskhwdnrWb5toP8ulccz1zbmdAgG7Yypq7wZg/C00/M0sUnXgAaikgS8ACwGijASVw9gcmqej6QDZQZwwAQkQkislJEVqanp1dW7LVbVjr8PBe6j3WmnPBgwc/7ueK1RaxLPcJLv+nOi7/pbsnBmDrGm//HpwLuF7fjgN3uDVQ1ExgHIM41i22uVyiQqqrLXE0/pZwEoapTgangFAyqxPhrrzUzoajAmbCulMIi5R/zfmbKDymc16Q+r9/Qk7aNz760qDGm5vJmD2IF0E5EElyDzGOAOe4NXHcqnfgJOx5YqKqZqroX2CUiHVzrhgDug9vmbKk6U2u06AeNzyuz+oOfdjDlhxTG9m3BF/f1t+RgTB3mtR6EqhaIyP3APMAfmKaqG0Tkbtf6KUBHYLqIFOIkgDvcdvEAMMOVQLbi6mmYc7RrGWT84pTFLCW/sIipC7fSu1VDnhvZ1QaijanjvHpRWVXnAnNLLZvi9n4p0K6cbZOAc5t72pSVOB2Cwj1OIfFl0m7SDh/nLyM6W3Iwxnj3QTlTzeRkOjOVdhnt1Gx2U1SkTF6QzHlN6jO4Q2MfBWiMqU4sQdQl6z+D/GMen32Yv3EvKenZ3Du4rfUejDGAJYi6JXE6NO4EzXuVWKyqvLEghVZRoVzRpYmPgjPGVDeWIOqKvethd6LTeyjVQ1icnMHa1CPcfUkbAvztPwljjMPOBnXF6vfBPwi6XV9m1RvfpxAbEcyons19EJgxprqyBFEX5OfAmllw3lUQ2qjEqsSdh1i69QDjB7QmOMDfRwEaY6ojSxB1weavIOewx8HpN75PoUG9QG7o17Lq4zLGVGuWIOqCxOkQ2RISLimx+Oe9R/l20z5uuyiesGCbZ8kYU5IliNru4DbY9oMz71KpqnGTFyQTGuTPbRfF+yY2Y0y1ZgmitkuaAeIHPW4osXjXwWP8e+0ebujbkoZhnmd0NcbUbZYgarOiQlg9A9oMgQZxJVa9uTAFP4HxF7f2UXDGmOrOEkRtlvwdHN1dZnB6/9EcPl6Zyq97xdGkQYiPgjPGVHeWIGqzxPcgLAbaX15i8b8Wb6OgsIi7BrbxUWDGmJqgQglCRAaIyInCPjEikuDdsMw5y9oPv3wD3ceUqBp35Fg+HyzdwRVdmxIfHebDAI0x1d1pE4SIPAX8EXjMtSgQ+MCbQZlKUFw1ruTlpelLt5OdV8i9g9r6KDBjTE1RkR7ESOAanLrQqOpuoL43gzLnSNV59qHlhRDTvnjxsbwC3lmyncEdYujULMKHARpjaoKKJIg8VVVAAUTErktUdzuXwoHkMjWnZy3fxcHsPO4bbL0HY8zpVSRBfCwibwKRInIn8C3wlnfDMuck8X0Iqg+dRxQvyiso4q1FW+kb34je8Y3K39YYY1xOOb+COJVjPgLOAzKBDsCTqvrfKojNnI2cI07VuO7XQ9DJzt4XSWnsOZLDc6O6+jA4Y0xNcsoEoaoqIl+oai/gjJOCiFwOvAr4A2+r6gul1jcEpgFtgBzgdlVd71q3HTgKFAIFqmr1qSti3adQcLzEsw+FRcqUBSl0ahrBoPYxPgzOGFOTVOQS008i0udMdywi/sAkYDjQCRgrIp1KNXscSFLVbsAtOMnE3WBV7WHJ4Qysfh9iu0CznsWL5m3Yy9aMbO4d3MbKiRpjKqwiCWIwTpJIEZG1IrJORNZWYLu+QLKqblXVPGAWcG2pNp2A7wBUdTMQLyKxZxC/cbdnLexe7QxOuxKBqjLp+2QSosMY3qWpjwM0xtQkFZnjefhZ7rs5sMvtcyrQr1SbNcAoYLGI9AVaAXHAPpy7puaLiAJvqurUs4yj7lj9PvgHQ7frihct3JLBht2Z/G10V/z9rPdgjKm40yYIVd0hIt2Bi12LFqnqmgrs29PZSEt9fgF4VUSSgHXAaqDAta6/qu4WkcbAf0Vks6ouLHMQkQnABICWLetw0Zv847D2I+h4dYmqcW98n0yTiBBGnh93io2NMaasijxJPRGYATR2vT4QkQcqsO9UoIXb5zhgt3sDVc1U1XGq2gNnDCIG2OZat9v1735gNs4lqzJUdaqq9lbV3jExdXgAdtNXzh1MPU8++7Bqx0GWbTvInQNbExRg024ZY85MRc4adwD9VPVJVX0SuAC4swLbrQDaiUiCiAQBY4A57g1EJNK1DmA8sFBVM0UkTETqu9qEAUOB9RX7k+qoxPcgshXEDyxe9Mb3KTQMDWRs3xan2NAYYzyryBiE4NxqekIhni8flaCqBSJyPzAP5zbXaaq6QUTudq2fAnQEpotIIbARJxkBxAKzXXfcBAAfquo3FfuT6qCDW2H7Irj0ieKqcZv2ZPLd5v089Kv2hAZZOVFjzJmryJnjHWCZiMx2fR4B/KsiO1fVucDcUsumuL1fCrTzsN1WoHtFjmGA1R+4qsbdWLxo8oIUwoL8ufWiVj4MzBhTk1VkkPplEVkADMDpOYxT1dXeDsxUUGGBUzWu7WUQ0QyAHQey+WrtbsZf3JrIUCsnaow5O6dNECJyAbBBVRNdn+uLSD9VXeb16MzpJX8LWXuh50vFi6b8sJUAPz/GD7CyHcaYs1eRQerJQJbb52zXMlMdJE6HsMbQfhgA+zJz+GxVKr/uHUfjCCsnaow5exVJEOKa7hsAVS2iYmMXxtuO7nWqxvUYC/6BgKucaFERdw1s7ePgjDE1XUUSxFYReVBEAl2vicBWbwdmKmDNTNDC4qpxh4/l8cFPO7i6ezNaRVnZDmPMualIgrgbuAhIc7364Xpy2fhQcdW4iyDaKQD03pIdHMsr5J5BbXwcnDGmNqjIXUz7cR5yM9XJjh+d5x8G/gGA7NwC3lmyjV91bMx5TaycqDHm3JXbgxCRO0Wkneu9iMg0ETnimtG1Z3nbmSqS+D4ER0AnZ4Lcmct3cvhYPvcMsnKixpjKcapLTBOB7a73Y3EeXGsNPEzZug2mKh0/DBu/gK6/hqBQcgsKeXvRNvolNKJXq4a+js4YU0ucKkEUqGq+6/1VwHRVPaCq3wI2AupL6z6BgpziqnGzE9PYm5nDfYOt92CMqTynShBFItJUREKAIcC3buvqeTcsc0qr34cmXaFpDwqLlDcXbqVL8wgubhft68iMMbXIqRLEk8BKnMtMc1R1A4CIXILd5uo7u5Ngzxrn1lYR/rN+D9sysrl3UFsrJ2qMqVTl3sWkql+JSCugvqoeclu1Erje65EZz4qrxv3GVU40hdYxYQzr3MTXkRljaplTPgehqgWlkgOqmq2qWeVtY7wo/zis/cS5c6leQxb8ks6mPZncfUkbKydqjKl0VmasJtk4B3JPVo174/tkmjUIYUSP5j4OzBhTG1mCqEkSp0PDBGg1gBXbD7Ji+yErJ2qM8ZpTPSg3TER+7WH5jSJymXfDMmUcSIEdi53eg58fb3yfTKOwIMb0aenryIwxtdSpfno+DfzgYfl3wDPeCceUa/X7TtW47jewYfcRvv85ndv7x1MvyN/XkRljaqlTJYhQVU0vvVBV92IPylWtwgJI+hDaDYOIpkxekEJ4cAA3Xxjv68iMMbXYqRJEiIiUuQ1WRAKp4INyInK5iPwsIski8qiH9Q1FZLZrfqflItKl1Hp/EVktIl9V5Hi11pb5kLUPet7Ctoxs5q7bw00XtKJBvUBfR2aMqcVOlSA+B94SkeLeguv9FNe6UxIRf2ASMBzoBIwVkU6lmj0OJKlqN+AWys7xNBHYdLpj1XqJ0yE8FtoN5c0fUgjw9+P2AfG+jsoYU8udKkE8AewDdojIKhFJxHmqOt217nT6AsmqulVV84BZwLWl2nTCGdNAVTcD8SISCyAiccCVwNsV/3Nqocw9sGUe9LiBvVkFfJaYynW942hc38qJGmO861RPUhcAj4rI08CJWeCSVfV4BffdHNjl9jkVp9iQuzXAKGCxiPQFWgFxOInpFeAPQP0KHq92WvMhaBGcfzNvLdpKkcJdA60gkDHG+8pNECIyqtQiBSJFJElVj1Zg354e7dVSn18AXhWRJGAdsBooEJGrgP2qukpEBp3yICITcFW4a9mylt3yWVTk1H1oNYBDIS2Yufx/XNO9GS0ahfo6MmNMHXCqinJXe1jWCOgmIneo6v9Os+9UoIXb5zhgt3sDVc0ExoFTlAjY5nqNAa4RkSuAECBCRD5Q1ZtKH0RVpwJTAXr37l06AdVsOxbDoW0w6DHeXbLdyokaY6rUqS4xjfO03DWB38eUvVxU2gqgnYgk4NSyHgPcUGpfkcAx1xjFeGChK2k85nrh6kH83lNyqPUS34fgBmS1uYJ3v1jKZZ1iaR9bt6+4GWOqzmlrUpemqjtct7qerl2BiNwPzAP8gWmqukFE7natnwJ0BKaLSCGwEbjjTOOptY4fgo1fQs+bmZmYzpHj+dxrvQdjTBU64wQhIh2A3Iq0VdW5wNxSy6a4vV8KtDvNPhYAC840zhpv7SdQmEte95t4672tXNQmivNbWjlRY0zVOdUg9b8pO6jcCGgK3OzNoOo8VefZhybd+DQtiv1Hd/PydT18HZUxpo45VQ/ixVKfFTgAbHGNGRhv2ZME+9ZROPwfvLkwhW5xDejfNsrXURlj6phTDVJ7mqgPEekvIjeo6n3eC6uOS5wOASF8Ixez48BWptzUy8qJGmOqXIXGIESkB84dSNfh3IZ62qk2zFnKOwbrPkU7XcM/l6TTtnE4QzvF+joqY0wddKoxiPY4t6aOxbm09BEgqjq4imKrmzZ+CbmZrI6+hs3Lj/LSb7rjZ+VEjTE+cKoexGZgEXC1qiYDiMhDVRJVXZY4HW3Umr+ub0TzyFyu6dHM1xEZY+qoU03WNxrYC3wvIm+JyBA8T59hKktGMuxcwq5Wo1m18zATBrYm0N/KiRpjfKPcs4+qzlbV64HzcJ5DeAiIFZHJIjK0iuKrW1ZPB/HnH/t7ER0exPV9Wpx+G2OM8ZLT/jxV1WxVnaGqV+HMp5QElCn+Y85RYT4kzSSz5RD+nVLEuP4JhARaOVFjjO+c0fULVT2oqm+q6qXeCqjOWvcpZO/ng/zB1A8O4OYLW/k6ImNMHWcXuKuDoiJY/DK5UR35x7aW3HxhKyJCrJyoMca3LEFUB5v/DRm/8Gm96wjy9+f2AQm+jsgYYyxB+JwqLHqJvIh4ntnWnjF9WhAdHuzrqIwxxhKEzyV/B3vW8HLOlTQIDeG+S9uefhtjjKkCZzzdt6lcuuhFDvpH8372hXxwVy8a1w/xdUjGGANYD8K3dixBdi7lnzlX8NTIHlbvwRhTrVgPwof2fvVXAjSCwD63cV1veyjOGFO9WA/CR35ZvYgm6Yv5NmI0f7j6fF+HY4wxZViC8IH9R3NInfMsRwll2LgnbL4lY0y15NUzk4hcLiI/i0iyiJSZnkNEGorIbBFZKyLLRaSLa3mI6/MaEdkgIk97M86qlFdQxF/f/YJBRcvIOf8OGjaK9nVIxhjjkdcShIj4A5OA4UAnYKyIdCrV7HEgSVW7AbcAr7qW5wKXqmp3oAdwuYhc4K1Yq9JTczYwcP8HaEAwMb/6ra/DMcaYcnmzB9EXSFbVra4a1rOAa0u16QR8B6Cqm4F4EYlVR5arTaDrpV6MtUp88NMOFq1YxUj/Jfj3Hgdh1nswxlRf3kwQzYFdbp9TXcvcrQFGAYhIX6AVzoyxiIi/iCQB+4H/quoyL8bqdcu3HeT/5mzg6ahvET8/uOgBX4dkjDGn5M0E4am4UOlewAtAQ1cieABYDRQAqGqhqvbASRh9T4xPlDmIyAQRWSkiK9PT0ysr9kq1+/Bx7p2xih6ROVx6fD7SYyw0KJ0rjTGmevFmgkgF3G/ujwN2uzdQ1UxVHedKBLcAMcC2Um0O4xQsutzTQVR1qqr2VtXeMTExlRZ8ZcnJL2TC+yvJyS/irfbLkKJ86P9bX4dljDGn5c0EsQJoJyIJIhIEjAHmuDcQkUjXOoDxwEJVzRSRGBGJdLWpB/wKp0Z2jaKqPPrZWjbszuT1EfE03PgBdB4FUW18HZoxxpyW156kVtUCEbkfmAf4A9NUdYOI3O1aPwXoCEwXkUJgI3CHa/OmwHuuO6H8gI9V9Stvxeotby/axhdJu/ndZe0ZdOQzyMuCix/2dVjGGFMhXp1qQ1XnAnNLLZvi9n4p0M7DdmuBGv148Q+/pPP8fzYxvEsT7u8fC69Mhg5XQGxnX4dmjDEVYo/wesH2jGwe+DCR9rH1efE33ZFV70LOYbj4d74OzRhjKswSRCXLyi3gzukr8fMT3rqlN2F+BbD0dUi4BOJ6+zo8Y4ypMEsQlaioSHn4oyS2ZmQz6YaetGgUCkkfQNY+6z0YY2ocSxCV6LX/bWH+xn08fkVH+reNhsJ8+PFViOsDCQN9HZ4xxpwRSxCVZN6Gvbzy7RZG94zj9v7xzsJ1n8LhnU7vQTw9N2iMMdWXJYhK8Mu+ozz8URLd4xrw15FdEBEoKoLFL0NsF2jv8Rk/Y4yp1ixBnKMjx/KZMH0l9YICmHJzL0IC/Z0Vm7+CjF9gwEPWezDG1EiWIM5BYZFy/8xE0g4f582be9K0QT1nhSosehEatYbOI30bpDHGnCVLEOfg799sZtGWDJ65tgu9WjU6uSLlO9izxuk9+Pn7LkBjjDkHliDO0pdJaby5cCs3XdCSsX1blly58CWIaA7dxvgmOGOMqQSWIM7C+rQj/OHTtfSNb8STV5WaOmPHEti5BC56EAKCPO/AGGNqAEsQZygjK5cJ01cSFRbEGzf1JCig1Fe46CUIjYaet/gmQGOMqSSWIM5AfmER985I5EB2Hm/e3Jvo8OCSDXYnQfK3cOG9EBTqkxiNMaayeHU219rmmX9vZPm2g7w6pgdd4xqUbbD4ZQhuAH3GV31wxhhTyawHUUGzlu/k/Z92MGFga67t4aFcaPrPsHEO9L0TQjwkD2OMqWEsQVTAqh0H+fOX67m4XTR/vPw8z40WvwIBIXDBPVUamzHGeIsliNPYeySHuz9IpFlkPV4f2xN/Pw9PRR/aAWs/gl63QVh0lcdojDHeYGMQp5CTX8hd76/kWG4BM8b3o0FooOeGS14D8YOLHqjaAI0xxossQZRDVfnT7PWsST3Cmzf3on1sfc8Nj+6FxPehx1ho4GFswhhjaiivXmISkctF5GcRSRaRRz2sbygis0VkrYgsF5EuruUtROR7EdkkIhtEZKI34/TknR+381liKhOHtGNY5yblN1w6CYryof9vqyw2Y4ypCl5LECLiD0wChgOdgLEi0qlUs8eBJFXtBtwCvOpaXgD8TlU7AhcA93nY1mt+TM7gr3M3MbRTLBOHtCu/4bGDsHIadB4FUW2qKjxjjKkS3uxB9AWSVXWrquYBs4BrS7XpBHwHoKqbgXgRiVXVPaqa6Fp+FNgEVMn1m10Hj3Hfh4m0jg7j5et74OdpUPqE5VMhLwsufrgqQjPGmCrlzQTRHNjl9jmVsif5NcAoABHpC7QC4twbiEg8cD6wzFuBnnAsr4A7p6+kqEh565behAefYogm9yj8NBk6XAGxnctvZ4wxNZQ3E4Snn95a6vMLQEMRSQIeAFbjXF5ydiASDnwG/FZVMz0eRGSCiKwUkZXp6elnHayq8sgna/ll31H+eUNP4qPDTr3Byncg57BTTtQYY2ohb97FlAq0cPscB+x2b+A66Y8DEBEBtrleiEggTnKYoaqfl3cQVZ0KTAXo3bt36QRUYW8sSOHrdXt4bPh5XNI+5tSN83Ng6euQcAnE9T7bQxpjTLXmzR7ECqCdiCSISBAwBpjj3kBEIl3rAMYDC1U105Us/gVsUtWXvRgjAP/bvI8X5//MNd2bMWFg69NvkDQDsvZZ78EYU6t5rQehqgUicj8wD/AHpqnqBhG527V+CtARmC4ihcBG4A7X5v2Bm4F1rstPAI+r6tzKjvPwsTwmzkqiU9MI/ja6G3K6+tGF+fDjKxDXBxIGVnY4xhhTbXj1QTnXCX1uqWVT3N4vBcrcR6qqi/E8hlHpIkOD+PvobnRrEUm9oAqUB13/GRzeCcP/DqdLJsYYU4PZk9TA8K5NK9awqAgWvQyxXaD95d4NyhhjfMwm6zsTm7+CjJ9hwEPWezDG1HqWICpKFRa9CI1aQ+eRvo7GGGO8zhJERaV8B3vWOL0HvwqMVRhjTA1nCaKiFr0MEc2h2xhfR2KMMVXCEkRF7FgKO36Eix6EgKDTtzfGmFrAEkRFLHoJQqOh5y2+jsQYY6qMJYjT2Z0Eyf+FC++FoFBfR2OMMVXGEsTpLH4ZghtAn/G+jsQYY6qUJYhTSf8ZNs6BvndCSANfR2OMMVXKEsSpLH4FAkLggnt8HYkxxlQ5SxDlObQD1n4EvW6DsGhfR2OMMVXOEkR5lrwG4gcXPeDrSIwxxicsQXhydB8kvg89xkKDKimFbYwx1Y4lCE+Wvg5F+dD/t76OxBhjfMYSRGnHDsLKadB5FES18XU0xhjjM5YgSls+FfKy4OKHfR2JMcb4lCUId7lH4afJ0OEKiO3s62iMMcanLEG4W/Uu5ByGi3/n60iMMcbnvJogRORyEflZRJJF5FEP6xuKyGwRWSsiy0Wki9u6aSKyX0TWezPGYvk5sOSfkHAJxPWukkMaY0x15rUEISL+wCRgONAJGCsinUo1exxIUtVuwC3Aq27r3gWqrvBz0gzI2me9B2OMcfFmD6IvkKyqW1U1D5gFXFuqTSfgOwBV3QzEi0is6/NC4KAX4zupMB9+fAXi+kDCwCo5pDHGVHfeTBDNgV1un1Ndy9ytAUYBiEhfoBUQdyYHEZEJIrJSRFamp6efXaTrP4PDO53eg8jZ7cMYY2oZbyYIT2daLfX5BaChiCQBDwCrgYIzOYiqTlXV3qraOyYm5syjLCpyyonGdoH2VXdFyxhjqrsAL+47FWjh9jkO2O3eQFUzgXEAIiLANter6uRnQ4u+0HaI9R6MMcaNNxPECqCdiCQAacAY4Ab3BiISCRxzjVGMBxa6kkbVCa4P175epYc0xpiawGuXmFS1ALgfmAdsAj5W1Q0icreI3O1q1hHYICKbce52mnhiexGZCSwFOohIqojc4a1YjTHGlCWqpYcFaq7evXvrypUrfR2GMcbUGCKySlU9PvxlT1IbY4zxyBKEMcYYjyxBGGOM8cgShDHGGI8sQRhjjPHIEoQxxhiPatVtriKSDuw4y82jgYxKDKcms++iJPs+SrLv46Ta8F20UlWP8xTVqgRxLkRkZXn3Atc19l2UZN9HSfZ9nFTbvwu7xGSMMcYjSxDGGGM8sgRx0lRfB1CN2HdRkn0fJdn3cVKt/i5sDMIYY4xH1oMwxhjjkSUIY4wxHtX5BCEil4vIzyKSLCKP+joeXxKRFiLyvYhsEpENIjLx9FvVbiLiLyKrReQrX8fiayISKSKfishm138jF/o6Jl8SkYdc/5+sF5GZIhLi65gqW51OECLiD0zCKVbUCRgrIp18G5VPFQC/U9WOwAXAfXX8+wCniNUmXwdRTbwKfKOq5wHdqcPfi4g0Bx4EeqtqF8Afp2pmrVKnEwTQF0hW1a2usqezgGt9HJPPqOoeVU10vT+KcwJo7tuofEdE4oArgbd9HYuviUgEMBD4F4Cq5qnqYZ8G5XsBQD0RCQBCgd0+jqfS1fUE0RzY5fY5lTp8QnQnIvHA+cAyH4fiS68AfwCKfBxHddAaSAfecV1ye1tEwnwdlK+oahrwIrAT2AMcUdX5vo2q8tX1BCEeltX5+35FJBz4DPitqmb6Oh5fEJGrgP2qusrXsVQTAUBPYLKqng9kA3V2zE5EGuJcbUgAmgFhInKTb6OqfHU9QaQCLdw+x1ELu4lnQkQCcZLDDFX93Nfx+FB/4BoR2Y5z6fFSEfnAtyH5VCqQqqonepSf4iSMuupXwDZVTVfVfOBz4CIfx1Tp6nqCWAG0E5EEEQnCGWSa4+OYfEZEBOca8yZVfdnX8fiSqj6mqnGqGo/z38X/VLXW/UKsKFXdC+wSkQ6uRUOAjT4Mydd2AheISKjr/5sh1MJB+wBfB+BLqlogIvcD83DuQpimqht8HJYv9QduBtaJSJJr2eOqOtd3IZlq5AFghuvH1FZgnI/j8RlVXSYinwKJOHf/raYWTrthU20YY4zxqK5fYjLGGFMOSxDGGGM8sgRhjDHGI0sQxhhjPLIEYYwxxiNLEMachogUikiS26vSniAWkXgRWV9Z+zOmMtXp5yCMqaDjqtrD10EYU9WsB2HMWRKR7SLyNxFZ7nq1dS1vJSLficha178tXctjRWS2iKxxvU5MzeAvIm+5agvMF5F6rvYPishG135m+ejPNHWYJQhjTq9eqUtM17uty1TVvsDrOLO/4no/XVW7ATOA11zLXwN+UNXuOPMYnXhqvx0wSVU7A4eB0a7ljwLnu/Zzt3f+NGPKZ09SG3MaIpKlquEelm8HLlXVra5JDveqapSIZABNVTXftXyPqkaLSDoQp6q5bvuIB/6rqu1cn/8IBKrqsyLyDZAFfAF8oapZXv5TjSnBehDGnBst5315bTzJdXtfyMmxwStxKh72Ala5CtMYU2UsQRhzbq53+3ep6/0STpafvBFY7Hr/HXAPFNe6jihvpyLiB7RQ1e9xihZFAmV6McZ4k/0iMeb06rnNbgtOXeYTt7oGi8gynB9bY13LHgSmicgjOFXYTsx6OhGYKiJ34PQU7sGpRuaJP/CBiDTAKWz1/6zEp6lqNgZhzFlyjUH0VtUMX8dijDfYJSZjjDEeWQ/CGGOMR9aDMMYY45ElCGOMMR5ZgjDGGOORJQhjjDEeWYIwxhjj0f8HFwEh3lwoi0IAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# plot training stats\n", "train.plot_train_stats()" ] }, { "cell_type": "code", "execution_count": 31, "id": "75e22bb5", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAFNCAYAAAANRGjoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABAxElEQVR4nO3dd3wU1frH8c+TTkJCCqEmVOldQxEFARGpIl4LimIDxF7u9er1d1X02q7dawfEioCKIipSRKqgEgTpJfTQEyAJ6eX8/pglhpDAEnYz2d3n/Xrlld2Zye53Qphn55yZc8QYg1JKKd/lZ3cApZRS9tJCoJRSPk4LgVJK+TgtBEop5eO0ECillI/TQqCUUj5OC4HyGCIyQkTmOrHdeyLyeGVkqgwislNE+joejxORz+zOpLyLFgLlEo6DVbaIHBeRgyLyoYhUd+V7GGMmG2P6ObHdWGPMf1z53ieIiBGRTMd+7hWRV0XE3x3vVREiEiEir4vIbkfGJMfzmnZnU1WXFgLlSkOMMdWB84HOwL9LbyAiAZWeyvU6OPbzEuA64Dab8wAgIkHAfKAN0B+IALoDqUCXCryeN/xbKSdoIVAuZ4zZC/wItIXiT9F3i8hWYKtj2WARWS0ix0RkmYi0P/HzIhIvIl+LyGERSRWRtxzLbxGRpY7HIiKvicghEUkTkTUicuL9PhKRZ0q83mjHJ+MjIjJTROqVWGdEZKyIbBWRoyLytoiIk/uZBPwCdCzxehXZr6Yi8rNjWYqITBaRyLP8tQOMBBoAw4wxG4wxRcaYQ8aY/xhjZpXY3/NKZCr+XYlILxFJFpFHROQA8KGIbBSRwSW2D3BkPN/xvJtjP4+JyJ8i0qsCuZXNtBAolxOReGAgsKrE4iuBrkBrx0FkEnAHEAO8D8wUkWBHM8v3wC6gEVAfmFrG2/QDegLNgUisT+apZWTpAzwPXAvUdbxu6dcbjHUG08Gx3eVO7mdLoAeQ5Hhe0f0SR8Z6QCsgHhjnTIZS+gKzjTHHK/CzJ9QBooGGwBhgCnB9ifWXAynGmD9EpD7wA/CM42f+AUwXkdhzeH9lAy0EypVmiMgxYCmwCHiuxLrnjTFHjDHZwGjgfWPMb8aYQmPMx0Au0A2rCaMe8LAxJtMYk2OMWVrGe+UD4UBLQIwxG40x+8vYbgQwyRjzhzEmF/gXcKGINCqxzQvGmGPGmN3AAkp8wi/HHyKSCWwEFgLvOJZXaL+MMUnGmHnGmFxjzGHgVaxmp7MVA5T1OzgbRcCTjizZwOfAFSIS6lh/g2MZwI3ALGPMLMfZxzwgEetDgPIgWgiUK11pjIk0xjQ0xtzlOJCcsKfE44bA3x3NCcccxSMe60AZD+wyxhSc7o2MMT8DbwFvAwdFZLyIRJSxaT2sT+Enfu441plD/RLbHCjxOAuoDiAi6x0drsdFpEeJbc53bHMd1llO2Lnsl4jUEpGpjs7ndOAzoCKdu6lYZz3n4rAxJufEE0fz10ZgiKMYXMFfhaAhcE2p/b3YBRlUJdNCoCpLyWFu9wDPOorGia9QY8wUx7oGznRUGmP+Z4y5AKtztDnwcBmb7cM6YAEgImFYn5z3OvH6bYwx1R1fS0qtM8aYL4DlwBPnuF/PY/1+2htjIrA+aTvVT1HKT8Dljn0sTxYQWuJ5nVLryxqO+ETz0FBgg6M4gLVPn5ba3zBjzAsVyK5spIVA2WECMFZEujo6fcNEZJCIhAO/YzVvvOBYHiIiF5V+ARHp7Pj5QCATyAEKy3ivz4FbRaSjiARjNVf9ZozZ6aJ9eQEYIyJ1zmG/woHjwDFHu3tZBc0Zn2IdnKeLSEsR8RORGBF5TERONNesBm4QEX8R6Y9zTVBTsfpk7uSvswGwzlyGiMjljtcLcXQ4x1Uwv7KJFgJV6YwxiVjt6W8BR7E6W29xrCsEhgDnAbuBZKwmmNIisA68R7GaflKBl8t4r/nA48B0rANxU2C4C/dlLVZ/yMPnsF9PYTU3pWF1vn5dwSy5WB3Gm4B5QDpWAaoJ/ObY7H5HjmNY/ScznHjd/VhnPt2BaSWW78E6S3gMOIxVhB5GjyseR3RiGqWU8m1auZVSysdpIVBKKR+nhUAppXycFgKllPJxWgiUUsrHedzogjVr1jSNGjWyO4ZSSnmUlStXphhjyhwHyuMKQaNGjUhMTLQ7hlJKeRQR2VXeOm0aUkopH6eFQCmlfJwWAqWU8nFaCJRSysdpIVBKKR+nhUAppXycFgKllPJxbisEIjJJRA6JyLpy1ouI/E9EkkRkjWPib6WUUpXMnWcEHwH9T7N+ANDM8TUGeNeNWZRSSpXDbXcWG2MWi0ij02wyFPjEWDPj/CoikSJS1zEbksvtTs3i1827KfILwvh53A3VVYq/CP3a1CYyNMjuKEopF7DziFgfa2q7E5Idy04pBCIyBuusgQYNGlTozdYmHyPqxzsJJp978u8lneoVeh1lOZrVkjsuaWp3DKWUC9hZCKSMZWXOm2mMGQ+MB0hISKjQ3JqXtq5N7uUjiPj5UZZf8CsZvZ+vyMv4vPzCInq8uID8wiK7oyilXMTOQpAMxJd4Hgfsc9ebhQT6E3LxKGjUibDY5oQFh0BBHgRo88bZ0AKglPex8/LRmcBIx9VD3YA0d/UPnCTuAggOh7ws+KAvLHoRivTgppTyXW47IxCRKUAvoKaIJANPAoEAxpj3gFnAQCAJyAJudVeWcgJCbEtY8CwcWAtXvgvB2m9QEfmFRaRn5wMQUz3Y5jRKqbMl1kU7niMhIcG4bD4CY2D52zDvcYhtBdd/DlGNXPPaXiq/sIhm//cjUaGBBAX4kZ5dQHZ+YfH6Bf/oReOaYTYmVEqVRURWGmMSylrn29dRikD3e6B2a/jyVvj2Hrjle7tTVWkBfsKtFzXiQFoOESGBRFQLICIkkAPpOUz+bTdHMvOqTCEwxiBS1jUJSqmSfLsQnNC0D4z+GcTRZVKYD34BVqFQJxERnhzS5pTli7YcZvJvu13+fvmFRaRl5xd/pZf6bj0uOHmbHOt7Zm4Bjw1sxageTVyeSylvooXghBjHNfHGwPRRVn/BoFchQNu8XSmvoIiU47kcycwjNTOPI5m5pB53PD7+17IT6zNyCk77eiGBftSoFkiNaoFEhARSt0YILeuEE1EtkGkr9rA9JbOS9kwpz6WFoDRjoGZzWPwiHN4M130G4XXsTuUxthzMIOV4LofScziYnsuhDOv7wfQcDmVYB/iyBPgJ0WFBxFQPJiYsiLioUKLDgogOCyIy1HGgdxzsiw/81QIIDvAvN8v3a8q/CK2wyHAsK4+jWfkcy8rjSGYex7LyOZqVR50aIQztWP+cfxdKeQotBKX5+UGf/4M6beGbO2F8L7husnXZqSpXoL/VjPavr9cWL/P3E2KrB1M7Ipj46FASGkVROzyEmuHBRIcFUbN6ENFh1uOIkAC3tOcn7jzC37/4k6NZedZXpnXwT8/Jp7zrJAL8RAuB8ilaCMrTeihEN4Wp18O0G+H+1dpMdBqdG0Xz1g2dqBboT+2IEGpFBBMTFoy/n339LM1qVWf1nmMczykgKiyIqFDrTCMqNJCo0CDru2N5VGgQUWGBfLJ8FxOWbLcts1J20EJwOnXawphFcHSHVQSMgaJC8NdfW2mB/n4Mbl/P7hgn+Xx017M+ywgJ8MMYeHXeFlKO55J6PJfuTWtyc/dG7gmpVBWgR7QzCY22vgAWvww7l8A1H/21TFVZFWlqio0IAeDNn7cSHRpEVl4hu1KztBAor6aF4GxE1IPdy2FCbxg+xbr/QHmVm7o1ZEj7uoSHBOLvJ9zxaSK7UrPsjqWUW+lUlWej0wi4ZRbk58DEvrDxO7sTKTeIDA2ytW9DqcqmheBsxXeGMQuhVkvrbuRje874I0opVZVp01BFRNS1zgx2L4dIx0jahQXaiezFsvIK2Hcsh/1p2ew7ls3B9FwGta9L01gdqFB5Pj1yVVRgCDTtbT3e+D0seA6GfwbROpyBt9l0IIPWT8w5ZXladj6PD9Z+IuX5tBC4QnB1SN8L43tbVxSdKBDK493UrRGx4cHUrVGN+pHVqBdZjXqRIQx4fQlFxlBYZLQ/QXk83x6G2pWObIcpN0DKZuj3LHS7Uwet82Ltxs0hN7+IgqIiXruu41ndiZyRk0/y0WySj2bTJDZMm5dUpdBhqCtDdBMYNQ++GQtz/gW1WumZgRcbe0lTth/OZPofyew5cvLlpZm5BY4DfVbx9z1Hskk+Zj0/lpVfvG3nRlF8ObZ7ZcdX6iRaCFwpOByu/RS2zoEmvaxlRYXgV/7AaMoz3d37PPILi5j+RzI/bzrExv0Z7HEc+EsPrBcc4EdcVDXio0PpGB9JXFQo8VGhTFiynYycAjYdSCf5yF+F40B6Drdd3JjzG0TZtHfK12ghcDU/P2gxwHp8cD1MuwmGvW9ddqq8ir8IseHBrElO40hmHvHRobSpV4P46GrERYVaB/+oUGpWDyrzLudvViXz08ZD9H99SfGykEA/cvKLqB9VTQuBqjTaR+BOhzbClOGQvg8GvwadbrQ7kXKxvIIi/AQC/M/+lpw1ycdYti2VuKi/CkdMWBAtH59Ns9rVaVYrnLt7N+W8WuFuSK58jfYR2KVWKxi9AL66Fb69Gw6shX7PgH+g3cmUiwQFVPyezPZxkbSPizxleVxUNTYfyGDd3nRa140oLgT5hUXsO5bN7iNZ7ErNwk+EG7o2qPD7K3WCFgJ3C42GEdNh3hPw69sQ2RAuvMvuVKoKm/vgJWTmFdB+3Fxm/rmPhVsOsftIFvuO5VBYdPIZfN/WtagVHmJTUuUttBBUBv8A6P8cNO5pzY8M2omsyuXvJ1QL9Ccuqhr7jmUT4B/K+Q2iuLJjKPHRoTSMDmXFziO8PHcLz8/aRL3IEB6+vKXdsZUH00JQmVr0t75npsLHQ6DXI9YEOEqVEujvx5J/9i53KO2s/EJE4Ls/91FQZHigb3MCK9BPoRTooHP2KCqAoFD4YiT8/CwUFdmdSFVBp5tPoXeLWmz6T38e6NusEhMpb6WFwA7hteGWH6DjjbD4RZg2AnLS7U6lPExwgDYtKtfQQmCXgGAY+hYMeBG2zIE5j9mdSHmwvq8u4tYPf7c7hvJQ2kdgJxHoegfUbgOxjs6+oiLrpjSlnHBBw2g6xEeSkpHLip1H+WP3UWKrBxMfHWp3NOVB9IhTFTS6GMJqQmE+fHolLHsTPOxGP2WPC5vG8O3dF9G/bR2O5xZw1TvLGPvZyuL1nnbDqLKHnhFUJYX5EFID5v7buvlsyBsQWM3uVMoD3HxhI+pHVuO7NfvYdug4909dRdKh4+xIyeTWixrp5aXqtPSMoCoJCoVrP4He/4Y10+DDAZC21+5UygM0iAnltosb06VRNOk5BSTuPEpM9WD8/YRthzLtjqeqOD0jqGpE4JKHoXZr+HoMfHkz3D5P5zZQTnl0QEsevKw5IYHWFUWXv7b4pPXGmNNelqp8kxaCqqrlIBg137rnQEQ7kZVTRKS4CJyw6UA693z+B1sPHmfXkUyeG9aOq86Psymhqoq0EFRltUq06/74MCDQ/3kdtE45rVZEMEu2plBoDM1rhbP5YAY7U7SpSJ1MC4EnMMbqNF72JhzeZM2LHFbT7lTKA0y6pTP5hUWEBln/1Rv/64fidcdzCziYnkPjmDD8dN5ln6aFwBOIWMNX124HM++F8b3h+s+hTju7k6kqLtDf75QxiKb/sZevV+0l+Wg2AJ/e3oUezWLtiKeqCG109iQdroPbfrT6DT4dBnlZZ/4ZpUro3DCa6sEBdGoQxcgLGwLw1HcbGPjGEg6m59icTtlFzwg8Tf0LYMxCOLTButwUrKYjvRJEOeGLsRcWP07LzmfxlsPkFhSSdOg4r/+0hdoRIdzbpxn+2lTkU7QQeKLw2tYXQOIka6yiq8ZbN6Mp5aQa1QJZ+HBv1u1NY/CbS5ny+x4AmsRWJ07nTPYp2jTk6YyBpJ9gYl9ISbI7jfJAbepFMOeBnjw3zOpzum/KKq5+dxkZOfk2J1OVRc8IPF3n26Fmc+vGswl94OpJ0Kyv3amUBxERWtQJp35UNXLyC1m3L42v/9hLfqGOU+Qr3HpGICL9RWSziCSJyKNlrK8hIt+JyJ8isl5EbnVnHq/VuAeMXgCRDeDzayF1m92JlAeqHhzAbRc3pkNcJAAfLN3OPZ//QZ9XFnLHp4n2hlNu5bZCICL+wNvAAKA1cL2ItC612d3ABmNMB6AX8IqIBLkrk1eLagi3z7HOCGKaWst05ElVAdWDrYaCtxdsY/WeY2TmFrBw82FGTvqdN+dvtTmdcgd3Ng11AZKMMdsBRGQqMBTYUGIbA4SLNfhJdeAIUODGTN4tKAzaXGk93vkL/PSkdfNZDR1OQDnvio71aFEnnAYxoUSEBPL+om28MncLv+9IZf3eNHYfyaJn81iGdKhnd1TlIu5sGqoP7CnxPNmxrKS3gFbAPmAtcL8xRifwdYX8LDi0Ccb3gl3L7U6jPEigvx9t69cgIsQayuSOS5qy6T/9GdapPqmZeXy5MplX523hme838PuOIzanVa7gzkJQ1oXIpdsqLgdWA/WAjsBbIhJxyguJjBGRRBFJPHz4sKtzeqdml8Ho+RAcAR8PsS4zVaqC/PyEp4e2JfHffendIpYdKZlMXLqDyb/tKnP71OO5LN2awvuLtvHcrI3kFhRWcmJ1NtzZNJQMxJd4Hof1yb+kW4EXjDWNUpKI7ABaAidNvmqMGQ+MB0hISNCGb2fFtoDRP8P02+H7ByG8HrTob3cq5aEC/f2oWT2Y16/rREpmLrd9tILCIsPOlEw27E9nw7704u8HSt2lPKBtHTrpfQlVljsLwQqgmYg0BvYCw4EbSm2zG7gUWCIitYEWwHY3ZvI91SLhhi9gzRfQrJ+1TO9EVuegRmggNUID8RPh+zX7+X7NfgD8/YTzYqvTvWkMretF0LpuBIeP53L/1NWnNAWoqsVthcAYUyAi9wBzAH9gkjFmvYiMdax/D/gP8JGIrMVqSnrEGJPirkw+y88fOl5vPT62G6bdCEP+B/U62hpLebbRPZqw6UA6bepF0LpuDZrVrn7KXAgLNx+yKZ06G269ocwYMwuYVWrZeyUe7wP6uTODKiUnHTJTYVJ/GPoWtLva7kTKQ93QtYHdEZSL6BATvqZOW2vQunqdrL6Dn8ZBkXbkKeXLtBD4ouqxMPJbSLgNlr4Gv7xhdyKllI10rCFfFRAEg1+DBt2hxQBrmXYiK+WT9IzA17W/BoKrQ14mfDTIGtJaKTc4lpXHkq2HWbxF7wWqavSMQFlyMyDvOHx+HVz6OFz8kJ4dKJcZ/XEiqZl5APgJPHRZc2pWD2Z4F+1wrgq0EChLeB24dbY1J/L8p+HAOuuqoqAwu5MpD9a8djgd4yOpH1mNtvVrcCAtm4+X7+LluVsI8BMWbD5Ei9rhPNSvhd1RfZoYDxuhMiEhwSQm6pC4bmOM1Xn80zhof60185lSLpKTX8j6feksS0rhlXlbipfHhAXRp2UtXrqmg43pvJuIrDTGJJS1Ts8I1MlE4OIHoHYbqNnMWqadyMpFQgL9uaBhFBc0jGJ0zybM/HMfH/6yk8MZOWw5mGF3PJ+lncWqbM0ug6hGVhGYfjusmKjzGyiXCgn059qEeH68vwdt6+t823bSMwJ1egU5kHscfvg77F8DA1+2Lj1VysU2H8yg1eOzuTYhjsEd6rEmOY3NB9K5sVtD2jtmTVPuoX0E6syKCmHBs7DkFYjvBtd9CtVr2Z1KeZH3F21j5p/7WL8vvcz10WFBvHxNe/q0rF3JybyH9hGoc+PnD5c+YfUbzLgbPvsb3LFY+w2Uy9xxSVPuuKQpP204yNq9abSPq0G7+jX494x17EvLZt3edLYdyqRPS7uTeictBMp5bf8GMc2sm89EtBNZuVzf1rXp2/qvT/3jRyZwPLeAtk/qjY7upIVAnZ267f96vOi/1k1ofZ+yzhqUUh5JrxpSFWMMZKXCsjdh8jWQfdTuREqpCtJCoCpGBAa+BEPegB2LYUIfOLTJ7lRKqQrQQqDOzQW3wC3fW5eYftjfmvhGKeVRtI9AnbsG3azJbpJXQEiE3WmUUmdJzwiUa9SoD22utB5vmAlf3GydJSilqjwtBMr1MvbDxpkw6XI4utPuNEqpM9BCoFyv6x0w4ktI2wPje1udyUqdowPpOczfeJDcAp1j29W0ECj3OK8vjF4AYbHwyZVwaKPdiZSHOnHL4gdLd3D7x4nM33jI1jzeSDuLlfvENIVRP8G66VCrld1plIcKCw7g2WFtScnI47WftpCTr2cErqZnBMq9QiIg4Vbr8YF18MlQyDhgbyblcUZ0bciVnerZHcNraSFQlSctGfb8DuN7QfJKu9MopRy0EKjK06I/3D4X/APhwwGweordiZQH2p+Wwy9JKRQVedYQ+lWZ9hGoylWnHYxeCF/eDDPGQlAYtL7C7lTKA4ij2/ilOZsBCA3yJzosiL/3a06gvx+D22vTUUVpIVCVLywGbvoGVnwALQbYnUZ5iPjoavyjX3My8wp5d+E2svIKycrL5sFpfwLQtXEMseHBNqf0TFoIlD38A6HbWOtxZgp8dRsM+K9eXaTKJSLc06cZAP+8vAW5BUXMWLWXjfvT+Xj5Lgq1qajCtI9A2S99HxzeBBP7wsbv7U6jPICIEBLoz/AuDWhZV8e3OldaCJT96ra3Bq2r2RymjYCF/4WiIrtTKQ/z0pzNjJu5ntV7jrHpgI6Ceza0aUhVDRH14NYf4fsHYOFz1nwHl/zT7lTKA9SoFgjA9D+SAfho2U5qRwTz22N97YzlUbQQqKojMASufBfiu0IrvZJIOad/mzosergXhUWGb1btZeWuo6zbm2Z3LI+iTUOqahGx7kQOi4HCfJg6ArYvtDuVqsL8/ISGMWE0ia3O3/u1oHntcLsjeRwtBKrqykqF1G3w6VXw67vWPMlKKZfTQqCqrvA6MGqeda/B7Efh27shP8fuVEp5HS0EqmoLDodrP4VLHoXVk2HGnXYnUsrraGexqvr8/KD3v6B2G4huYnca5SGO5xYQ4Gfdb6BOT88IlOdofQXUaWs9/vFRWDXZ3jyqykrPKaD9uDnc9MFvdkfxCHpGoDxPfg4c2gC/vQsH1kK/Z8Bf/5SVpVeLWHalZrIjJZPU43l2x/EIbj0jEJH+IrJZRJJE5NFytuklIqtFZL2ILHJnHuUlAkPgxq+h211WMfjsKsg6YncqVUX0alGLD2/tQru4SLujeAy3FQIR8QfeBgYArYHrRaR1qW0igXeAK4wxbYBr3JVHeRn/AOj/PAx9B3Yvh4+HQJFOYahURbjzfLoLkGSM2Q4gIlOBocCGEtvcAHxtjNkNYIzRWanV2ek0AmJbWCOY+mmnoDrZsex8Hp+xji6NoxnSQecrKI87m4bqA3tKPE92LCupORAlIgtFZKWIjHRjHuWt4hKs2c8AVkyEBc/poHWK6sEBHMnM49Nfd/HJ8p12x6nS3FkIpIxlpW8NDQAuAAYBlwOPi0jzU15IZIyIJIpI4uHDh12fVHmPA2th0X9h2o2Qm2F3GmWjfw1syewHetC1cbTdUao8dxaCZCC+xPM4YF8Z28w2xmQaY1KAxUCH0i9kjBlvjEkwxiTExsa6LbDyAoNfh/7/hS2zYeJl1hAVyidFhATSsk4E/n5lfSZVJbmzEKwAmolIYxEJAoYDM0tt8y3QQ0QCRCQU6ApsdGMm5e1ErJnPbvoajh+AiZfqFUVKnYHbOouNMQUicg8wB/AHJhlj1ovIWMf694wxG0VkNrAGKAImGmPWuSuT8iFNesHoBbBjEYRq04BSpyPGw0Z0TEhIMImJiXbHUJ5mxxJYMxUGvmLdh6B8xg0TfiX5aDaXta5N/chqhAX7071pTeKjQ+2OVqlEZKUxJqGsdU6dEYjIRcA4oKHjZwQwxhgd+EV5hgNrYNVncHADDJ9szYimfEJokD+7j2TxwdIdxctu6d6IcVe0sTFV1eJs09AHwIPASkDv2lGe58K7IbIhfHMHjO8F130G8V3sTqUqwXPD2nFPnxyy8wrZn5bNuJnryS/Uy4tLcrazOM0Y86Mx5pAxJvXEl1uTKeVqrQbDqJ8gMBQ+GgT7/7Q7kaoEtSJC6BgfyYVNY7jq/DiCAnSszdKcPSNYICIvAV8DuScWGmP+cEsqpdylVisY/TOs/AjqtLc7jVJVgrOFoKvje8mOBgP0cW0cpSpBaDT0eMh6fHQnzH3cuv8gLMbOVKoSJR06znOzNtK8djh5BUUMbFeHyNAgu2PZxqlCYIzp7e4gStni4AbYMgcm9ILhn0OddnYnUm4WHODPbzuO8NuOv+4vMRhGdG1oYyp7OXX5qIjUAJ4EejoWLQKeNsakuTFbmfTyUeVye1fC1BGQkwZXvgNthtmdSLnRpgPpZOYWcig9h/ScfB6ZvpYezWpSKzyER/q3oFaEd15efLrLR53tNZkEZADXOr7SgQ9dE08pm9W/AMYshNpt4ctbYN3XdidSbtSyTgQXNIxiQLu6DGpfj0B/YcnWFKb/kcyKnUftjmcLZ/sImhpj/lbi+VMistoNeZSyR3gduOV7WPo6NL/c7jSqklQPDuCnhy5hf1oOw8f/ancc2zh7RpAtIhefeOK4wSzbPZGUsklAMPR6BILCrJFLvxipg9b5gIYxYUSH+W5HMThfCO4E3haRnSKyC3gLGOu+WErZ7MgOa1iKCb0h6Se70yjlVk4VAmPMamNMB6A90M4Y08kYo3fjKO9Vt73Vb1AjHiZfA7/8DzxsXC6lnHXaPgIRudEY85mIPFRqOQDGmFfdmE0pe0U1hNvnwow7Yd7jkJ8FvR61O5VSLnemzuIwx/dwdwdRqkoKCoNrPoZlb0KbK+1Oo5RbnLYQGGPed3x/qnLiKFUFicBF91mPi4rg+weg4w3QoJutsZRyFaf6CETkRRGJEJFAEZkvIikicqO7wylV5WSlwM4l8NFga7wipbyAs1cN9TPGpAODseYZbg487LZUSlVV1WtZg9Y17gnf3Q8//B0K8+1OpdQ5cbYQBDq+DwSmGGN0Eljlu6pFwYgvoft9sGIiTB9ldyKlzomzdxZ/JyKbsG4iu0tEYoEc98VSqorz84d+/7EGqasRZ3capc6Js6OPPioi/wXSjTGFIpIJDHVvNKU8QPtr/3q8+CWIbgJt/1b+9kpVQadtGhKRPo7vVwG9gaGOx/2B7u6Pp5SHKMyHpPnw1W3w01NQpDO6eqL/zt7Ehc/PJ/lolt1RKtWZzgguAX4GhpSxzmDNWKaU8g+EkTPhx4dh6atwcB38bSKE1LA7mXJCrfBg6kdWA2B/Wg7JR7OJiwq1OVXlOdN9BE86vt9aOXGU8mABQTDkDWsKzB//CZMGwB2Lwd/Zrjhll8jQIH55tA/LtqVww4Tf7I5T6Zy9j+A5EYks8TxKRJ5xWyqlPFnn262zg+73ahHwcBk5+RxI8/7rYpz9Kx1gjHnsxBNjzFERGQj82z2xlPJwjS4CLrIer58BR7bDxQ9adymrKu/9RdsYN3M9mw5kANC9aQyx4cG8MbyTzcncw9n7CPxFJPjEExGpBgSfZnul1AlJ82D+U1ZHcp5vdUJ6mhPzEqzYeZTY8GBa1Y1wPD/Cd3/uszOaWzl7RvAZMF9EPsTqJL4N+NhtqZTyJle8BdFNYf7TkJoEwz+HyHi7U6kytKwTwYr/60t0WBD+ftbZW25BIW/9nMTbC5JsTuc+zs5H8CLwDNAKaAP8x7FMKXUmItDjIbjhCzi6E8b3goyDdqdS5YgNDy4uAgDBAf42pqkcZ9OTtREoMMb8JCKhIhJujMlwVzClvE7zftY4RRtnQnhtu9Oos1RkrPsM6kVW46ZuDe2O41LOXjU0GvgKeN+xqD4ww02ZlPJeNZtBj79bj/evgVn/hII8ezOpMwoLtj4zv7twG/+bv9XmNK7nbGfx3ViXQKQDGGO2ArXcFUopn7BjEfz+PnxyBRw/ZHcadRq3dG/EzHsu4qpO9e2O4hbOFoJcY0zxxxYRCcDqNFZKVVT3e+FvH8C+1TC+t/VdVUkhgf60j4skONDqL8jIySe3wHuGEXG2ECwSkceAaiJyGfAl8J37YinlI9pdDbfPsTqUJ10OySvtTqTO4HBGLu3GzaXduLmMX7yNFTs9f1R+ZwvBI8BhYC1wBzALvZlMKdeo2wFGL4Auo6Fue7vTqNPo26oWfVtZreJ5BUU8N2uTV/QZiDGnb+ERET9gjTGmbeVEOr2EhASTmJhodwyl3Of4YZg/Dvo9Y02Co6qcvcey2XIgg5fnbqZaoD+PDmhJk9jqxTekVUUistIYk1DWujOeERhjioA/RaSBy5MppU6VvAL+nAYTLoXDm+1Oo8pQP7IavVvWIjTIn8RdR7n6veW8OHuT3bEqzNmmobrAesfE9TNPfLkzmFI+q+VAuPk7yE23isHmH+1OpMpxV6/zuO/SZtSsHkRWnud2Hjt7Q9lTbk2hlDpZwwthzEKYegNMud6a26Dd1XanUqX0blmL3i1refw4RKctBCISAowFzsPqKP7AGFNQGcGU8nk14uDW2bDgWWjax+40youdqWnoYyABqwgMAF5xeyKl1F+CQuHyZyE02roD+bsH4Oguu1OpMhxIy+GT5TvZc8QaYdYYw9HMPM50QU5VcKamodbGmHYAIvIB8Lv7IymlypSyBdZ9bY1VdO0n0OhiuxMph0B/4fedR/h95xFgPb1axPLr9lRy8osAayrMz0Z1pXntcHuDluNMZwT5Jx5UpElIRPqLyGYRSRKRR0+zXWcRKRQRbQRVqjx12lqD1oXGwCdD4fcJ4AGfNn3BK9d0ZMLIBMKCrDuPD6TlcHmbOgA0iQ3jUEYuL8/ZzCNfrSG7CnYqn/Y+AhEpBDJPPAWqAVmOx8YYE3Gan/UHtgCXAcnACuB6Y8yGMrabB+QAk4wxX50usN5HoHxeTjp8PRq2zIZLHoHej535Z1SlyMjJx0+keJA6gN2pWfR8acFJ29WtEcLcB3sSHhJYadlOdx/BmSavP5eBuLsAScaY7Y4QU4GhwIZS290LTAc6n8N7KeU7QiJg+BRY9F9oPdTuNKqEsg7sDWJC+eXRPmTmFnDv56vIKyxiR0omRzPzK7UQnI6z9xFURH1gT4nnyY5lxUSkPjAMeM+NOZTyPn5+0PtfULu11Tw0/2nYq+MUVVX1I6vRvHY4cx7syT29z7M7zincWQjKmqW7dDvU68AjxpjTNpqJyBgRSRSRxMOHD7sqn1LeIesIrPkSJg2w7khWHuH6Cb/S99VFFBQW2R3FrYUgGSg5MWscUPquiwRgqojsBK4G3hGRK0u/kDFmvDEmwRiTEBsb66a4SnmosBgYswDiu8A3Y2Duv6Go6nVIKkuLOuG0qB1OoL+QdOg4uQXeXQhWAM1EpLGIBAHDgZOGpTDGNDbGNDLGNMKaAe0uY8wMN2ZSyjuF1YSbvoEuY2DZm/DVrXYnUuVoW78Gcx7syQ1dreHbrn1/OaM+XkFeQZFtZwdnM2fxWTHGFIjIPcAcwB/riqD1IjLWsV77BZRyJf9AGPgS1GlnXWKqqrRmtcKpFR7MwfRc1u9Lp82Ts2kfF8n0O7tXepYzDkNd1ejlo0qdhcQPoXotaDnI7iSqHMuSUpi4dAc7UzJJOZ7LdZ3j6dQgioHt6rr0fc5pGGqllIcqKoTVn1sD1y16EYrsb4tWp+p+Xk0m3dKZns1jSc8pYMKSHby/aFulZtBCoJS38vOHm2dC++usgeu+vBlyj9udSpXjn/1bMO/BnvRoVrPS31sLgVLeLLAaDHsf+j0Lm76HSf2hINfuVKoMoUEBNKsdjr9fWVfeu5fbOouVUlWECHS/B2q1gsObICDY7kSqitFCoJSvOO9S6wtg+yKrKHQZYxUKVaXkFxrWJqexYX8amw8cZ1D7OlzQMNpt76eFQClftOYLWP0Z7F8Dg1/Vs4QqRIAN+9MZ8tbS4mUZOflaCJRSLnbFm1CjvjVwXcpmuO4zCK9jdyoFjO7ZhLb1a9CqbgSt60Zw/YRf3f6eWgiU8kV+ftbw1bXbwDd3wvheMGq+VRyUrbo3rUn3pn9dOVQZDXdaCJTyZa2HQsx51v0GEfXsTqNsopePKuXrarex5kUWgSM7rCGtC896QkLlwbQQKKX+sul7WPIKTP6bNby18glaCJRSf+l+Lwx9G3Ytgwm94WDpCQWVN9JCoJQ6Wacb4ZZZkJ8DE/vCnhV2J1JupoVAKXWq+M4wZiG0vcrqQ1BeTQuBUqpsEXVh6FsQFAq5GTD7Meu78jpaCJRSZ7bzF/jtPZh4GRzZbnca5WJaCJRSZ9aiP9w4HTL2w/jesG2B3YmUC2khUEo5p2lvGLMAwuvCZ1fB2q/sTqRcRAuBUsp50U1g1DzoOALiu9qdRrmIFgKl1NkJDrc6kSPjrekvfxoH6fvsTqXOgRYCpVTFpWyG3ydY/QZ6v4FbFRUZcvIL3fLaWgiUUhVXqxXcPg8CQ+CjgbDqM7sTeaXZ6w7Q5sk5vLvQPZPaayFQSp2b2q1h9AJo2B2+vRsWvmB3Iq8yqH1dOjaI5PouDejS2D2T0+gw1EqpcxcaDSOmw09PQpNedqfxKv83qLXb30PPCJRSruEfYA1n3aCb9Xz5O3Bgnb2ZlFO0ECilXC8nDZa9CR9cBhu+tTuNOgMtBEop1wupYd18VrsNfDESfn7WutRUVUlaCJRS7hFeB275wRrWevGL8NWtYIzdqVQZtLNYKeU+AcFwxVtQpwMEVrOmw1RVjhYCpZR7iUDXMX893/AtBFWH8y61L5M6iTYNKaUqT1GR1Yk8+WpY9pY2FVURWgiUUpXHzw9umgEtB8Pc/4Nv7oD8bLtT+TwtBEqpyhVcHa79BHr/G9ZMgw8HQF6W3al8mvYRKKUqnwhc8rB1eWnyCms6TGUbLQRKKfu0HGh9AexbBQfWwvkj7c3kg7RpSClVNfw+EWbeCz/8Awrz7U7jU/SMQClVNQx5A0KjrKuKDm+Caz6CsJp2p/IJekaglKoa/AOg3zMwbDzs+d2a7ObYbrtT+QQ9I1BKVS0droOazeD38RBez+40PkHPCJRSVU/982HYe9ZZQsZBWPKqDlrnRm4tBCLSX0Q2i0iSiDxaxvoRIrLG8bVMRDq4M49SygOt/QLmPwVTr7eGt1Yu57ZCICL+wNvAAKA1cL2IlJ5qZwdwiTGmPfAfYLy78iilPNSF98DAlyHpJ5jYF1KS7E7kddx5RtAFSDLGbDfG5AFTgaElNzDGLDPGHHU8/RWIc2MepZQnEoEuo62hKbJSYUIfqzNZuYw7C0F9YE+J58mOZeW5HfjRjXmUUp6scQ8YvcD6HnOe3Wm8ijsLQVkDj5c51KCI9MYqBI+Us36MiCSKSOLhw4ddGFEp5VGiGsLwyRAaDQW5sOhFHbTOBdxZCJKB+BLP44B9pTcSkfbARGCoMSa1rBcyxow3xiQYYxJiY2PdElYp5WG2L4IFz8Gky+HYnjNvr8rlzkKwAmgmIo1FJAgYDswsuYGINAC+Bm4yxmxxYxallLdp3g+unwqp22FCb9i1zO5EHstthcAYUwDcA8wBNgJfGGPWi8hYERnr2OwJIAZ4R0RWi0iiu/IopbxQi/4w+mcIqQEfD4G1X9mdyCOJ8bAZghISEkxiotYLpVQJ2cfg+wfgkkehVku701RJIrLSGJNQ1jq9s1gp5fmqRVqD1NVqaU1/uewtOK4XljhLC4FSyrsc2Q4/PwPje8G+1Xan8QhaCJRS3iWmKdw223o8qb/2GzhBC4FSyvvU6whjFlrfp98OC/9rc6CqTQuBUso7VY+FkTMh4Xao297uNFWazkeglPJeAUEw+NW/nv85FeqdD7HN7ctUBekZgVLKN+Rlwk/jYOKlsHm23WmqFC0ESinfEBQGt8+D6MYwZTgsecW61FRpIVBK+ZDIeLh1NrT9G8x/GqaP0mKA9hEopXxNUCj8bSLUaQcYa74DH6eFQCnle0Tg4gf+er59Efj5Q6OLbYtkJ20aUkr5NmNgwbPwyVBYMdEnm4q0ECilfJsIjPgSml4KP/zdGryuIM/uVJVKC4FSSoXUgOunwMUPwcqPrCGtczPsTlVptI9AKaXA6iPo+yTUaQvbfoag6nYnqjR6RqCUUiW1/RsMfdtqMkrd5hOD1mkhUEqp8vzyhjVo3dx/Q1Gh3WncRpuGlFKqPINeAf8gWPYmHNwAV38A1aLsTuVyekaglFLl8Q+EQS/DkDdgx2KY0AeO7rI7lctpIVBKqTO54Ba45XuIaQbVa9mdxuW0ECillDMadIMRX0BgNchJh98neM3NZ1oIlFLqbK2eDLP+AV/eYg1v7eG0s1gppc5W17FQmA8/PQmpSTB8MkQ1sjtVhekZgVJKnS0RuOg+a2iKtD0wvjfs+d3uVBWmhUAppSrqvL4wegHUbgMR9e1OU2FaCJRS6lzENLWuKKpR37rp7Lf3oSDX7lRnRQuBUkq5yo7F8OM/4aPBkHHA7jRO00KglFKu0rQ3XPMxHFxn9RvsXWl3IqdoIVBKKVdqcyXcPhf8A2DSAFj3td2JzkgLgVJKuVqddjB6ITTuATXi7E5zRloIlFLKHcJi4MbpEN/Fer5qMmQdsTdTObQQKKWUux3bbU2BOaE3HNpod5pTaCFQSil3i2wAt/wA+dkwsS9s/N7uRCfRQqCUUpUhvguMWQg1m8O0EbD4ZbsTFfOKsYby8/NJTk4mJyfH7ihKnSQkJIS4uDgCAwPtjqKqgoh6cOuP8N39EF7X7jTFvKIQJCcnEx4eTqNGjRARu+MoBYAxhtTUVJKTk2ncuLHdcVRVERgCw96zxisC2DwbYltAtH1/I17RNJSTk0NMTIwWAVWliAgxMTF6pqpOdeJYlZ8D3z9odSJvX2hbHK8oBIAWAVUl6d+lOq3AELj1B6heBz69Cn5915bJbrymENhNRLjpppuKnxcUFBAbG8vgwYPP6nUaNWpESkrKOW9TnilTpvDss8+etGz16tXMmjWrQq8HMGPGDDZs2OD09vfffz/169enqKioeNm4ceN4+eWTO89K7ueBAwcYPnw4TZs2pXXr1gwcOJAtW7ZUODNAbm4u1113Heeddx5du3Zl586dZW43bdo02rdvT5s2bfjnP/950rovvviC1q1b06ZNG2644YZzyqN8VHQTGDUPWgyA2Y/Ct3dDif8blUELgYuEhYWxbt06srOzAZg3bx7161e9YWlnz55N//79T1pWmYWgqKiIb775hvj4eBYvXuzUzxhjGDZsGL169WLbtm1s2LCB5557joMHD1Y4M8AHH3xAVFQUSUlJPPjggzzyyCOnbJOamsrDDz/M/PnzWb9+PQcPHmT+/PkAbN26leeff55ffvmF9evX8/rrr59THuXDgsPh2k/hkkehWhT4Ve6hWQuBCw0YMIAffvgBsD55X3/99cXrjhw5wpVXXkn79u3p1q0ba9asAawDTb9+/ejUqRN33HEHpsRp4WeffUaXLl3o2LEjd9xxB4WFheW+9xdffMFDDz0EwBtvvEGTJk0A2LZtGxdffDFgHVBXr17N+eefX/xzeXl5PPHEE0ybNo2OHTsybdo0MjMzue222+jcuTOdOnXi22+/BeC+++7j6aefBmDOnDn07NmTZcuWMXPmTB5++GE6duzItm3bTvs7WrBgAW3btuXOO+9kypQpTv1eFyxYQGBgIGPHji1e1rFjR3r06OHUz5fn22+/5eabbwbg6quvZv78+Sf9/gG2b99O8+bNiY2NBaBv375Mnz4dgAkTJnD33XcTFRUFQK1a3jepuapEfn7Q+1/Q7xnr+b5VsGdFpby1W68aEpH+wBuAPzDRGPNCqfXiWD8QyAJuMcb8cS7v+dR369mwL/1cXuIUretF8OSQNmfcbvjw4Tz99NMMHjyYNWvWcNttt7FkyRIAnnzySTp16sSMGTP4+eefGTlyJKtXr+app57i4osv5oknnuCHH35g/PjxAGzcuJFp06bxyy+/EBgYyF133cXkyZMZOXJkme/ds2dPXnrpJQCWLFlCTEwMe/fuZenSpcUHzFWrVtGhQ4eT2q2DgoJ4+umnSUxM5K233gLgscceo0+fPkyaNIljx47RpUsX+vbtywsvvEDnzp3p0aMH9913H7NmzaJp06ZcccUVDB48mKuvvhqA9957D+CkA/cJJwrk0KFDeeyxx8jPzz/jpZXr1q3jggsuOOPvH6BHjx5kZGScsvzll1+mb9++Jy3bu3cv8fHxAAQEBFCjRg1SU1OpWbNm8TbnnXcemzZtYufOncTFxTFjxgzy8vIAipumLrroIgoLCxk3btwpZ1tKnbUT/z/n/B8kr4DBr0OnEW59S7cVAhHxB94GLgOSgRUiMtMYU7INYQDQzPHVFXjX8d0jtW/fnp07dzJlyhQGDhx40rqlS5cWf5Ls06cPqamppKWlsXjxYr7+2hqdcNCgQcWfLufPn8/KlSvp3LkzANnZ2af9xFmnTh2OHz9ORkYGe/bs4YYbbmDx4sUsWbKEq666CrCahQYMGHDG/Zg7dy4zZ84sbrPPyclh9+7dtGrVigkTJtCzZ09ee+01mjZtWubPl1UAwDr7mDVrFq+99hrh4eF07dqVuXPnMmjQoHI7Vc+2s/VE4XVG6U//Zb1fVFQU7777Ltdddx1+fn50796d7du3A1Y/0NatW1m4cCHJycn06NGDdevWERkZeVaZlSrTdZ/BlzfDt3fBgbXWmYK/ew7Z7jwj6AIkGWO2A4jIVGAoULIQDAU+Mdb/yF9FJFJE6hpj9lf0TZ355O5OV1xxBf/4xz9YuHAhqampxctPd9Ap62BnjOHmm2/m+eefd/q9L7zwQj788ENatGhBjx49mDRpEsuXL+eVV14BrAP8iWJ0OsYYpk+fTosWLU5Zt3btWmJiYti3b5/TuU6YPXs2aWlptGvXDoCsrCxCQ0MZNGgQMTEx7N9/8j97RkYGkZGRtGnThq+++sqp9zibM4K4uDj27NlDXFwcBQUFpKWlER0dfcrPDhkyhCFDhgAwfvx4/P39i3++W7duBAYG0rhxY1q0aMHWrVuLi7dS5yQ0Gm78BuY9Dr++A4c2WMUhJMLlb+XOPoL6wJ4Sz5Mdy852G0RkjIgkikji4cOHXR7UlW677TaeeOKJ4oPdCT179mTy5MkALFy4kJo1axIREXHS8h9//JGjR48CcOmll/LVV19x6NAhwOpj2LVr12nfu2fPnrz88sv07NmTTp06sWDBAoKDg6lRowZpaWkUFBQQExNzys+Fh4efdPC8/PLLefPNN4uL16pVqwDYtWsXr7zyCqtWreLHH3/kt99+K/PnyzNlyhQmTpzIzp072blzJzt27GDu3LlkZWXRs2dPZs6cWfw6X3/9NR06dMDf358+ffqQm5vLhAkTil9rxYoVLFq06JT3WLJkCatXrz7lq3QRAKtof/zxxwB89dVX9OnTp8yifOLf4OjRo7zzzjuMGjUKgCuvvJIFCxYAkJKSwpYtW4r7ZpRyCf8A6P88DH0HgsIgMNQ972OMccsXcA1Wv8CJ5zcBb5ba5gfg4hLP5wMXnO51L7jgAlPahg0bTllW2cLCwk5ZtmDBAjNo0CBjjDGpqanmiiuuMO3atTNdu3Y1f/75pzHGmJSUFHPZZZeZTp06mQceeMA0aNDAHD582BhjzNSpU02HDh1Mu3btzPnnn2+WL19ujDGmYcOGxduUlJSUZACzefNmY4wxl112mbn33nuNMcZ8+eWX5sknnywze2pqqklISDAdOnQwU6dONVlZWWbMmDGmbdu2pk2bNmbQoEGmqKjIXHrppebbb781xhiTmJho2rZta7Kzs83SpUtNq1atTMeOHU1SUpJ59913zbvvvnvSe2RmZpqoqCiTlpZ20vJhw4aZqVOnGmOMee+990z79u1Nhw4dzGWXXWa2bdtWvN3evXvNNddcY5o0aWJat25tBg4caLZs2XKaf5Ezy87ONldffbVp2rSp6dy580nv16FDh+LHw4cPN61atTKtWrUyU6ZMKV5eVFRkHnzwQdOqVSvTtm3bk9aVVBX+PpUXKCo6px8HEk05x1Uxbrp5QUQuBMYZYy53PP+Xo/A8X2Kb94GFxpgpjuebgV7mNE1DCQkJJjEx8aRlGzdupFWrVq7fCS8yatQoRo0aRbdu3eyO4nP071NVBSKy0hiTUNY6d/YRrACaiUhjYC8wHCh9x81M4B5H/0FXIO10RUBV3MSJE+2OoJSqotxWCIwxBSJyDzAH6/LRScaY9SIy1rH+PWAW1qWjSViXj97qrjxKKaXK5tb7CIwxs7AO9iWXvVfisQHudmcGpZRSp+c1dxa7q69DqXOhf5fKE3hFIQgJCSE1NVX/06kqxTjmIwgJCbE7ilKn5RUT08TFxZGcnExVv8dA+Z4TM5QpVZV5RSE4cWenUkqps+cVTUNKKaUqTguBUkr5OC0ESinl49w2xIS7iMhh4PSjr5WvJlCxOR49l+6zb9B99g3nss8NjTGxZa3wuEJwLkQksbyxNryV7rNv0H32De7aZ20aUkopH6eFQCmlfJyvFYLxdgewge6zb9B99g1u2Wef6iNQSil1Kl87I1BKKVWKVxYCEekvIptFJElEHi1jvYjI/xzr14jI+XbkdCUn9nmEY1/XiMgyEelgR05XOtM+l9ius4gUisjVlZnPHZzZZxHpJSKrRWS9iJw6sbOHceJvu4aIfCcifzr22aPnNRGRSSJySETWlbPe9cev8uaw9NQvrElwtgFNgCDgT6B1qW0GAj8CAnQDfrM7dyXsc3cgyvF4gC/sc4ntfsaaF+Nqu3NXwr9zJLABaOB4Xsvu3JWwz48B/3U8jgWOAEF2Zz+Hfe4JnA+sK2e9y49f3nhG0AVIMsZsN8bkAVOBoaW2GQp8Yiy/ApEiUreyg7rQGffZGLPMGHPU8fRXwNOHxHTm3xngXmA6cKgyw7mJM/t8A/C1MWY3gDHG0/fbmX02QLiICFAdqxAUVG5M1zHGLMbah/K4/PjljYWgPrCnxPNkx7Kz3caTnO3+3I71icKTnXGfRaQ+MAx4D+/gzL9zcyBKRBaKyEoRGVlp6dzDmX1+C2gF7APWAvcbY4oqJ54tXH788ophqEuRMpaVvjTKmW08idP7IyK9sQrBxW5N5H7O7PPrwCPGmELrw6LHc2afA4ALgEuBasByEfnVGLPF3eHcxJl9vhxYDfQBmgLzRGSJMSbdzdns4vLjlzcWgmQgvsTzOKxPCme7jSdxan9EpD0wERhgjEmtpGzu4sw+JwBTHUWgJjBQRAqMMTMqJaHrOfu3nWKMyQQyRWQx0AHw1ELgzD7fCrxgrAb0JBHZAbQEfq+ciJXO5ccvb2waWgE0E5HGIhIEDAdmltpmJjDS0fveDUgzxuyv7KAudMZ9FpEGwNfATR786bCkM+6zMaaxMaaRMaYR8BVwlwcXAXDub/tboIeIBIhIKNAV2FjJOV3JmX3ejXUGhIjUBloA2ys1ZeVy+fHL684IjDEFInIPMAfrioNJxpj1IjLWsf49rCtIBgJJQBbWJwqP5eQ+PwHEAO84PiEXGA8esMvJffYqzuyzMWajiMwG1gBFwERjTJmXIXoCJ/+d/wN8JCJrsZpNHjHGeOyopCIyBegF1BSRZOBJIBDcd/zSO4uVUsrHeWPTkFJKqbOghUAppXycFgKllPJxWgiUUsrHaSFQSikfp4VAqTI4RitdLSLrHCNbRrr49XeKSE3H4+OufG2lzpYWAqXKlm2M6WiMaYs1ANjddgdSyl20ECh1ZstxDOolIk1FZLZjQLclItLSsby2iHzjGBP/TxHp7lg+w7HtehEZY+M+KFUur7uzWClXEhF/rOELPnAsGg+MNcZsFZGuwDtYg539D1hkjBnm+Jnqju1vM8YcEZFqwAoRme4F4zwpL6OFQKmyVROR1UAjYCXWiJbVsSb4+bLEaKbBju99gJEAxphCIM2x/D4RGeZ4HA80A7QQqCpFC4FSZcs2xnQUkRrA91h9BB8Bx4wxHZ15ARHpBfQFLjTGZInIQiDEHWGVOhfaR6DUaRhj0oD7gH8A2cAOEbkGiueOPTH383zgTsdyfxGJAGoARx1FoCXWtIJKVTlaCJQ6A2PMKqy5cocDI4DbReRPYD1/TZt4P9DbMQLmSqANMBsIEJE1WCNk/lrZ2ZVyho4+qpRSPk7PCJRSysdpIVBKKR+nhUAppXycFgKllPJxWgiUUsrHaSFQSikfp4VAKaV8nBYCpZTycf8PnJmxOFUMJRAAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# plot pr curve\n", "train.plot_pr_curve(X_test, y_test)" ] }, { "cell_type": "markdown", "id": "fa34b9d3", "metadata": {}, "source": [ "### Interpret the Model" ] }, { "cell_type": "markdown", "id": "2ca793cf", "metadata": {}, "source": [ "#### Categorical and Numerical Features" ] }, { "cell_type": "code", "execution_count": 32, "id": "5ad792da", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/xgboost/sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n", " warnings.warn(label_encoder_deprecation_msg, UserWarning)\n", "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/sklearn/utils/validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", " return f(*args, **kwargs)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "[16:15:16] WARNING: ../src/learner.cc:1095: Starting in XGBoost 1.3.0, the default evaluation metric used with the objective 'binary:logistic' was changed from 'error' to 'logloss'. Explicitly set eval_metric if you'd like to restore the old behavior.\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAEICAYAAADC9PcJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAstklEQVR4nO3deZgcVb3/8fcnwxJIQgISvCGgAxhlC4QwgMhyg6wCKjyg8cqS4MJFEBV/CFFBIOIVhCtc9IoGBGTf4QIREwRCkDUTSDKAgkCi7Mg2BAKRJN/fH3VGOk33TC8z0zOpz+t5+qG66tQ531M9pL51TnWXIgIzMzPLrwGNDsDMzMway8mAmZlZzjkZMDMzyzknA2ZmZjnnZMDMzCznnAyYmZnlnJMBM6uZpJD0sUbHYWb1cTJgllOSBktaIOnLBeuGSPq7pAMbGVs5kk6WdGkXZRZIekfSWwWvdetsd4Gk3eqpo8r2uuxnb5E0UdKfGh2H9SwnA2Y5FRFvAYcD/yNpeFr9M6A1Iq5tXGTd4rMRMbjg9Xwjg5G0UiPbr1V/jduq52TALMciYjowFThH0jjgi8BRHdslfUjSzZLelDRL0qklrhL3lvS0pFcknSFpQNp3gKQTJP1N0suSLpY0tKDuz0l6VNIbkmZI2qRg2/GSnpO0UNLjknaVtBfwA2B8utqfW01fJQ2V9FtJL6S6T5XUlLZtJOkOSa+mflwmaVjadgnwEeDm1O5xksZJerao/n+NHqQr+2slXSrpTWBiZ+1XEHtIOlLSX9Mx+XGK+b702VwtaZVUdpykZyX9IPVlgaSDio7DxZL+kT6bEwo+s4mS7pF0lqTXgKuAXwPbp76/kcrtI+nh1PYzkk4uqL85xTshjTK9IumHBdubUmxPpb7MlrR+2raxpNskvZY+9y9W8xlbHSLCL7/8yvELWBN4AXgFOKxo25XptTqwKfAM8KeC7QHcCaxFdsJ8Avha2vYV4ElgQ2AwcD1wSdr2ceBtYHdgZeC4VHYV4BOpnXVT2WZgo7R8MnBpF/1ZAOxWYv2NwG+AQcA6wIPAf6ZtH0uxrAoMB2YCZ5erExgHPFuu3RTne8B+ZBddq3XWfolYl+tnOs43AWsAmwGLgdvTsR0KPAZMKIhtCfDz1J9/T8f6E2n7xcD/AUPSsX0C+GraNjHtezSwUop7YuFnXtDG6NS3LYCXgP0KPq8Azkv7b5ni3SRt/x7Qlj5npe0fSsflGeCw1PZYsr/JzRr9/0geXg0PwC+//Gr8C/gjsAgYWrCuKZ3QPlGw7lQ+mAzsVfD+SOD2tHw7cGTBtk+k+lYCTgSuLtg2AHgunWQ+BrwM7AasXBTncifJMn1ZALwFvJFeNwIfTiek1QrK/QdwZ5k69gMeLqqz2mRgZsG2attfrp/pOO9Q8H42cHzB+/8mJS+8nwwMKth+dTrmTSmOTQu2/ScwIy1PBP5eFMtEipKBEvGeDZyVlptTvOsVbH8Q+FJafhz4fIk6xgN3F637DXBSo///yMPL80FmOSfpYLJ/wP8InA4ckTYNJztxP1NQ/Bk+qHDd34COm/XWTe8Lt61EdmJcbltELJP0DDAyImZI+g7ZCXEzSdOA70Z18/77RcQfC/q4LdkIxAuSOlYP6Ihd0jrAOcBOZFfMA4DXq2ivlMLj8tHO2q/QSwXL75R4/28F71+PiLcL3nd8LmuTjb4Ufy4jy8RdkqTtgNOAzVN9qwLXFBV7sWB5EdnoEMD6wFMlqv0osF3HVESyEnBJV/FY/XzPgFmOpZPgWcDXya4Qvyhp57T5H2RXmOsV7LJ+iWoK130E6DhpP0/2D3zhtiVkJ7Hltik7Q65PNjpARFweETumMkGWpJCWa/EM2RXx2hExLL3WiIjN0vafprq3iIg1gIPJhrA7FLf7NtnUSUf8TWTJU6HCfbpqv7utKWlQwfuOz+UVstGZ4s/luTJxl3oPcDnZtMX6ETGU7L4ClShXyjPARmXW31VwfIZFdvPnNyqs1+rgZMAs334J3BgRd0bEC2Rz9+dJWjUilpLN858saXVJGwOHlqjje5LWTDeBfZvspjOAK4BjJG0gaTDwX8BVEbGEbNh6n3Rj4MrA/yM7Wd4r6ROSPi1pVeBdsqvepanOl4DmjhveKpX6Nh34b0lrKLu5cSNJ/56KDCFNLUgaSTavXeglsvn5Dk8AA9ONdCsDJ5BdHdfafk84RdIqknYC9gWuSZ/p1cBPlH2N9KPAd4HOvsb4ErBexw2KyRDgtYh4N426fLn0riWdD/xY0ihltpD0IeAW4OOSDpG0cnpto4IbS63nOBkwyylJ+wE7UnDii4jzgWeBH6VV3yS7Qe1FsuHaK8hO2oX+j2wOew7ZNxN+m9ZfkPaZCcwnO7Efndp5nOzq+xdkV6ufJfs64D/JTqqnpfUvkt1s94NUZ8dQ9KuSHqqyy4eSDWk/RjYFcC0wIm07heyGtfbUh+uL9v0pcIKybz4cGxHtZPdHnE92Vf022XGrtf3u9mJq43ngMuCIiPhL2nY0WbxPA38iu8q/oJO67gAeBV6U9EpadyQwWdJCsr+Vq6uI7eep/HTgTbK/l9UiYiGwB/ClFPeLZCNCZZMs6z6KqHXUzczyRtLpwL9FxIRGx2KlKfuK6KURsV4XRc3+xSMDZlZW+t73Fmk4d1vgq8ANjY7LzLqXv01gZp0ZQjY1sC7Z1/3+m2xawMxWIJ4mMDMzyzlPE5iZmeWcpwms31l77bWjubm50WGYmfUrs2fPfiUiin8PA3AyYP1Qc3Mzra2tjQ7DzKxfkfS3cts8TWBmZpZzTgbMzMxyzsmAmZlZzjkZMDMzyzknA2ZmZjnnZMDMzCznnAyYmZnlnJMBMzOznPOPDlm/0/ZcO82TpjY6DDOzXrXgtH16rG6PDJiZmeWckwEzM7OcczJgZmaWc04GzMzMcs7JgJmZWc7lOhmQNENSS6PjqISkIyQdmpYvknRgo2MyM7MVg79aWCNJK0XEkt5qLyJ+3VttmZlZvvSLkQFJzZL+LOk8SY9Kmi5ptcIre0lrS1qQlidKulHSzZLmS/qmpO9KeljS/ZLWKqj+YEn3SnpE0rZp/0GSLpA0K+3z+YJ6r5F0MzC9TKxXSdq74P1Fkg6oNCZJX0/tzpV0naTV0/qTJR1b4fFaIOm/JN0nqVXSWEnTJD0l6YiCct9Lbc2TdEpB36em9h+RND6tP03SY6nsmWndZyU9kPrwR0kfTuuHS7pN0kOSfiPpb5LWTtsOlvSgpDlpW1N6XZTaa5N0TIk+HZ760rp0UXslh8HMzCrUL5KBZBTwvxGxGfAGcEAX5TcHvgxsC/wEWBQRWwH3AYcWlBsUEZ8CjgQuSOt+CNwREdsAuwBnSBqUtm0PTIiIT5dp90qg4wS6CrAr8PsqYro+IraJiC2BPwNf7aKf5TwTEdsDdwMXAQcCnwQmp9j2IDum2wJjgK0l7QzsBTwfEVtGxObAH1Kisj+wWURsAZya2vgT8MnUhyuB49L6k8iO31jgBuAjqc1N0rHZISLGAEuBg1L7IyNi84gYDVxY3JmImBIRLRHR0rT60BoPiZmZldKfpgnmR8SctDwbaO6i/J0RsRBYKKkduDmtbwO2KCh3BUBEzJS0hqRhwB7A5wquxAeSTmjAbRHxWift3gqcI2lVshPrzIh4R1KlMW0u6VRgGDAYmNZFP8u5qaDuwQXtvlvQxz2Ah1O5wWTJwd3AmZJOB26JiLslrQS8C5wvaSpwS9pnPeAqSSOAVYD5af2OZMkDEfEHSa+n9bsCWwOz0vFYDXg5HYcNJf0CmEqZURczM+sZ/SkZWFywvJTsRLKE90c3BnZSflnB+2Us3+8o2i8AAQdExOOFGyRtB7zdWZAR8a6kGcCeZFfBV1QZ00XAfhExV9JEYFxn7XWisO7idlci6+NPI+I3xTtK2hrYG/ippOkRMTlNoewKfAn4JvBp4BfAzyPiJknjgJM7qigTk4DfRcT3S7S5JdkxOwr4IvCVintqZmZ16U/TBKUsILvShGwYvBYdQ/o7Au0R0U52NX600uWrpK2qrPNK4DBgJ6q/sh8CvCBpZbIh9J4yDfiKpMEAkkZKWkfSumTTF5cCZwJjU5mhEfF74Dtkw/oAQ4Hn0vKEgrr/RHZC75iOWDOtvx04UNI6adtakj6a7icYEBHXAScCY3uiw2ZmVlp/Ghko5UzgakmHAHfUWMfrku4F1uD9q9EfA2cD81JCsADYt4o6pwMXAzdFxD+rjOdE4AHgb2RD/EOq3L8iETE9zeHfl3Ket4CDgY+R3SOxDHgP+EaK4f8kDSS7uu+4we9k4BpJzwH3Axuk9acAV6SbD+8CXgAWRsQrkk4ApksakOo/CngHuDCtA/jAyIGZmfUcRRSPkpvVJ90vsTQilkjaHjg33TDYLVYdMSpGTDi7u6ozM+sX6n1qoaTZEVHyt3X6+8iA9U0fIRuxGQD8E/h6g+MxM7NOOBmokaTRwCVFqxdHxHa9GMMNvD803+H4iKj1GwjdIiL+ClR7n0XFRo8cSmsPPtfbzCxvnAzUKCLaeP9GukbFsH8j2zczsxVDf/82gZmZmdXJyYCZmVnOeZrA+p2259ppnjS10WGYmfWoer89UA2PDJiZmeWckwEzM7OcczJgZmaWc04GzMzMcs7JgJmZWc41LBmQNENSyd9IbiRJwyQd2eg4epKk30sa1qC2//W5S1qQnlhoZmYN1C9HBiT15FcihwG9mgz0RH8kNZXbFhF7R8Qb3d2mmZn1T10mA5KaJf1Z0nmSHpU0XdJqRVd4a0takJYnSrpR0s2S5kv6pqTvSnpY0v2S1iqo/mBJ90p6RNK2af9Bki6QNCvt8/mCeq+RdDPZI4LLxXucpDZJcyWdltaVi3UzSQ9KmiNpnqRRwGnARmndGcqckWJsS4/lRdI4SXdJulrSE5JOk3RQqq9N0kap3HBJ16X+zJK0Q1p/sqQpkjoed1yqL6XiQ9LBBet/03Hil/SWpMmSHgB+IOnqgrrGpWO33BW5pENT3XMlXdJZzGViHCzpwtTneZIOSOvPldSa/mZOKbd/wWc+NcXwSMcxLipzeKqvdemi9s6qMzOzKlV6RToK+I+I+Ho6wRzQRfnNyR5UMxB4kuzhOVtJOgs4FDg7lRsUEZ+StDNwQdrvh8AdEfEVZUPZD0r6Yyq/PbBFRLxWqlFJnwH2A7aLiEVFiUcpRwD/ExGXSVoFaAImAZt3PHI3ndzGAFsCawOzJM1M+28JbAK8BjwNnB8R20r6NnA08B3gf4CzIuJPkj4CTEv7AGwN7BgR71Qan6RNgPHADhHxnqRfAQeRJRSDgEci4kfKRhueljQoIt5O+1xVdLw2IzveO0TEKwXHq7OYi50ItEfE6FTnmmn9DyPitZSo3C5pi4iYV6aOvYDnI2KfVMfQ4gIRMQWYAtkjjMvUY2ZmNag0GZgfEXPS8myguYvyd0bEQmChpHbg5rS+DdiioNwVABExU9Ia6eS/B/A5ScemMgPJHokLcFu5RCDZDbgwIhalejsrC3Af8ENJ6wHXR8RfJRWX2RG4IiKWAi9JugvYBngTmBURLwBIeor3RyzagF0KYtq0oN41JA1Jyzd1kgiUi29XsiRiVqpzNeDlVH4pcF3q+xJJfwA+K+laYB/guKL6Pw1cGxGvpH06jlfJmNNnWmw34EsdbyLi9bT4RUmHk/2NjQA2BcolA23AmZJOB26JiLs7OSZmZtbNKk0GFhcsLyU7AS3h/WmGgZ2UX1bwfllRm8VXeAEIOCAiHi/cIGk74O0u4lSJOikXa0RcnobU9wGmSfoa2RV+cZ3lVNLPAcD2xSf9dKLttD9l4hPwu4j4fold3k1JS4ergKPIRi5mlTiZlzteJWMu4wN1SNoAOBbYJiJel3QRH/wb+ZeIeELS1sDewE8lTY+IyRW0bWZm3aCeGwgXkF2hAhxYYx0d8+87kg01t5MNSR+tdLaUtFUV9U0HviJp9bRvx7B3yVglbQg8HRHnADeRjVosBDqu3AFmAuMlNUkaDuwMPFhlTN8saHNMpTuWie924EBJ66Qya0n6aJkqZgBjga9TNEWQ3E52Bf+hjrpqiLm47JrAGmSJTrukDwOf6aKf6wKLIuJS4MwUs5mZ9ZJ6koEzgW9IupdsLr0Wr6f9fw18Na37MbAyME/SI+l9RSLiD2QnzVZJc8iuTjuLdTzwSCq7MXBxRLwK3JNuZDsDuIFseHsucAdwXES8WEUfvwW0pJvrHiO7D6BSpeJ7DDgBmC5pHnAb2TD8B6RRglvITsa3lNj+KPAT4C5Jc4Gf1xDzqcCa6XjNBXaJiLnAw8CjZPeC3NNFP0eT3Rsyh+wehlO7KG9mZt1IEb4Xy/qXVUeMihETzm50GGZmPaq7n1ooaXZElPx9n375OwNmZmbWfXryx3t6jKTRwCVFqxdHxHaNiKdekvYETi9aPT8i9m9EPKVIOgz4dtHqeyLiqEbEY2Zm3cfTBNbvtLS0RGtra6PDMDPrVzxNYGZmZmU5GTAzM8s5JwNmZmY51y9vILR8a3uuneZJUxsdhpmtoLr7K339gUcGzMzMcs7JgJmZWc45GTAzM8s5JwNmZmY552TAaiJpjKT7JD2aHmg0vtExmZlZbfxtAqvVIuDQiPhregTxbEnTIuKNBsdlZmZV8siAdUnSNunqf6CkQZIeBVaJiL8CRMTzwMvA8E7qWCDpFEkPSWqTtHFav5akG1P990vaolc6ZWZm/+JkwLoUEbOAm4BTgZ8Bl0bEIx3bJW0LrAI81UVVr0TEWOBc4Ni07hTg4YjYAvgBcHGpHSUdLqlVUuvSRe119cfMzJbnZMAqNRnYHWghSwgAkDSC7AmSh0XEsi7quD79dzbQnJZ3TPsTEXcAH5I0tHjHiJgSES0R0dK0+gc2m5lZHZwMWKXWAgYDQ4CBAJLWAKYCJ0TE/RXUsTj9dynv36+iEuX8KE0zs17kZMAqNQU4EbgMOF3SKsANwMURcU0d9c4EDgKQNI5sKuHN+kI1M7Nq+NsE1iVJhwJLIuJySU3AvcCXgJ3JhvUnpqITI2JOldWfDFwoaR7ZNxQmdEvQZmZWMScD1qWIuJh0Y19ELAW2S5tK3uxXpo7mguVWYFxafg34fDeFamZmNfA0gZmZWc55ZMC6laQbgA2KVh8fEdMaEY+ZmXXNyYB1q4jYv6fbGD1yKK05fN64mVlP8TSBmZlZzjkZMDMzyzknA2ZmZjnnewas32l7rp3mSVMbHYaZ1WiB7/npczwyYGZmlnNOBszMzHLOyYCZmVnOORkwMzPLOScDZmZmOedkoAqShkk6sosyzZK+XEFdzZIeqaLtGZJaKi3fHSS1SDqnjv0nSlq3O2MyM7Pu52SgOsOATpMBoBnoMhnoDyKiNSK+VUcVEwEnA2ZmfZyTgeqcBmwkaY6kM9LrEUltksYXlNkplTkmjQDcLemh9PpUJQ1JWk3SlZLmSboKWK1g27mSWiU9KumUtG7X9JCgjjK7S7q+k/rfknS6pNmS/ihp2zT68LSkz6Uy4yTdkpZPlnRBQZlvpfXLjXBIOjaVPRBoAS5Lx2I1SVtLuiu1OU3SiLTPtyQ9lvp6ZZl4D099bl26qL2SQ2hmZhVyMlCdScBTETEGuB8YA2wJ7AackU5uk4C7I2JMRJwFvAzsHhFjgfFApcPu3wAWRcQWwE+ArQu2/TAiWoAtgH+XtAVwB7CJpOGpzGHAhZ3UPwiYERFbAwuBU4Hdgf2ByWX22RjYE9gWOEnSyuUqj4hrgVbgoHS8lgC/AA5MbV6Q+gXZMdsq9fWIMvVNiYiWiGhpWn1oJ90yM7Nq+RcIa7cjcEVELAVeknQXsA3wZlG5lYFfShoDLAU+XmH9O5MSh4iYJ2lewbYvSjqc7PMbAWyaylwCHCzpQmB74NBO6v8n8Ie03AYsjoj3JLWRTXWUMjUiFgOLJb0MfLjCvgB8AtgcuE0SQBPwQto2j2wE4UbgxirqNDOzbuBkoHaqsNwxwEtkIwgDgHeraCM+0Ki0AXAssE1EvC7pImBg2nwhcHNq45qIWNJJ3e9FREf9y4DFABGxTFK5v4vFBctLyf5+lrD8CNNAShPwaERsX2LbPmTJz+eAEyVt1kXsZmbWjTxNUJ2FwJC0PBMYL6kpDc3vDDxYVAZgKPBCRCwDDiG7Iq7ETOAgAEmbk00JAKwBvA20S/ow8JmOHSLieeB54ATgomo7V6OXgHUkfUjSqsC+BdsKj8XjwHBJ2wNIWlnSZpIGAOtHxJ3AcWQ3aQ7updjNzAyPDFQlIl6VdE+6Ye5WsuHtuWRX8MdFxIuSXgWWSJpLdkL+FXCdpC8Ad5KdyCtxLnBhmh6YQ5ZoEBFzJT0MPAo8DdxTtN9lwPCIeKz2nlYuTS1MBh4A5gN/Kdh8EfBrSe+QTVscCJwjaSjZ397ZwBPApWmdgLMi4o3eiN3MzDJ6f6TYVgSSfgk8HBG/bXQsPWXVEaNixISzGx2GmdXITy1sDEmz083nH+CRgRWIpNlkIw//r9GxmJlZ/+FkoMEk7QmcXrR6fkTsX21d6St7xfU/AKxatPqQiGirtv6+YvTIobT6ysLMrNs4GWiwiJgGTOvB+rfrqbrNzGzF4G8TmJmZ5ZyTATMzs5zzNIH1O23PtdM8aWqjwzCzIv6WQP/lkQEzM7OcczJgZmaWc04GzMzMcs7JgJmZWc45GTAzM8s5JwMNJGmYpCO7KNMs6csV1NWcHqBUbnuLpHOqiaerOqsl6SJJB6blGZJK/ka2mZn1LicDjTUM6DQZAJqBLpOBrkREa0R8qxviMTOzFYyTgcY6DdhI0hxJZ6TXI5LaJI0vKLNTKnNMulq/W9JD6fWpShqSNE7SLWn5ZEkXpKvzpyV1JAnLxVNBnU2SzkzxzpN0dFr/I0mzUl+mSFIXdVxU0O9jypQ7XFKrpNali9or6bKZmVXIPzrUWJOAzSNijKQDgCOALYG1gVmSZqYyx0bEvgCSVgd2j4h3JY0CrgBqGW7fGNgFGAI8LuncwnhSW81d1HE4sAGwVUQskbRWWv/LiJic6rgE2Be4uUwdY4CREbF5Kj+sVKGImAJMgewRxl13z8zMKuWRgb5jR+CKiFgaES8BdwHblCi3MnCepDbgGmDTGtubGhGLI+IV4GXgwzXUsRvw64hYAhARr6X1u0h6IMX4aWCzTup4GthQ0i8k7QW8WUMcZmZWBycDfUfZofQixwAvkY0gtACr1Nje4oLlpdQ2SiRguat0SQOBXwEHRsRo4DxgYLkKIuJ1sr7MAI4Czq8hDjMzq4OTgcZaSDZMDzATGJ/m0IcDOwMPFpUBGAq8EBHLgEOAph6KpxLTgSMkrQSQpgk6TvyvSBoMHNhZBZLWBgZExHXAicDYqqM2M7O6+J6BBoqIVyXdk76+dyswD5hLdrV9XES8KOlVYImkucBFZFfd10n6AnAn8HYPxvO/XexyPvBxYJ6k94DzIuKXks4D2oAFwKwu6hgJXCipIzH9fs0dMDOzmijC92JZ/7LqiFExYsLZjQ7DzIr4qYV9m6TZEVHyhnNPE5iZmeWcpwlWMJL2BE4vWj0/IvbvS3XWY/TIobT6CsTMrNs4GVjBRMQ0YFpfr9PMzPoOTxOYmZnlnJMBMzOznPM0gfU7bc+10zxpaqPDsBWM74S3PPPIgJmZWc45GTAzM8s5JwNmZmY552TAzMws55wMmJmZ5ZyTgSpIGibpyC7KNEv6cgV1NacHAvVZko6QdGgd+/+gO+MxM7Oe4WSgOsOATpMBoBnoMhnoDyLi1xFxcR1VOBkwM+sHnAxU5zRgI0lzJJ2RXo9IapM0vqDMTqnMMWkE4G5JD6XXpyppSFJTqn+WpHmS/jOtv0rS3gXlLpJ0QLnyZeoeJ+kuSVdLekLSaZIOkvRg6stGqdzJko5NyzMknZ7KPCFpp7R+oqRfFtR9S6r/NGC1dBwuS9sOTvvPkfSbFHNT6kPHcTymTMyHS2qV1Lp0UXslh9DMzCrkZKA6k4CnImIMcD8wBtgS2A04Q9KIVObuiBgTEWcBLwO7R8RYYDxwToVtfRVoj4htgG2Ar0vaALgy1YOkVYBdgd93Ur6cLYFvA6OBQ4CPR8S2wPnA0WX2WSmV+Q5wUmfBR8Qk4J10HA6StEmKe4d0/JYCB5Edw5ERsXlEjAYuLFPflIhoiYiWptWHdta0mZlVyb9AWLsdgSsiYinwkqS7yE7CbxaVWxn4paQxZCfAj1dY/x7AFpIOTO+HAqOAW4FzJK0K7AXMjIh3JJUrP79M/bMi4gUASU8B09P6NmCXMvtcn/47m2w6pBq7AlsDsyQBrEaWKN0MbCjpF8DUgjjMzKyXOBmonSosdwzwEtmV+ADg3SrqPzo9MXD5DdIMYE+yK+0ruipfxuKC5WUF75dR/u+io8zSgjJLWH6EaWCZfQX8LiK+/4EN0pZk/TkK+CLwla6CNzOz7uNpguosBIak5ZnA+DTnPRzYGXiwqAxkV+gvRMQysuH4pgrbmgZ8Q9LKAJI+LmlQ2nYlcBiwE+8/Wriz8j1pATBG0gBJ6wPbFmx7ryMe4HbgQEnrpPjWkvRRSWsDAyLiOuBEYGwvxGxmZgU8MlCFiHhV0j3pK4G3AvOAuUAAx0XEi5JeBZZImgtcBPwKuE7SF4A7gbcrbO58sqH4h5SNq/8D2C9tmw5cDNwUEf+soHxPuodsKqINeAR4qGDbFGCepIfSfQMnANMlDQDeIxsJeAe4MK0D+MDIgZmZ9SxFRKNjMKvKqiNGxYgJZzc6DFvB+KmFtqKTNDsiWkpt8zSBmZlZznmaoMEk7QmcXrR6fkTs3w11jwYuKVq9OCK2q7fuRho9ciitvoozM+s2TgYaLN39X+k3AKqtu43se/xmZmZleZrAzMws55wMmJmZ5ZynCazfaXuuneZJUxsdhq1A/E0CyzuPDJiZmeWckwEzM7OcczJgZmaWc04GzMzMcs7JgJmZWc45GaiBpGGSjuyiTLOkL1dQV3N68FG57S2Szqkmnq7qrJSkyZJ2q3HfLo+RmZn1DU4GajMM6OpE1wx0mQx0JSJaI+Jb3RBPLW3/KCL+WOPuw+iBmMzMrPs5GajNacBGkuZIOiO9HpHUJml8QZmdUplj0tX63ZIeSq9PVdKQpHGSbknLJ0u6QNIMSU9L6kgSloungjonSrpR0s2S5kv6pqTvSnpY0v2S1krlLpJ0YFpeIOmUFHubpI0LYjq2oO5HJDWXiknS9yTNkjRP0ilp3SBJUyXNTfuOpwRJh0tqldS6dFF7JYfOzMwq5B8dqs0kYPOIGCPpAOAIYEtgbWCWpJmpzLERsS+ApNWB3SPiXUmjgCuAko+S7MLGwC7AEOBxSecWxpPaaq6gns2BrYCBwJPA8RGxlaSzgEOBs0vs80pEjE3D/8cCX+uk/uKY9gBGAdsCAm6StDMwHHg+IvZJ5YaWqiwipgBTIHuEcQX9MzOzCnlkoH47AldExNKIeAm4C9imRLmVgfMktQHXAJvW2N7UiFgcEa8ALwMfrrGeOyNiYUT8A2gHbk7r28imOEq5Pv13didlytkjvR4GHiJLakal9naTdLqknSLCl/1mZr3MIwP1U4XljgFeIhtBGAC8W2N7iwuWl1L7Z1hYz7KC98s6qbOjTGG7S1g+qRxYZl8BP42I33xgg7Q1sDfwU0nTI2Jy1+GbmVl38chAbRaSDdMDzATGS2qSNBzYGXiwqAzAUOCFiFgGHAI09VA8vW0BMBZA0lhggzIxTQO+ImlwKjtS0jqS1gUWRcSlwJkddZmZWe/xyEANIuJVSfekr+/dCswD5gIBHBcRL0p6FVgiaS5wEfAr4DpJXwDuBN7uwXj+t7vqrsB1wKGS5gCzgCdKxRQR35O0CXCfJIC3gIOBjwFnSFoGvAd8oxdjNzMzQBG+F8v6l1VHjIoRE85udBi2AvFTCy0PJM2OiJI3rnuawMzMLOc8TdBHSNoTOL1o9fyI2L8v1dkXjB45lFZfyZmZdRsnA31EREwju8muT9dpZmYrHk8TmJmZ5ZyTATMzs5zzNIH1O23PtdM8aWqjw7B+yN8aMCvNIwNmZmY552TAzMws55wMmJmZ5ZyTATMzs5xzMmBmZpZzTgasbpImSPprek3oouwMSS1peYGktvR6TNKpklbtnajNzKyDkwGri6S1gJOA7YBtgZMkrVlFFbtExOi074bAlO6P0szMOuNkwComaRtJ8yQNlDRI0qPAUcBtEfFaRLwO3AbsVW3dEfEWcASwX0owzMysl/hHh6xiETFL0k3AqcBqwKXAe8AzBcWeBUbWWP+bkuYDo4AHCrdJOhw4HKBpjeG1VG9mZmV4ZMCqNRnYHWgBfgaoRJmoo/5S9RERUyKiJSJamlYfWkf1ZmZWzMmAVWstYDAwBBhINhKwfsH29YDna6lY0hCgGXiivhDNzKwaTgasWlOAE4HLgNPJHpG8h6Q1042De1DDY5MlDQZ+BdyY7j0wM7Ne4nsGrGKSDgWWRMTlkpqAe4ExwI+BWanY5Ih4rYpq75QkssT0hlSXmZn1IicDVrGIuBi4OC0vJfs6YYcLKqxjXMFyczeGZ2ZmNfI0gZmZWc55ZMB6hKQbgA2KVh8fEVXfT2BmZj3LyYD1iIjYv6fqHj1yKK2n7dNT1ZuZ5Y6nCczMzHLOyYCZmVnOORkwMzPLOd8zYP1O23PtNE+a2ugwrA9b4HtKzKrikQEzM7OcczJgZmaWc04GzMzMcs7JgJmZWc45GbCaSZog6a/pNaGTcjdImiPpSUntaXmOpB0kzZa0c0HZ6ZK+0Ds9MDMz8LcJrEaS1gJOAlqAAGZLuqnU44c7fo1Q0jjg2IjYt6CeI4HzJY0FDsyKxzU93wMzM+vgkQHrkqRtJM2TNFDSIEmPAkcBt0XEaykBuA3Yq9q6I+IBskchnwz8V6rXzMx6kUcGrEsRMUvSTcCpwGrApcB7wDMFxZ4FRtbYxPdTXWdHxJOlCkg6HDgcoGmN4TU2Y2ZmpXhkwCo1GdidbFrgZ4BKlIka694ZaAc2L1cgIqZEREtEtDStPrTGZszMrBQnA1aptYDBwBBgINlIwPoF29cDnq+2UkmDyJKLTwPDJe1df6hmZlYNJwNWqSnAicBlwOnANGAPSWtKWhPYI62r1o+AqyPiL8CRwFmSBnZTzGZmVgHfM2BdknQosCQiLpfURHbD3xjgx8CsVGxyRLxWZb2bAvsDWwJExBxJ04DjgVO6KXwzM+uCImqd5jVrjFVHjIoRE85udBjWh/lBRWYfJGl2RLSU2uZpAjMzs5zzNIF1K0k3ABsUrT4+Imq5n8DMzHqBkwHrVh2/NtiTRo8cSquHgc3Muo2nCczMzHLOyYCZmVnOORkwMzPLOd8zYP1O23PtNE+a2ugwrAx/rc+s//HIgJmZWc45GTAzM8s5JwNmZmY552TAzMws55wMmJmZ5ZyTAauJpAmS/ppeEzopd4OkOZKelNSeludI+pSkGZJaC8q2SJrRKx0wM7N/8VcLrWqS1gJOAlqAAGZLuikiXi8u2/HzxJLGAcdGxL4F9QCsI+kzEXFrL4RuZmYleGTAOiVpG0nzJA2UNEjSo8BRwG0R8VpKAG4D9qqxiTOAEyqI43BJrZJaly5qr7EpMzMrxcmAdSoiZgE3AacCPwMuBd4Bniko9iwwssYm7gMWS9qlizimRERLRLQ0rT60xqbMzKwUJwNWicnA7mTTAj8DVKJM1FH/qVQwOmBmZj3DyYBVYi1gMDAEGEg2ErB+wfb1gOdrrTwi7kj1frKOGM3MrEZOBqwSU4ATgcuA04FpwB6S1pS0JrBHWlePnwDH1VmHmZnVwN8msE5JOhRYEhGXS2oC7gXGAD8GZqVikyPitXraiYjfS/pHXcGamVlNnAxYpyLiYuDitLwU2K5g8wVV1DMDmFG0blzR+61rDNPMzOrgaQIzM7Oc88iAdRtJNwAbFK0+PiLqvZ9gOaNHDqX1tH26s0ozs1xzMmDdpuPXBs3MrH/xNIGZmVnOORkwMzPLOScDZmZmOedkwMzMLOecDJiZmeWckwEzM7OcczJgZmaWc04GzMzMck4R9TyG3qz3SVoIPN7oOHrJ2sArjQ6il+Slr3npJ7ivfc1HI2J4qQ3+BULrjx6PiJZGB9EbJLW6ryuWvPQT3Nf+xNMEZmZmOedkwMzMLOecDFh/NKXRAfQi93XFk5d+gvvab/gGQjMzs5zzyICZmVnOORkwMzPLOScD1qdI2kvS45KelDSpxHZJOidtnydpbKX79jW19lXS+pLulPRnSY9K+nbvR1+5ej7TtL1J0sOSbum9qGtT59/vMEnXSvpL+my3793oq1NnX49Jf7uPSLpC0sDejb5yFfRzY0n3SVos6dhq9u1TIsIvv/rEC2gCngI2BFYB5gKbFpXZG7gVEPBJ4IFK9+1Lrzr7OgIYm5aHAE/01b7W08+C7d8FLgduaXR/erKvwO+Ar6XlVYBhje5TT/QVGAnMB1ZL768GJja6T3X0cx1gG+AnwLHV7NuXXh4ZsL5kW+DJiHg6Iv4JXAl8vqjM54GLI3M/MEzSiAr37Utq7mtEvBARDwFExELgz2T/wPZF9XymSFoP2Ac4vzeDrlHNfZW0BrAz8FuAiPhnRLzRi7FXq67PlewH71aTtBKwOvB8bwVepS77GREvR8Qs4L1q9+1LnAxYXzISeKbg/bN88CRXrkwl+/Yl9fT1XyQ1A1sBD3R/iN2i3n6eDRwHLOuh+LpTPX3dEPgHcGGaEjlf0qCeDLZONfc1Ip4DzgT+DrwAtEfE9B6MtR71/LvSr/5NcjJgfYlKrCv+7mu5MpXs25fU09dsozQYuA74TkS82Y2xdaea+ylpX+DliJjd/WH1iHo+05WAscC5EbEV8DbQl+eY6/lc1yS7Qt4AWBcYJOngbo6vu9Tz70q/+jfJyYD1Jc8C6xe8X48PDh+WK1PJvn1JPX1F0spkicBlEXF9D8ZZr3r6uQPwOUkLyIZYPy3p0p4LtW71/v0+GxEdIzzXkiUHfVU9fd0NmB8R/4iI94DrgU/1YKz1qOfflX71b5KTAetLZgGjJG0gaRXgS8BNRWVuAg5Ndyp/kmyI8YUK9+1Lau6rJJHNLf85In7eu2FXreZ+RsT3I2K9iGhO+90REX31ChLq6+uLwDOSPpHK7Qo81muRV6+e/1f/DnxS0urpb3lXsvte+qJ6/l3pV/8m+amF1mdExBJJ3wSmkd2Je0FEPCrpiLT918Dvye5SfhJYBBzW2b4N6EZF6ukr2RXzIUCbpDlp3Q8i4ve92IWK1NnPfqUb+no0cFk6cTxNHz4Odf6/+oCka4GHgCXAw/TRn/KtpJ+S/g1oBdYAlkn6Dtm3Bt7sT/8m+eeIzczMcs7TBGZmZjnnZMDMzCznnAyYmZnlnJMBMzOznHMyYGZmlnNOBszMzHLOyYCZmVnO/X88qiWCoamOfAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "preds_xgb = interpret.train_xgb()" ] }, { "cell_type": "markdown", "id": "9e80253e", "metadata": {}, "source": [ "#### Textual Features (focus on customer chats that result in churn)" ] }, { "cell_type": "code", "execution_count": 9, "id": "7f18dcf5", "metadata": {}, "outputs": [], "source": [ "chats, df_sub = interpret.get_chats(df=df, churn=1, speaker='Customer')" ] }, { "cell_type": "code", "execution_count": 10, "id": "0b65267d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[\"Well, I just want to be able to cancel the contract because I don't think that I want to stay. My local provider has been terrible and I really would like to switch. Sure, I can.\",\n", " \"Well, it's the old TelCom billing system for the last 5 years. I don't trust anymore and I think you should change to the newer billing system. I would like to give you a call back number. Okay, I can see why you need the new billing system, but I don't know if I can do that. I would like to know your cancellation policy.\",\n", " \"Well, I've been getting phone calls from a very good friend who's a TelCom agent and I have told him the same thing and the problem has not been resolved. He has offered me a $20/mo deal but that's not good enough for me because I'm getting $20 out of his pocket. Sure. $60 to cancel for nine months with a $20/mo bonus.\"]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "chats[:3]" ] }, { "cell_type": "markdown", "id": "8c73e7dd", "metadata": {}, "source": [ "##### Candidate keywords (POS tagging, lower casing, lemmatization)" ] }, { "cell_type": "code", "execution_count": 11, "id": "3db8f636", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['want', 'able', 'cancel', 'contract', 'think', 'want', 'stay', 'local', 'provider', 'terrible']\n" ] } ], "source": [ "# find candidate keywords\n", "keywords, tokens = interpret.get_keywords(chats)\n", "print(keywords[:10])\n", "\n", "# map keywords to original tokens\n", "keywords_dict = interpret.map_to_orig_tok(keywords, tokens)" ] }, { "cell_type": "markdown", "id": "172eed77", "metadata": {}, "source": [ "##### Relevant keywords (semantic similarity)" ] }, { "cell_type": "code", "execution_count": 12, "id": "07be6aa0", "metadata": {}, "outputs": [], "source": [ "relevant_keywords, simMat = interpret.get_relevant_keywords(\n", " text = chats, \n", " keywords_dict = keywords_dict\n", ")" ] }, { "cell_type": "code", "execution_count": 13, "id": "267a00b0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['voicemail', 'bored', 'spam', 'backlog', 'sick', 'frustrated', 'incompetence', 'overcharge', 'angry', 'disappointed', 'scam', 'lag', 'termination', 'resign', 'nightmare', 'incompetent', 'frustration', 'lagging', 'yesterday', 'friday', 'monday', 'discontinue', 'cheat', 'dead', 'annoyed']\n" ] } ], "source": [ "print(relevant_keywords[:25])" ] }, { "cell_type": "markdown", "id": "3ba0da26", "metadata": {}, "source": [ "##### Impactful kewords (marginal contribution)" ] }, { "cell_type": "code", "execution_count": 14, "id": "8c877937", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 250/250 [11:25<00:00, 2.74s/it]\n" ] } ], "source": [ "# get marginal contribution to prediction for each keyword\n", "marg_contr_df = interpret.perform_ablation(\n", " df = df_sub,\n", " keywords = relevant_keywords,\n", " keywords_dict = keywords_dict\n", ")" ] }, { "cell_type": "markdown", "id": "d43f8aa6", "metadata": {}, "source": [ "##### Create joint metric (semantic similarity + marginal contribution + count)" ] }, { "cell_type": "code", "execution_count": null, "id": "c02d300a", "metadata": {}, "outputs": [], "source": [ "# load from local disc if available\n", "#results_df = pd.read_csv('model/ablation_results.csv')\n", "#results_df = results_df.rename(columns={'Unnamed: 0' : 'keyword'})" ] }, { "cell_type": "code", "execution_count": 15, "id": "2e112e00", "metadata": {}, "outputs": [], "source": [ "results_df = interpret.get_important_keywords(\n", " simMat_df=simMat,\n", " marg_contr_df=marg_contr_df\n", ")" ] }, { "cell_type": "code", "execution_count": 16, "id": "efd90445", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
keywordsimchgcountjoint
0voicemail90.0765530.00669550.628774
1cancel61.187359-0.0813211680.545633
2sick74.789459-0.12724210.538919
3turnover60.896118-0.28663010.533321
4disappointed70.248131-0.09174050.522520
5spam77.940460-0.00002230.506429
6bored78.131271-0.03893210.502213
7unhappy65.601990-0.024782370.493910
8frustrated73.496033-0.00602350.486930
9mistake66.247879-0.09330930.470609
10late65.971649-0.003873180.458023
11error63.123695-0.06560970.448412
12faulty55.486092-0.23981710.448232
13angry70.865860-0.00296730.444018
14backlog74.8085480.00002010.442185
15customer52.072552-0.0069854800.439737
16lag69.912178-0.00465030.436584
17overpay65.573105-0.09384610.429261
18disconnect64.002014-0.010485110.429104
19incompetence73.107452-0.00075410.427229
\n", "
" ], "text/plain": [ " keyword sim chg count joint\n", "0 voicemail 90.076553 0.006695 5 0.628774\n", "1 cancel 61.187359 -0.081321 168 0.545633\n", "2 sick 74.789459 -0.127242 1 0.538919\n", "3 turnover 60.896118 -0.286630 1 0.533321\n", "4 disappointed 70.248131 -0.091740 5 0.522520\n", "5 spam 77.940460 -0.000022 3 0.506429\n", "6 bored 78.131271 -0.038932 1 0.502213\n", "7 unhappy 65.601990 -0.024782 37 0.493910\n", "8 frustrated 73.496033 -0.006023 5 0.486930\n", "9 mistake 66.247879 -0.093309 3 0.470609\n", "10 late 65.971649 -0.003873 18 0.458023\n", "11 error 63.123695 -0.065609 7 0.448412\n", "12 faulty 55.486092 -0.239817 1 0.448232\n", "13 angry 70.865860 -0.002967 3 0.444018\n", "14 backlog 74.808548 0.000020 1 0.442185\n", "15 customer 52.072552 -0.006985 480 0.439737\n", "16 lag 69.912178 -0.004650 3 0.436584\n", "17 overpay 65.573105 -0.093846 1 0.429261\n", "18 disconnect 64.002014 -0.010485 11 0.429104\n", "19 incompetence 73.107452 -0.000754 1 0.427229" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results_df.head(20)" ] }, { "cell_type": "markdown", "id": "7036dc83", "metadata": {}, "source": [ "##### Context of keywords" ] }, { "cell_type": "code", "execution_count": 17, "id": "5c0f0d2b", "metadata": {}, "outputs": [], "source": [ "keyword_of_interest = 'spam'" ] }, { "cell_type": "code", "execution_count": 18, "id": "041befbf", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Basically, I'm getting a lot of spam calls every day from a guy named Michael who's calling from a really weird number.\n", "TelCom started to flood me with emails and phone calls, spamming me with thousands of phony invoices.\n", "I just got some spam messages last night, and today it's been getting a lot of texts that I \"don't have my SIM card\" and \"I need my SIM card.\n" ] } ], "source": [ "interpret.obtain_context(\n", " chats_list = chats,\n", " keyword = keyword_of_interest\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "a294f494", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "conda_pytorch_p36", "language": "python", "name": "conda_pytorch_p36" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.13" } }, "nbformat": 4, "nbformat_minor": 5 }