{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Players encounters anomalies with SageMaker Random Cut Forests\n", "\n", "***Unsupervised anomaly detection on timeseries data a Random Cut Forest algorithm.***\n", "\n", "---\n", "\n", "1. [Introduction](#Introduction)\n", "1. [Setup](#Setup)\n", "1. [Training](#Training)\n", "1. [Inference](#Inference)\n", "1. [Epilogue](#Epilogue)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction\n", "***\n", "\n", "Amazon SageMaker Random Cut Forest (RCF) is an algorithm designed to detect anomalous data points within a dataset. \n", "In this example we are going to detect players moves during encounters in game sessions events. \n", "\n", "In this notebook, we will use the SageMaker RCF algorithm to train an RCF model on a banchmark conducted based on tpy game which records players moves over the course of six weeks gameplay. We will then use this model to predict anomalous events by emitting an \"anomaly score\" for each data point. The main goals of this notebook are,\n", "\n", "* to learn how to obtain, transform, and store data for use in Amazon SageMaker;\n", "* to create an AWS SageMaker training job on a data set to produce an RCF model,\n", "* use the RCF model to perform inference with an Amazon SageMaker endpoint.\n", "\n", "More about RCF please check out the [SageMaker RCF Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/randomcutforest.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Setup\n", "\n", "***\n", "\n", "*This notebook was created and tested on an ml.m4.xlarge notebook instance.*\n", "\n", "Our first step is to setup our AWS credentials so that AWS SageMaker can store and access training data and model artifacts. We also need some data to inspect and to train upon." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Select Amazon S3 Bucket\n", "\n", "We first need to specify the locations where we will store our training data and trained model artifacts. ***This is the only cell of this notebook that you will need to edit.*** In particular, we need the following data:\n", "\n", "* `bucket` - An S3 bucket accessible by this account.\n", "* `prefix` - The location in the bucket where this notebook's input and output data will be stored. (The default value is sufficient.)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "isConfigCell": true, "tags": [ "parameters" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training input/output will be stored in: s3://percona2020-player-events/sagemaker/rcf-benchmarks\n" ] } ], "source": [ "import boto3\n", "import botocore\n", "import sagemaker\n", "import sys\n", "\n", "\n", "bucket = 'percona2020-player-events' # <--- specify a bucket you have access to\n", "prefix = 'sagemaker/rcf-benchmarks'\n", "execution_role = sagemaker.get_execution_role()\n", "\n", "\n", "# check if the bucket exists\n", "try:\n", " boto3.Session().client('s3').head_bucket(Bucket=bucket)\n", "except botocore.exceptions.ParamValidationError as e:\n", " print('Hey! You either forgot to specify your S3 bucket'\n", " ' or you gave your bucket an invalid name!')\n", "except botocore.exceptions.ClientError as e:\n", " if e.response['Error']['Code'] == '403':\n", " print(\"Hey! You don't have permission to access the bucket, {}.\".format(bucket))\n", " elif e.response['Error']['Code'] == '404':\n", " print(\"Hey! Your bucket, {}, doesn't exist!\".format(bucket))\n", " else:\n", " raise\n", "else:\n", " print('Training input/output will be stored in: s3://{}/{}'.format(bucket, prefix))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We curated the csv file using curate scripts `curate.sh` and `curate.py`. Load the results file and observe before training and exported the data to `players_cheat_model/player_encounters-full-curated.csv`" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Index(['playerx', 'playerz', 'quadrant', 'sector', 'event'], dtype='object')\n", "CPU times: user 1min 24s, sys: 18.5 s, total: 1min 42s\n", "Wall time: 1min 45s\n" ] } ], "source": [ "%%time\n", "\n", "import pandas as pd\n", "import urllib.request\n", "import boto3\n", "\n", "data_filename = 'player_encounters-full-curated.csv'\n", "data_objectname = 'players_cheat_model/player_encounters-full-curated.csv'\n", "data_source = 'percona2020-player-events'\n", "\n", "\n", "s3 = boto3.client('s3')\n", "s3.download_file(data_source, data_objectname, data_filename)\n", "\n", "player_data = pd.read_csv(data_filename, delimiter=',')\n", "print(player_data.columns)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before training any models it is important to inspect our data, first. Perhaps there are some underlying patterns or structures that we could provide as \"hints\" to the model or maybe there is some noise that we could pre-process away. The raw data looks like this:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
playerxplayerzquadrantsectorevent
0199.082749.08284Quadrant 3Sector 1 -2TraverseSector
1178.2321146.22680Quadrant 3Sector 0 0TraverseSector
2186.6547-115.94910Quadrant 1Sector 0 0EventResponseWormholeAnomaly
3178.9562-116.42950Quadrant 1Sector 0 0EventResponseWormholeAnomaly
4185.8585-103.28220Quadrant 1Sector 0 0EventResponseWormholeAnomaly
\n", "
" ], "text/plain": [ " playerx playerz quadrant sector event\n", "0 199.0827 49.08284 Quadrant 3 Sector 1 -2 TraverseSector\n", "1 178.2321 146.22680 Quadrant 3 Sector 0 0 TraverseSector\n", "2 186.6547 -115.94910 Quadrant 1 Sector 0 0 EventResponseWormholeAnomaly\n", "3 178.9562 -116.42950 Quadrant 1 Sector 0 0 EventResponseWormholeAnomaly\n", "4 185.8585 -103.28220 Quadrant 1 Sector 0 0 EventResponseWormholeAnomaly" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "player_data.head()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
playerxplayerzquadrantsectoreventquadrant_encodedsector_encodedevent_encoded
0199.082749.08284Quadrant 3Sector 1 -2TraverseSector3257
1178.2321146.22680Quadrant 3Sector 0 0TraverseSector3217
2186.6547-115.94910Quadrant 1Sector 0 0EventResponseWormholeAnomaly1213
3178.9562-116.42950Quadrant 1Sector 0 0EventResponseWormholeAnomaly1213
4185.8585-103.28220Quadrant 1Sector 0 0EventResponseWormholeAnomaly1213
\n", "
" ], "text/plain": [ " playerx playerz quadrant sector event \\\n", "0 199.0827 49.08284 Quadrant 3 Sector 1 -2 TraverseSector \n", "1 178.2321 146.22680 Quadrant 3 Sector 0 0 TraverseSector \n", "2 186.6547 -115.94910 Quadrant 1 Sector 0 0 EventResponseWormholeAnomaly \n", "3 178.9562 -116.42950 Quadrant 1 Sector 0 0 EventResponseWormholeAnomaly \n", "4 185.8585 -103.28220 Quadrant 1 Sector 0 0 EventResponseWormholeAnomaly \n", "\n", " quadrant_encoded sector_encoded event_encoded \n", "0 3 25 7 \n", "1 3 21 7 \n", "2 1 21 3 \n", "3 1 21 3 \n", "4 1 21 3 " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import csv\n", "import sys\n", "import pandas as pd\n", "pd.set_option(\"display.max_rows\", None, \"display.max_columns\", None)\n", "from sklearn.preprocessing import OneHotEncoder\n", "from sklearn.preprocessing import LabelEncoder\n", "\n", "label_encoder = LabelEncoder()\n", "integer_quadrant_encoded = label_encoder.fit_transform(player_data.quadrant)\n", "player_data[\"quadrant_encoded\"]=integer_quadrant_encoded\n", "\n", "integer_sector_encoded = label_encoder.fit_transform(player_data.sector)\n", "player_data[\"sector_encoded\"]=integer_sector_encoded\n", "\n", "integer_event_encoded = label_encoder.fit_transform(player_data.event)\n", "player_data[\"event_encoded\"]=integer_event_encoded\n", "\n", "player_data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The folowing printouts help us to determine the encoding strategy. We apply the same strategy in the RDS side as we use it for calling the model using aurora_ml. We also use the `.size` method to determine we have only numbers in the data set. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "quadrant_encoded\n", "0 3837916\n", "1 28939141\n", "2 29053511\n", "3 3495932\n", "dtype: int64\n", "event_encoded\n", "0 7852\n", "1 9643885\n", "2 5581043\n", "3 8987499\n", "4 116480\n", "5 3499828\n", "6 3437154\n", "7 34052759\n", "dtype: int64\n" ] } ], "source": [ "quadrant = player_data.groupby('quadrant_encoded').size()\n", "print(quadrant)\n", "sector = player_data.groupby('sector_encoded').size()\n", "print(sector)\n", "event = player_data.groupby('event_encoded').size()\n", "print(event)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
playerxplayerzquadrant_encodedsector_encodedevent_encoded
0199.082749.082843257
1178.2321146.226803217
2186.6547-115.949101213
3178.9562-116.429501213
4185.8585-103.282201213
\n", "
" ], "text/plain": [ " playerx playerz quadrant_encoded sector_encoded event_encoded\n", "0 199.0827 49.08284 3 25 7\n", "1 178.2321 146.22680 3 21 7\n", "2 186.6547 -115.94910 1 21 3\n", "3 178.9562 -116.42950 1 21 3\n", "4 185.8585 -103.28220 1 21 3" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "player_data=player_data.drop('event',axis=1)\n", "player_data=player_data.drop('quadrant',axis=1)\n", "player_data=player_data.drop('sector',axis=1)\n", "player_data.head()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Index(['playerx', 'playerz', 'quadrant_encoded', 'sector_encoded',\n", " 'event_encoded'],\n", " dtype='object')\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
playerxplayerzquadrant_encodedsector_encodedevent_encoded
count6.532650e+076.532650e+076.532650e+076.532650e+076.532650e+07
mean-4.124771e-01-5.616098e-011.493023e+001.505362e+014.970812e+00
std1.636912e+021.637119e+026.888253e-011.016202e+012.413060e+00
min-2.927359e+02-2.928427e+020.000000e+000.000000e+000.000000e+00
25%-1.493577e+02-1.506637e+021.000000e+005.000000e+003.000000e+00
50%-1.826301e+01-1.612999e+011.000000e+001.800000e+017.000000e+00
75%1.501168e+021.496264e+022.000000e+002.400000e+017.000000e+00
max2.927549e+022.927474e+023.000000e+003.500000e+017.000000e+00
\n", "
" ], "text/plain": [ " playerx playerz quadrant_encoded sector_encoded \\\n", "count 6.532650e+07 6.532650e+07 6.532650e+07 6.532650e+07 \n", "mean -4.124771e-01 -5.616098e-01 1.493023e+00 1.505362e+01 \n", "std 1.636912e+02 1.637119e+02 6.888253e-01 1.016202e+01 \n", "min -2.927359e+02 -2.928427e+02 0.000000e+00 0.000000e+00 \n", "25% -1.493577e+02 -1.506637e+02 1.000000e+00 5.000000e+00 \n", "50% -1.826301e+01 -1.612999e+01 1.000000e+00 1.800000e+01 \n", "75% 1.501168e+02 1.496264e+02 2.000000e+00 2.400000e+01 \n", "max 2.927549e+02 2.927474e+02 3.000000e+00 3.500000e+01 \n", "\n", " event_encoded \n", "count 6.532650e+07 \n", "mean 4.970812e+00 \n", "std 2.413060e+00 \n", "min 0.000000e+00 \n", "25% 3.000000e+00 \n", "50% 7.000000e+00 \n", "75% 7.000000e+00 \n", "max 7.000000e+00 " ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(player_data.columns)\n", "player_data[['playerx','playerz','quadrant_encoded','sector_encoded','event_encoded']].describe()" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
quadrantplayerz
0199.082749.08284
1178.2321146.22680
2186.6547-115.94910
3178.9562-116.42950
4185.8585-103.28220
\n", "
" ], "text/plain": [ " quadrant playerz\n", "0 199.0827 49.08284\n", "1 178.2321 146.22680\n", "2 186.6547 -115.94910\n", "3 178.9562 -116.42950\n", "4 185.8585 -103.28220" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "features_of_players = pd.DataFrame({'quadrant': player_data['playerx'], 'playerz': player_data['playerz']})\n", "\n", "features_of_players.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's take a look at a plot of the data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Lets look at perceiving patterns. " ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD8CAYAAAB6paOMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzsvXm4HFd5Jv6eXqq6u7r77pukK8mS5UW2LLC8gdlCIGAT4gA2ExKWgSQkQ5hABvgFmJmE5EkmzENIMjwkJIQwkJAQ9mDCYoiHzTaysWxJXpG1WHfX7dv7vtX5/XHqVFd3197d996S+n0ePa17u7q6b1f3+c77ft/3foRSiiGGGGKIIS5d+Lb6BQwxxBBDDLG1GAaCIYYYYohLHMNAMMQQQwxxiWMYCIYYYoghLnEMA8EQQwwxxCWOYSAYYoghhrjEMQwEQwwxxBCXOIaBYIghhhjiEscwEAwxxBBDXOIIbPULsIPJyUm6d+/erX4ZQwwxxBCewrFjxzYopVNWx3kiEOzduxcPP/zwVr+MIYYYYghPgRBy3s5xQ2loiCGGGOISxzAQDDHEEENc4hgGgiGGGGKISxzDQDDEEEMMcYljGAiGGGKIIS5xDAPBEEMMMcQljmEgGGKIIYa4xDEMBEMMoeAnZ5I4vpjZ6pcxxBCbDk80lHkFx86n8M5/eRQNmcJHAB8h8BECov6f3f7aLXvw6y+4bKtfriWqjSYCPh/8PrLVL2XgOJ8s4q2feQg37BnH537j5q1+OUNcxFhIlvBr/3AU5ZqsWScAQgh8vta68fKDM/jg7VdvymsaBoI+4ntPrmOjUMWdR+ZBKYVMKWQKyJSCKrf3PbOBe5+64IlAcNff/gTzYxH89a9dv9UvZaCQZYrf/8pJVOoy8pX6Vr+cIS5y/PCZBBZTZdx5ZBeCfgJZBprKesHXiWPn07jnibVhIPAiTixmcHAujj977SHDY9786YeQLQ94saEU+N4fAId/BZi5xtUpZJni6dU8Ti5l8UtPrOEV18z2+UVuH/zLQws4ejaFWCiAfLWx1S9niIscJxYzmJAEfOTO60CIPtv+4Ncew3efuLBpr2mYI+gTZJniseUsrts1anqcJPhRHPRiU04DD3wMePpbrk+xUayi1pRBCPChu59A4SJdIJczZXz420/j1ssncPu1cyhULs6/c4jtg5NLGVy3a8QwCACbtE5oMAwEfcLZjQIK1Qau2zViepwkBlAa9AWuZNltLe/6FCuZCgDgHS/Zj7VcBR/97s/68cq2FSil+OBXH4NMKT782usQCwUu2oA3xPZAodrAM+sF6w2jGEC53kRTppvyuoaBoE84scgW3+fMWzOCgS821ZxyW3B9ipVMGQDwqkM78Mab9+CzDzyLk0sXV0XNVx5Zxg9PJfD7r7wK8+MRREMBlGqb9+Ub4tLD48tZUGpnnWCqfam2ORuTYSDoE04sZSAJfuybipoeJ4lssaF0gItNRQkEtd4Dwc7RMN73yisxERXxwa89hkZT7scr3HKs5yr44288gRv3juFNt+wBAERF9uUrbtKXb4hLD3wzZUc5AIBSrTnw1wQMA0HfcGIpi0O7RixLLSUxgIZMUW0McEHljKBWdH2K5UwZkuBHPBxAPBTEh159DR5fzuEzDzzbn9e4haCU4r//2+OoNmT879ddB59yzXggGOYJhhgUTixmsWssjImoaHqcJPoBYNOkymEg6ANqDRlPreRw2EL3A5g0BAw40nNGUO0lR1DGjtGwmtC6/dAsXnrVNP7ie6ewrLAFr+LfT67ie09ewHt+4Yo2BhcNKYFgmCcYYkA4sZSxuU4ojKA6ZASewdNrOdSasmUCCAAiXH4Y5GJT7Yc0VMGO0bD6MyEEf/RL14BS4A+//vhgpa0BIlmo4g/vfgKH50fx6y/Y13YfZwT5ISMYYgBIFqpYSpdxeN5cFgKAyJAReA8nllii2M4F3hQduk/JYm0gAID58Qh+7+UH8B9PreOeTaxx7ic+9I0nka/U8ZE7r+uS8WJDRjDEAHFSWSfsbBij4jBZ7DnwBpGdHQunHiKKNFQcJOWr9JYjqNSbSBZr2Dka6rrvrbdehqvn4vjQ3U94rgv3nifW8I0TK/jdlx7AFTOxrvulzWBrQ1yyOLGUASHAtTttMAKBbxiH0pBncHIpg8Pzo6YNIhzRTZWG3OUIeMVQJyMAgKDfhz977SFcyFfw0e+ecv0SNxvZUh3/498ex9Vzcfz2S/brHjNMFg8xSJxYzODAdFT9nJlhU9YJDYaBoEe0GkSsozzQivQDpXwVjTTkQsvnzWR6gQBgNdBvvmUPPvuTZ3HCI26df/ejM9goVPGRO69D0K//sY+JQQAY2kz0Gd95fBVnEu5lyosBlFKcXLJ2HuDgOYJhIPAIeIOInUoAQLPrHKQ0xBkBbQKNiuOHa3sIjPCeV1yJ6ZiID3z1MaSKNVcvc7OQq9TxTz85j9uvnTOl5WrJ3pAR9BW/94UT+OxFUHbcC5bSZSSLNRy2aCTj4FVDA5WQNRgGgh5ht0GEg0f6TWEEgKs8wXKmDEKAmXh3joAjHgrij++4Fk+t5XDLn92L//aF43hkIb0tq4n+6Sfnka828F8MJCGOgN+HcNA/bCjrI0q1Bsr15iWfgOeJ4sM21wm/jyAU9G1asnjoPtoj7DaIcLQYwSbkCADWSyBNOnr4SqaM6ZgIIWC+T3jFNbO4590vwueOnsdXH1nGVx9dxjU74njTLXvwS8/ZocpgW4lyrYlP33cOL7lyylaSThIDw/LRPoKzxc2qh9+uOLmUgeD34arZuO3HRMXN877qmREQQuYJId8nhDxJCHmCEPIu5ffjhJDvEUKeUW7HlN8TQsjHCCGnCSEnCSGeNrs/oSSK7UIM+OAjA/5iVHJASFn0XPQSrGYrhvmBTlwxE8Mf33Etjn7w5/Env3wtmjLF+7/6GG7+X/fiQ3c/gdPrW6sNf+GnC0gWa3jHSy63dfzQeK6/SBdZZdmlzrKOL2Zw9Y645eZKi4gQ2DSLiX5s2RoA3kMpfYQQEgNwjBDyPQD/GcC9lNIPE0LeD+D9AH4fwG0ADij/bgbwCeV2oPjekxeQKlZNj9k7IeHmfRO2z8kbRN78vD22H0MIgTToSF/NASPzzIXURS/BSqaMq3fY37kAbPfyxlv24Ndu3o1j59P4p6Pn8c8PnsdnHngWrz68Ax+967CjL0E/UGvI+OSPzuLGvWO46bJxW4+JigEUPFYWu52RVL5zm7Wg9YoHTm9gMV0yPWY6FsLPXTVt+5xNmeLx5Sxed2SXo9cy8HVCg54DAaV0FcCq8v88IeQpADsB3AHgJcphnwXwA7BAcAeAf6RMTD5KCBklhMwp59FH4mkgcQqYusLVazybKOA3//Fhy+OCfoKjH/h52zKPkwYRLSQhMDjtT24yFhDfASSecswIKKVYzpTxsoMzrp6eEIIb9o7jhr3j+J+/eBCfuf9ZfPz7p1FrNPHxX73esGJnEPj68WWsZCv409cYDwrqxGbS8UsB6RKThgZe/VLcAL7+TuC2DwNje92dotrAmz79kC332W+/64W4es7eZulMooBirWm7oIRDEvzezBEQQvYCeC6ABwHMaBb3NQB8ZdkJYFHzsCXld8aBoF4GyinXr+vUBbYY/t+33ogrdRqJAGAxVcJ/+uRR/NvxFdtjJE8sZeAjwCEb2rMWkugfXDUAzw/Ed7Bbh4EgVayh2pCxY8Q4UWwXk1ER733FlZiKifjDu5/Au/71UXzsV56LwCYEg6ZM8YkfnsHVc3G85Mop24+TxIDnvZS2E5IFJUcwaEbwwMeAU98Grn2d60BwbqOIpkzxv15zyPAzU6w28KqP3YcvPbyEP3j1QVvn5SXWdpwHtJDEADKDnmaooG+BgBASBfAVAO+mlOa0zVWUUkoIcVROQgh5O4C3A8CROV9PdglnN9hjb9w7btjMsWM0jMO7RvClhxfxtlv32moOO7GYweXTUbUj1S4kMTA4zZRXDMV3sluH75tVD4EbvOX5e1FvyviTbz4Fv+8E/vL1hwceDL77xBrOJor4+K8+19a15GA5gotEGsqvAbGtHTHKGcFAd7alFPDTf2D/72EYE+91OLJnzPTz/7KD0/i348t4/21X2ZI7Ty5lERUD2DdpblHfCUn0q6Xcg0Zfvo2EkCBYEPhnSulXlV9fIITMKffPAVhXfr8MYF7z8F3K79pAKf0kpfQGSukNAHq6wGcTRczERcuOvjtvmMfTa3k8sZIzPU55fTi5lHVM9wAmDQ2MKquMYI7dOmQEyyZdxb3gN164D++/7Sp848QK3vflkwMd/kIpxV//4DQum5Rw27Vzjh7LcgQXgTS0fAz46JXAhSe39GXwqqGB1sM/+Letz3kvG8ZEEYQAeyYipsfddWQeqWIN/+9pe35bJ5YyOLRzRLU7t4uBrhMd6EfVEAHwDwCeopT+heauuwG8Rfn/WwB8XfP7NyvVQ7cAyJrmBzh6uMBnEgVb0fiXrtsBIeDDFx9etDyWN4hc56BiiGOg0hBnBDFFGnLMCAYTCADgt1+8H+/9hSvwtUeX8ftfOQl5QMHgx89s4PHlHH77xfss50N0InqxVA2lzim3Z7b2ZSiBYGBjFys5FgiufBX7uQfH3TOJAnaNhREK+k2Pe+GBSUzHRHzp4SXLc1YbTTy1mnNUWcjBlAPvNJTdCuBNAF5KCDmu/LsdwIcBvJwQ8gyAlyk/A8C3AJwFcBrA3wN4h61ncXmBKaU4myhi35RkeexIJIhXXDOLrx9fQaVufgGcNoho0as0dPeJFXzwa4/pL6ScEUgTgF9w/L6tZMoIBX0YiwRdvz4zvPOlB/Cunz+ALx9bwn//N4O/oUf89fdPY24khNc811mVBsAYQb1JUW1sYZXLxjPA9/7AlT2IilKS3RbWzY8bMHj5KMCCQd/x00+x6rgXvw8Qoj0zAjsbxoDfh9devws/OJXAet68c/+p1TzqTepynWAD7N02aX7u6Hnbx/YcCCil91FKCaX0Okrpc5R/36KUJimlP08pPUApfRmlNKUcTymlv0Mp3U8pPUQptS7nAVxf4FSxhmy5jv0WIyQ57jqyC9lyHf/xlDntO+GiQYQjIgR6YgTffWIN//LgAj7xQ53dHmcE4gj7YjgNBNn2gTSDwLtfdgC/83P78fmHFvEHd/d3tsGx8yk8eC6F33jhPlflqtvCeO7JrwP3/x+m8btFcaP9douQ1JRsl/rNtGpF4CcfBy5/ObDjucrn3Z2ELMsU5zaK9teJG3ahKVN87ZEuVbsNqvOAC0YQEdg0w5rL8bBWZbBaeMNigvhcX+AzCWaxYIcRAMCtl09ibiRkSftOuGgQ4Ygqkd4teOfrR7/7Mzx0rqOaijOCUBwQne+QljMVW3bavYAQgvf+wpX4rRftw+eOLuCPvvFk34LB33z/DMYiQbzhpnnrg3WwKZ3fVqgoRn58V+8G/LHFLWYEpbo6la/vMsexz7K/80XvZT+7+LxzrOYqKNebtteJ/VNRHNkzhi8dWzL97J5YzGIyKrqqwms5kLp735x0yHsnELi8wGeVSgC7kd7vI3jd9bvw42cSWMvq0z7eIPIcF3QPYJG+F820UG3g8Pwodo9H8Luff7Td9K3CJCuIcXeMIFPGjpHBBgKABYP333YV3nbrZfjMA8/ie0/2PujmyZUc7n16HW+99TLX9hZ8XOWW2kyU0+y21MNufhtIQ02ZIl2qYX6cJV/7mvisV1jJ6N4XArtvYb9z8Xnn4OuE3UAAMPXg9HoBj5o48LLRlCOuGHZrdom7980Jq/VIIPC7v8AbRQgBn6Pk551HdkGmwFce0WcFvEHEaSMZR6/Th/KVOubiIXz8V69HqljDe754vKW1V3MsNxAMOf5iVBtNJPLVgSSK9UAIwe+9/AAA4NmkuyE6Wnzih2cgCX685Xl7XZ8jth0YQVlZWHqRdVRGsHXSULZcB6XArjH2eeprL8Hxfwbyqy02AABirIcNI/v8XW5zwwgAr7puDqGgz1A9yFfqOJMouEoUA71PM3QyOMobgcDXGyPYNyk5qh7ZOynhpsvG8WUD2ue2QYSj5UDq7otRqDQQCwVw7c4R/M9fvBrf/1kCf//js+zOSo6xAcAxVeYMaIfOZLJBISoGEPARpEu91e4/u1HEN0+u4I3P24ORHhLdmzGlbCVTRq1hovuqjMB9E6X62C2Uhrily64xhRH0q5egWQfu+ytg143AZS9u/V6IMpNFFzibKCAqBjAVs+cqAACxUBC3XzuHfz+xgrLOd/kxxaLerjNxJyI9SkNONjPeCATE31OOwAnd47jryC6c2yji4fPprvtOLGVcNYhw9KpD5ysNVcJ44y17cPuhWXzknp/h2Pk0YwQhJRA4ZATLNuYQ9BuEEIxJAtI9zjT4ux+dQcDvs90VboTogOcW1xoyXv4XP8TnH1owPqgv0pDy2ELC/Tl6REqpGFIZQb9Kpk9+EcguAC96H6CVXET3yWK+TjiVcO68YRfy1QbueaI7sd+qLHTLCHqThi6+HIFLRlBryFhIlVwt2LcfmkNE8ONLOj0FJ5eyrhpEONQpZS6+GLJMUag1EAuxXS8hBB9+3XWYGw3hdz//KOqlTIsROCynG0RXsR2MRYJqB6pb3H18Bb90eAemY72xGS4NDSpHkCnXUKw1zTtGe5WGKGXSEPED1SzQMDdbHBR47qqvjEBuAj/+KDB7CDjwC+339SQNMeXAKW65bALz42F86ZjeOpHB7vEIxiTB1WvqdZrhxRcIiN8V5VtIldCUqStGIIkBvOrQHL55crXtQvTSINI6tzIJy0WkL9YaoLS1YAFsSMzH33A91vMVnF9eAxUVPyXRGSPgi9NsH3yGnGAsIrTVmztFudZEsda0XRBghkEzglyZnTdr5iGjMgKXVUPVHCA3gAllEE+xB1bQQzUXDwTz45wR9OE9feJrrEmukw0ArpPFpVoDK9mKq8+Pz0dw5/XzeOBMEoup9nLNE4tZ17IQ0Ps0w4svR0B8ri6w04qhTtx1wzyKtSa+9ViL9vXSIMIh9RDp+QIVC7VXxRyeH8X7b7sazXIWC0XlPv7FsPllXsmUMRkVLTsr+42xiNATI0gpjx2Xem+CCwf98JHB5QhyypczZ/QlrZeBhsIW3AYC/ripq9it28qhSg748wPAU//u6uH8mrYYQY/SkCwzNjB5JXDVq7vvF2NsNGvT2bU7t8FLzN2tE687wny9tMUliXwVy5kyntPDhpFXDblZJyilF2mOwAXlc9pD0Ikb945h70SkTR7qpUGEQ+ohR8DpXjTUXR75tlv3YjJYw7ELTZbQFqNsZ2hTGljOlLFzExPFHGNSb4GA5xfGIu4ouBZ8XsSgpKGcwgQ4M+hCWVOK6FYa4oni6at7O0/6WcYmfvopVw9PFmqQBD/ioQD8PtK78dypbwPrTwIvfA+TizshKAu5wzxBr+vErrEInr9/Al8+tqRW77VG2G7NOlGuN+GkOt0bgcDnc7Sz5TibKGAqJqp6ulMQQnDnkV148FwKC0lG+44vZlw3iHBIPVQN8QVK728ihGDcX0YjGMM7P/8IykTR+m2yqZVMedPzAwDPEdRdN5Uli5wR9B4IACa7DUwaUq6fISPgslBoxD0j4As/ZwRuK4d4Z/O5HwJ5530e6VINY5IAQggiQo/+WpQCP/oIs5i+9nX6x4hKIHC4aTybKIAQ4DIXOQKOu47MYyldxtFz7JqdWMrCR4Brdzp3HuAQAz4WQF28b043Mt4IBMQPgDoexH52o4j9LqM8x+uO7AIhwJeVZBBzHHXXIMLRS4ki1/10nVRlGaRWwAuu3Y+ldBn3LygNcTYCAaUUKxn7Iyr7iXFJQFOm6iLpFCoj6FMgiIYG50DaYgQGgYB3FU9czgKB7MJeoF/SUEEJBFQGHv+K44cnizU1OPc8jOnMvcDKo8AL/hvgN2gWVBmB00BQxM5Ra7M5M7zimlnExAC+rPQUnFzK4IqZWE9zuwkhkAR/T8qBXXgkECgv0+EFPpMouNb9OOZGwnjhgSl8+dgSsuXeGkQ4IkFeFuY80hvlCAAolJhix8w0DkxH8XSKN5lZv2+ZUh3lenNLAsGoIulkXMpDPCk50a9AMEBGwJPEhkGPM4KJAwBttgKDE/BAMDoPBCVDaWgxVcKpCyYyCmcE0weBx77o+GWkNYEgIvp7yxE88HE2Y+PwG4yP4UUSDhlBP9aJsODHq5+zA996fBW5Sh0nFjM9JYo5JNFdAHWSKAa8Egh8SqR2cIFTxRoypbqrkrBO3HVkF1ayFfzdD8/01CDCEfD7IAZ8rsrpChWTQKAazsVx/e4xPL6h7CZtBNBWD8Hm5wh4kjflspcgXarBR1j1VD8QDQUHKA21GIGuFKYGgsvZrZumslKSdZcLUSA6ZSgN/ck3n8S7/vW48Xnya0B4HHjOr7Hd+MZpRy8jVaxhPKJhBG7f03IGePbHwHWvBwImwd5FjoBSZjbXr3WiUpfxtz84g3Sp3vOGEYBrSc3p59cbgUBlBPYvcK8VQ1q8/OAM4qEAPvVj5vHeSwKIIyq6GzqhJov1pCGN4dz1u8ewXlWOsRFAV7Nb00MAtJK8GZfdxaliDWMRwXVfRyeiojs6bgc8SdyQqb4tMw8EkzwQuEj0lpJAZIKVV0rThtLQhVwVq1mTfob8GhCbUzR54pgVpLSMQPC7t5g48/9Y0cMVrzQ/zkWOYC1XQanWxP7p3teJ58yP4vLpqLpOuG0k0yLq0rL+IpWGnDMC7h3Sj0AQCvpxx3N2otaUsXs80pekZMSlA2m+2mDfbz3tUcsI9oyiCGV3b4MRDHIgjRV4IHDLCFLFWt/yA8Bgp5Rpk8S6lUPlNPu887m7bip+eCAAAGnK8BzpEmPNhnYX+VUgNsOm3V32ItbRazOhX641Ua431evCJA6XgeDUPYyZ7LrR/DgXOQJ1negDIyCE4K4ju1BryhACPlw5qz8f3QkiLqeUOf38eiMQ+JznCM4kChACPuwc68/CdtcNbMhJP3Q/QBlD56pqqI6oENDf/aqMYAT7JqMgov0vxkqmDCHg65vO7gR8sXBbQqqVIPqBqDhAaUiTJNatHCpngPAoEJlkP7upHNIGAhNpiAde7cyANhQuMEYAMFkmfY6NwLQB3tsxoWEErjqL5SbwzHdZF7HPIpmr5gjsKwdnVNfR3jeMAPCa63fC7yO4ZkccwT7M5ZZEd7NLDKvSDOCNQOCCEZxJFLF3IuJ4VKERDu0cwW+9eB/eeMuevpxPcikNccM5XWgYgc9HsG+n8iW28b4tZ8rYMRIa6EAaI/A6c7eBIF2q9a10FGBVQ8VaYyDT03KVBgLKZ1K3cqicBsJjgMQDQa+MYFqpPmpfTOpNWZUPNvI677ssM2koOsN+vvrVgF9krMAGOiu5WI7ABSNY+ilQTgFXvML6WM4IHASCs4kiJMGPmbh9szkzTMdC+P9ecSV+84X7+nI+SXQXQIc5AgVnN+zNKbb9EgjBB267Grfsm+jL+dzOI9UaznWhqswiUEznDu5hgaBSylqed6t6CADFeC4SVE3KnCJVrPdZGvIzu54BjFbMl+vq+6zPCJRAEAwrFT89MgJpipV/diSdtUE3UdCZu1HaYFVLnBGERoArXwk88VVbnbvJjkquiMsFDae+A/gCwP6XWh8bENmxDpWDfVPRvm6AfuvF+3H7obm+nMstI8hXGmpnsh14IxA4rBqqN2UsJEvYP9277jcoSIK7HEGh2jBukNMwAgA4fNk0atSPxIb1YrJVPQQcoxHBVfmorAw/6Ye9BEdUZOcaRJ4gW66rbpyGOYLwGPt/ZMK5NNRsMHlJKw0BXfKQNjGfyOtIQ7x0NDbb+t2h17NO47M/sHwZuoyg1nTeNHjqHmD385hcZgVCHBvP2Z1nvlVwvU6YKQc68EYg4IzAJuVbSJXQkGlfGUG/IYnuyunylbp+xRDAcgS+ANtNglUxFBFGKm1eglhvyriQ39pAMB4RXCWL85UGmjLti70ER8t4rrcZCZ2glCJX0QQCI0YQUhY9acK5NFROA6Dt0hDQVTmkfa83Cjrvu14gOPByxgxOfsHyZfDz89xNRPSjKVNUzeYwdCJ9nllKWFULaSHEbDOCcq2J5Ux5268TbqYZ5qsm64QOvBEIAEfOgmd79A7ZDLjuGKxa5AjEuOrKGAsFUfVFkM+ZNyWtZSugdGt6CDhGI0FX5aMtw7n+BQLXVtQbp4F/fj3whTfq3l2py6g3KXaOMhM23RxBJaNhBJPOq4Y4g5A00hDQdR7t/AddRlDQCQQBETj4y8DT37Ts8k8VWW/HSJixq5bRogOZ45nvslsngUC0P5yGm81tb+XAnUFlvmKiHOjAW4HA5gU+2+dKgEGAl9M5pcp5M8qnHUrDIUioFLOmic+tLB3lGJcEdVF3glSffYaAFiOwrc3WisC9fwx84nnAM/cAp+/VPYwzgImogHDQ321FLTfZzOk2achhQxkPBBbSEH+vo2LAXBriyWKO614P1IvA098yfRmpUntvh6v5u6e+A4zvb/VU2IGTDeOGsk5sc0YAOPclM10ndOCdQODAW/9MooDJqKjuRrYjJDGAhlOqDK79meQIxPZAEAjHITTL6odeDyvZrQ8EPEfgNDCmBxAI+C7MUhqiFHjybuDjNzF75GteC9zyDqBe0t0xcwYwEg4iHg505wgqSlKfBwJp0rk01BkIQqOAL9glDfH37cBMFImCXiBYZbX7gY5qmt3PB+K7LJvLUoX2Si7HC1q1AJz7kTM2ADgaz3pmnV2jXszmBg23s0sKZsqBDjwUCOwngbZ7Aghg0hDgLNLXmzLK9aZ5jiDU3ucQiY4gSsp45LyxPKROJhvZSkYQRL3pzEMdaDGCfuYI+BfIVBraOA187rXAF9/E3vO3fht47d8xXx5AV9LhjCAeDiIeCnbnCHhXsZYR1EtArX3giSk6AwEhuk1lqSLTkHeMhrGhywg0PQRa+HzAodcx1mMiW6VK7U1+KiOwK3Gc+yHQrNkrG9XCISPYORpG2EF1zWZDcjnNsFBpDHME/XAdHTQiLhxI+bGGF1iHEYSjI4j7qmyesQGWM2WMS8JUUHCcAAAgAElEQVSWfiHc2kwMIkdgOlO6VgT+44+Av7kFWHoYeOX/Bn7rR8Ce57P7DTR5oGU4Fw8FEA+bBQIlWcwXcyeVQ5xBRDRlzjpNZelSDaORIKaiojEjiM10/x5g1UO0yaaFGSBVrLU1J6qMwO6Cduo77LO8+3n2jue4yDaMEZeMIF+pX6Q5ApsXOF2sIVWs9cVaYpDgi42T2uq8meEcoJsjIGIMo/4qHlkwDgSsh2DrEsWAe5uJdLEGMeBzVDNtBUOb8HIG+JvnAff9BXDoTuCdDwO3/Ha7LbJJIxiXghgj0JGG+FAarTRkcC5DlFKsckYr6ej4DXEfoKmYiHylgUpnz0TBgBEAwOy1jPmYNJeliz0wAlkGTn2X9Q6YmczpQbA3wJ5SirOJgmfWCSfJ4qZMUayZKAc68E4gsHmB1QTQdo/0LpJnloFAhxFAkCCRKp5ZLxjOyV3JlLdUFgLc20zwBa2fDUFCwAch4EO+89qsHgcy54HXfBJ4zd/q75j54q0zJ1iVhkJBC0agqRoCnDWVlZJAZLzjNXVLQ2klmTsVZQGjLWHc2VWsh0N3AUsPAalzXXfx3o42RuCk+mX1OKtacpofAFo5Aotc04VcFcVa0wPrhPMpZaZW9QbwTiCwmQRSx85t40oAQMMIHGh/rQusQ/lkWb9qSIxClFvT1TpBKcVyeuu6ijnGIuxvchMI+pkf4IjpGc/lVtntrhuMH6gu3nqMgOcIAixH0BmY9XIEgENpKNkuCwEtaUizOPIAOhlj792GVh7q7CrWw6E72e1jX+66K1uuQ6bteRsucdj6vJ+6BwBhfQtOIUTZa2/odEtr0E934kEi6qJqaMsCASHk04SQdULI45rfjRNCvkcIeUa5HVN+TwghHyOEnCaEnCSEXG/rSWzmCM4migj6idq0s13BI70zRmAynaxWAEB1GEEUPrkOkdTxiE6eIFdpoFhrYucWBwKu8Tu1mUj12WeIIxrSGU6TW2a3ZgukIAGBsAEjaCAU9EEM+FnVUKXRXiWljqnUNJQBDqUhnUAgTbPEa6VlNZIuckbAJME2RqDXTNaJ0d2sguixbkdSvbyNI0Zw6jvA/E0tduUENo3nzmxs/14jQBtAna8TW5Ej+AyATh73fgD3UkoPALhX+RkAbgNwQPn3dgCfsPUMYoxFeQufk7OJAvZOSAj0wflvkGjlCPoU6TWzCNqgfDEOT/l18wTboYcAYHKJjzifUtapRfcLulbU+VW2SAsR4wcaVOkAjBHw4TnxUBBNmbbv9Mpppu/znENolBkuOmkqK+oFgvYEdqXeRLHWxLgUxFSMSUNt3cV2AgEAXHcXsHEKWD3R9mu93o6w3al8uVUmDTmtFuKwaTx3NlFARPBjNr61uTErSIIL5cBsZokB+rJaUkp/BKCz8+UOAJ9V/v9ZAL+s+f0/UoajAEYJIdYOTTanDzETqe0d5QF3kZ6PN9Q1nevwGVIhsPfihh0Cji9muhrLWoFga78QPh/BqAubic7qlH5Bd1xlbhWI77B+sIE1RK5SR1zpbeG3bXkCbVcxwIKKU7+hUrJ7J93RVMYrs8YkARNR9t61MQK9rmI9HPxl1qPw2Jfafq0XCHw+gnDQb80I3HQTa8EZgYV6cEapGNoKt10n8PsIQkFn0wzzZuuEAQa5bZ6hlCqiKtYA8MzTTgCLmuOWlN+1gRDydkLIw4SQhxOJhK3pQ42mjIVUaVt3FHO4qRrikV53JKMRI1AC6OHpAPKVBk4n2t+/FXVE5dZLaU5tJupNGblKYyA5At1AkF8xl4U4pCldaShbriOufDl5s2Nb5VA53W2uJk3aDwT1Muv67UoWt/sNaX2Agn4fxiLBdgdSo67iTkTGmY7fMdjeqNtbsjO3+NQ9wMh8qx/DKWxOKTub6K878SDhdJohL3KIb5NAoIIyIdRRyyil9JOU0hsopTdMTU3Zmj60mC6j3qR9mT86aIgBH3zEufYX8BGIAZ3LpjKCjsE5yhfjmkn2mM48wXKmgqCfYDLaHz/2XuDUeC6tatH97yDXzxGs2GMEkUndSp9cudFiBCEdRqB1HlXPNWFfGuJ2FIbSEAtO/H3jktpkVGyfSWDUVayH+ZvZ8ZqF1ygQRKzmFtcrwNnvM1nI7U5dsGYElbpiNucB5QBwPqWslUvcHn0EF7jko9zyQuZlAPOa43YpvzOHjSTQmXWlEqAP80cHDUKIY69x3jauS2c5I+DvE4fyxdgZbmI0EuzKE6xkypgbCfdt3m8vGI0IjqqG0sWWxNFvdOUImnW2o7YlDU2yRbcjiZqraHIEYbZba6scMgoEdhlBZ1ex9hwgaiDoXKinYh1NZUZdxXrgrKFwQf1VqlhDRPAjFGzv7WBTykw+78/exzqp3cpCgIYRGK8T5zaKoHR7e5Fp4XR2ScGqzFwHgwwEdwN4i/L/twD4uub3b1aqh24BkNVISMawkQTiPQT7PUL5mEe7M+3PeCiNkTTEdj2kVsBz50fxyEJ7Cel2aCbjGJeCjgJBp9VxP9HFCAoXAFD70lCz2vVZzZXrqiRkzAj0pCG7jECnqxhgyefIhCoN8YQ8l9SmYmJH1ZBJV3EnoorspJHC0gYlvcxo0eTzfuo7QDAC7H2hvefWgw3loDXP3BuMQBJs5FY0yFca8BFs/mAaQsjnAfwEwJWEkCVCyK8D+DCAlxNCngHwMuVnAPgWgLMATgP4ewDvsPUkNubvnk0UMSEJGIlsX7M5LSTR74gR5CsNY7pnlCxW37cirt89htPrBWQ1OvxqdmvnEGgxJglIl+q2jedUaSg6gEAgBFBtyK3B7rkVdhvvSmd1Q6cjmM0iaKhMIN6ZI6BUmVfcyQgmWYCwMRXMUBoC2vIWvER3VPmeTEbF9j6C/FpvjKBUU5PQWkQEk887pSw/sO8lQLCHjYmNHAHvIdjOZnNaSGIABYfKQVQ0UA4MYJ87mIBS+gaDu35e51gK4HccP4kqDZkHAq/ofgCnfM60P1N7CeJXGYAKjWZ6/R62yDy6mMZLrpxGoyljLVfZFoligO1Qaw0ZpVpTtXkww6AZAcByOEJA0AQCm4wAYNr+OJtdW6qx4SKcCfDrqHZ714qAXNeXhgAWDHj1jxFUaUin/j461ZYjiIcC6nD1qZiIUq2JYrUBKehT7CUsKobU83YPvjFq8pOEAC7kDBq91p8CsgvAi95j73mNYIcRbBSxYySk9vJsd0iiXy3qsIOcQ58hwEudxTaSQGc84B2iheQwCVSoNtShKV2o5Fiw7NwFaDTTw/Oj8BGo8tB6voqmTDG3xfYSHHxBtysP8UAwOqCqIUDT2p9X1MuYnWSxsnhrkryq4ZzCBIJ+5o+k5gg6u4o5nDSVlZIAiP5YR43fEO8q5uCFAhuFaqurOGozEEQm2ATBjhyBXpNfRPQbd8ie+ja7PfAL9p7XCD4/k5fMcomJgifyiByO1wmHswgALwUCiyRQtlRHsljzGCNwLg05GkoDAH5BHegdFQO4YiaGR5WE8XbpIeDgUkXaZndxqlhDTAxA0Kui6hFdVtS5ZcAvdpdm6qGjSgdo9xniaLOiNgoEJpYVXSgl2eN9Otqwpskt3WERzZvKEvmq/WYyDp+fndtGIOBzi3Vx6h5g7rC9ZLwVTFwImNlc0ROVhRyOk8VVZxbUgJcCQUBkzSsGF/iMB6YNdcKpNFSomiSLK7nu0lGAMQQhqg5KuX7PGI4vsMay5W3UQwC0qljsMoLOBa2f4LkY9frkVpksZEd31TGeazmPtq5f23CaTnuJznPZqRwqbujnBwAmDdXyQL3MFmoNi2oznnMaCAAmDylso1JvolRrGjIC3Z1tMQksPtRbtZAWJr5kiXwVhWrDMxVDAN8wNmznzpxOJwO8FAgA0wusVgJ4iPKx+mB7kZ5Sau4xbsQIgDYL7+t3jyFfbeCZ9YI6kGZumwSCURfS0CB8hgDNZKiKRhqyIwsBQDDMgq9m8VYN54wYQaXDgpojoi8NpYu17koSPZ8h9Q9qafmdthxtxnOqBOYkEMx0N6sZMIJqQ0aj2TGV79kfAaDA5S5M5vRgMtb2jAfmmXciIrBphrXO980AbMN4seYIAJYnMGIEiQKCfoL5bW42p0XUaIekg2qDDT53MpRGhcbC+/rdbMf5yEIaK5kyRsJBxzRyUFAZgc2mskEGAlUaqmqkISeyBe8lUMAXfO341DYrakNpiOcb2hnBmz79IP7km0+1H1tKGRu1aRLYnUZ9E5IIH1EYAZd4rLqKtdAJBHrJYl7OWOqcfZBdYreTB+w/pxlE83UC2P6uo1o4dSo2LSoxgLcCgWgc6c8mCtg9Htn2ZnNaRIQAynVWTWKFglXbeDVrzAgESWUEl01KGIsE8cj5tNJDsH0C50g4CEKAlE2bCaN69X6AS0OFSoOVNnJpyC46bCZyHcliAO3DaYwCgT/IRmF2SEPPbpRwer1jsdObRaD+QSwQVLNrqNTltvfN7yMYl5SmMiddxeq5p1kAoVRlc3rlo4ZTynKrzLE1pCNtuoEJIzibKCIc3P5mc1o4nV2Sr5gUlRjAO6smYJoEYqWj3onygLPpQ5ZGUmaMQGzlCAgheO7uMTyykMZypoyd2yRRDLAFaSQctM8ISrWB2EsArfe5UK0rdfxV+9IQ0GUzkS13d3t2MQK/yGSlrnO1m9hV6k0Uqo32UkxKLaQhFghKaSb9dL5vk1EBiXzNWVexeu5pVvpaTttiBF15sbyD/IsdmDCCsxsFXDYpbYtOertw4ktWa8ioNuRLgRF0X+BGU8azyaKn6B7QciC1M3RCbRvXayijlO2ADBlBewC9fvcoziSKeDZZ3FaMAGALiJ0cQbnWZDvbAUlDkSCfFdtszSHoURqKCH61dh+AOpyGUtrqKtZbDCOTbVVDSWWxXctWWgnEao4txhY5glqGST+dJbeqzYSTrmIOTS8BDwR6jrCGg9idNLDZgUku0SvuxFpEHEhDXDm4eKuGAENGsMTN5jx2gU2HpHdANZLSi/S1Iqv9NmQE7fOer9/N5IdKXd6GgcCezQQffjIIC2qA2SarfkN8MpnTQFDaUP2GtLMIOOLhAGSqzKTQ6ypuO1fL5T2pdAFXG3KrIc3IZ4gjGALEOBo5Fgg6cytTUREbvGrI6aKs6S5OFWvwkXYJjEO1Xu9iBCvOktNWMFgnKvUmltJlz20Yow4s61vrxMWcLBZjutqf6jHksUAQMdoh6YAnLR0NpeEQpLY5DryxDNj6gTSdGJcEW30EqYKxBNEvMCvqOluoAGcLpDQFyA21GojNImi/dqrfULmubzjHERlvk4aSmiEya1weUu0lTKZ6SZOgykyCzvdtKiZio1AGddJVzKEGAsYIRiMC/DrSi+6UMkoHwAhizLxObv9enU+WFLM5j64TDiTki1saEvQp31mPzCnuhFqiaCvSm0hDRj5DHJo+Ava8AVw5y47dTjkCwL4Dqd44xH5DNZ7LrQAgzhbIjqlguXKjrWII0MwkqNTNGQGXhhR2ofUFWsvyQGDBCABAmoZfCShdjCAmItbIgjjpKuZQjefWTSu5JL25xZUMmzzYz0BgYDPhlTnFnWgpBzY2jOo6cTEHAlEpg+xorDiTKGBcEgamFw8KTua4FsykIZURGFRdiFE2s7bRWmB5Gem2ZAQ2AgFPKA/ymke52VduhS12fgd0u8NmQmtBzdFmPGfGCKRJpv8rbDipSaZ3BwKTzufoFIKVDRCCrqA0GRUxQ5TKJaeMIDTCEt2KNGTk/aS7s8256FuwgoHx3BmPmc1xqGW3dtYJVTlwJg1tjwJyuxCiAJXZJCbN3NhzG0XsnTCZI7tNITnKEZgkgSwZgcanKcAWijfctBt+H8FMbLsxgiAqdRnlWhNhExtds6Rkv8ByBHUlgeq0kqbdZiJXqeOKmfZZEV3SUGdXMYe2qSwUR7JQheD3oaaYBrLnMbCg7nhN4VoKo+Fgl3QzFRMx5TYQEKJ2F6dLNUNmrjt/N+8i/2IFA0ZwbqOEmbhoy9BwO8HZOmGyYTSBxxiBvvHcYqqMPRPeivJAiyrbqhqqNiAGfPq+OtUsuzXLEQBt+ZVrd47gj++4dtuV0dk1nkuXlKSkw52PE6jjKu3OKtaiw4o6W6p39YDwnEG+VGQjJs2kIUAtR00WapiKiZiMCq0S0lKSWbB0DiZqe03TkJpZTES6AyxjBEp3s5vdudJLkCoa236E9Xa2biwtrGDgVLyYLmHPuPfWCTHgg99HbOUSC2a5RBN4MxBoFrR6U8ZqtuypjmIOSS0Ls470uUrDmO5ZMQLNTILtDl7WaDWyMqk0kw0ykEVDvGrIYVcx0GYWJ8sU+Wqjq5KGB7FKTkn06rmGAhoHUhYINoo1TEYFzMRD7dKQNGlei680le0NdVsaT8VETENhBE66itVzz4AWLiBdqhuyNCHgg+D3tRuouUnEW0FlBO2FJYupEnaNe2+dIIRAEvy9Kwcm8FYg0KF8K5kyZArsGveeNMRr1e3WB5s6jwImjMDawnu7gCcarYbYd/rlDAJRMYBatcQSmk4XqoDAtPNiAoVaA5R2sxd+PesFHgiMGEG731CyUMVEVMRsPIS1nJI4LqXMZSFAlat2i92fg9FwELO+DEqBEWddxRzRadD8OpoyNb0uEdHfPrc4v8YkMb1GOrfQyRFUG02s5SqYH/PeOgHYmO6mIF9pQPD7usaEWsFbgUDnAi+m2O7Gixc44PchFPTZ6hg09Q+p5JgnvGBQDWFjjut2wZhiRZ2ykIbMkpL9QlQMIFZVBq640bCVah9uL9GZoA34fZAEP5q8A9lSGuKBQGEEI6F2acjKIltpKtsR6A4EPh/BfCCLrN8imBghOgNS2oAfTdO8jSR0WCr3u3QU0B1ru5KpgFJg3oMbRgC255sXqnXH+QHAa4FAZ2e7mC4BAOY9SPkA+0MnChUTj/GqwVAaDp4j8AAjGFMZgXWOYJClowCThmaIslt3EwgUvyE9C2qOeDgIqvoMGUhDggQEQkApCUopksUWI0gVa6g2mowtWDACqjCCGX9O9/5ZXwZJYhCMrBCdBgHFOPLmjKBz/m5uxZmHkx3o5BIXU8o64UEJGWBzi+1tGJ1bUANeCwQ6O9vFVAkBH9k2U7acgkX6Hi+w0SwCDsFDOQJl12yVIzBLSvYLUTGAGSiBwInPEIc0CZSSukNpOOKhIIiRBTUHIYrfUBK5SgP1JsWEJKjGaeu5qrnPkIJikDGGKV9W9/5JpLFK3QYClleYIhlTptY1t3iTGAHfMO72YHUhYH+dMN0wmsBbgUAnR7CQKmHnWFi3k9ELiAh+W9OH2NQhF7MIAFvznrcLAn4f4qGAaY5AlinSpfrADOc4YqEAZnlJpZtdq+I31DmmUot4OAC/VSAA2CJf3FDtJSajImZHWCBYTRdYQ5pFIEg3RFRpEKOyTiCQZYw001iuu3QAVWSnKZLFuI7zqPpnCBqtW246m49sF8Ewm9/dsU4Ift+2K5e2C7uzSy4xRqCVhsqezA9wRG0zAoscgVHFEGBYRbFdMS4JpowgX2mwpOQm5AhmSQrNYMy8LNMI0hTbxZfY4m3ECIK1LABizuoUdsGbySaighoIUsl1ANTcXgLM3juBEYzI6e47FX3/fC0G2YYteheiPBCYM4K28azFBPPI6jcjIKTLeG4pVcbOsfC2K5e2i6hoUxoy2zCawFuBQIcRLKVKns0PAMxZ0IoRUEqtq4bMGEFAZDskDzACwNpmYjPsJQBGx2dJCtWIi3JKgC3MVEY1z5LBRjkCoZFl+QGfyddRsaLmjGBCEjGjSEP5pFKLb5EsTpVqSNI4IvVU951KY9eKPNoysnMCJRDM+XOmjYBtjECdhtbnQAB0DbFaTJewy6P5AUBZJ2wxAudDaQCvBQKfHwhGVO2vWG0gWaxhl6cZgfWUslKtCZmaNIlULRgB3yF5IEcAWNtMmI1D7CcYI0ijJE67O4HSVNbMs8ojvT6QeCiAcCNn3FXMocw32FAM5yajAnts0I+SYi1tKQ0Va9igIwjV9AIBO0eCjjI7aqcQJFR8EcwHzVmnJGqkULWZbACBoGOI1WKq5NmKIcC+cmC6YTSBtwIB0DZ9aCnNSkd3e/gCR4RAe121DlpNIiYNZVbShcmYz+2G0UjQ1IF0swIByxGkUOgxENDiBmJiQDePFQ8HEZELoGb5AYA1ldXyyOTYNRyTBBBCMDsSQjWnzD2wCAQpJRAEyxvddyq78wt0jI2sdIGMb8ywIomj7fOeU5rJ+l01BLRZUReqDaRLdU9LyBHBbznNkM01vxSSxYCys2UXeIGXhHk4EKg2BiYoVE38Qyi1loYA0zGf2w3jFtJQ2mQKVj8RDRJMIYNcYMrdCZRyTVLa0E0UAyxHMIoCmqIVI2CLfDm7jtFIUB1wMxMXIRds+AyBldymSBxEMydBhTKrOIHRNndTJ0hiFFNEvyKJQxL8KNWbbKBOfg0AURPNfYUmR8BLR728YbQzzbBSl9GUqWPDOcCLgUDovsBerQ0GeF11szVpSgc5M4/xepn53ptJQ4Ayk8AbjGBMElCqNVHpHHKuYNNyBI0UAkRGKuCyyUoJBIHyhiFdj4cDGEEBNcGiWkdJBDfyibaGrdl4CKTMZxFYMYI6SsEJELnRmpHMkV+FHB5HDUHXjOACHcEE1UlEaxARA6CULVrIryqurgMwgRP0NoxeXid4IDDOE7g1nAO8GAjEuHqBF9MlRAT/wBeEQUISA2jIFNWGbHhMwcxj3MpegkPwTo6A7/SNSkjTxRqEgE+15x0UxBLTsDd85tU4hgiz5K1QTZkzAlJEJWBx/Xi+oZDARLRlATEzEkKwmgYVomwKmQnSxRqqopJQ1ozRBADk10BisxD8PteBYLURR7xpHggk7dxiN66udqGZytfaMHqXEdiZXcKHV3WaG9qBBwNBS+JYTLHSUdKvoddbAEmwdiBtTR0yG0pjsaPsGFe5naHaTBiUkKaKNUwoGvkgQRTdfJ1aWDcYwR8AwuMI1VKGLqkjIT/iKKLss8jxKLt9Uk5iUlOnPxcPYRQ5yCHr15gq1dAIc7sKvUAw15pd7BDVRhMrjTjCzTxQrxj/GdqpfINoJuMQomq59FK6jKgYwGhksH0ng4ThvGcN3BrOAVsYCAghrySE/IwQcpoQ8n7bD9RQvqW0t0tHAe1gauNIb5ojcMQIvJEjsLKZSCnOowOHMjTFdbctAEhTkBqZLp8hjlF/BX5CUbAMBGwBD1ZSmJBajGB2JIRx5FEVLHIMUHIrkdag+Tbk14DYLCZjoitGkC6yHgUAQHHd8DhJ7GQEfW4m4+AbRkqZ6+hY2NMbxogNRlAw2zBaYEsCASHED+CvAdwG4CCANxBCDtp6sJIEopRiweMlYUAreps1i5jOIa0oyTmrHEFHg812Bl/kjYznUpvgMwQAyC2jjgAuNHoYbShNIt7M6PYQAMAImFyXIxbPER4FJT6I9QwmNIxgJh7CGMkz11ALpEs1+OJ8rKSGEWg6fKeiglqi6gTJYhUJqgSjziCjAWcE5XKR2WL0cyCNFkKU5c4aVSymS55OFAP2ksVqjsBDjOAmAKcppWcppTUA/wrgDluPVBhBqlhDqdb0tO4HtMbQmTECHgg4PWyDbUbgpWQx29GkTXIEmzKWNL+KtG8c+apx/sYKNDKJUZo1lIbilLG0DLUIBD4/aGgME8i15Qg4I8j5zAMBpcyWIxSfYE612kBQSrIO3+gsk4bcMgKqvAaTQMAZQT0zgIE0Wijl1LSaZxKyxwMBD6B2cgRe6iPYCWBR8/OS8jsVhJC3E0IeJoQ8nEhoPrRiDKiXsJhki5rXL7DKCCy0v6hBHbrlUBoOIdY1t3i7gjOCtEmOYHwz9N7cCrLBSVsDQYxQE8cxTnKGyeJIk12/FLWenFUXxzFO8pjUBMGpqIgxkkeamktLOcWWY1QKM5lJu1jnW3ODp6IiUsWqab26HtoZwQXD4/iCRnkPwSCTxQBS6RTK9aanKwsBLSMwXicKZsqBBbZtsphS+klK6Q2U0humpjR13IrNxGqC1U57Pkcg2MsRmFpQA/b6CABPsIKg34eYGNDtJag3ZeQqDYxLLoanOEVuBXlhpqdAUBHGMU4KGDEgMP4ak/aSDesNTUUYwzjJtzGCgFxDlFSw3jRnFGltE150up0RKF3FiM1hMiZCptbur3rnT0L5DJoxAs5qB9lVDKjrxPoGXye8vWHkOQI7yoGXpKFlAPOan3cpv7OG2HGBPS4NtXIEJpHerG28kgNAWrMajGAw0Hu7YkwSdBlBWu0hGDAjoBTIr6IkTqtfMDcoBlmiecJv8L4r9fyJhvWGpugfwThybTkCPr5ytWHOKHi+ZUwSVFdUFSojmMGUEmScykOpUh1NEgANj5szAmVB8xcHHAiUdWIjxd4frwcCSbBWDgrVOsJBPwJ+58v6VgWCnwI4QAi5jBAiAPgVAHfbeqSyoCVTSUxIgjr316uwG+kNm0T4UBozwzJAM8DeI4EgEkRKJ0fArScGniOoZIF6CZXwtC2PFyPkFe1+HAbWC0oguFC3tkfO+kYUaUjDhpRAsFix7iEAWNc2pOkOaUhZlKMzmIwpgcBhCWmqWMVYRACJzpgGAr6gBUoXAL9gPVXNLZSNUSbN3l8vG84BgN9HLKcZurWgBrYoEFBKGwDeCeAeAE8B+CKl9AlbD1a0v0wm7ck5xZ2wWzVkSPesLKg51KlNHmkqkwTd8tGUdkEbJJRdci0yh3K9iUbTXcI4owQC3RkAAFDOoEJCSFWtSxvTiGEMecRDmq+tEgjOlcwXupSZNFRYY81vAVFlBBsOGUG6WGf9H9FpU2koFPSBECBUUuYQDKqkU2EEuWwak1FRlWC9DCvjuXzVZMNogS3LEVBKv0UpvYJSup9S+qe2H6gwgkIu7fkEEACIAR98xIoR1A2rTmz5DAGem0kwFggGu/0AACAASURBVNGfSZDWShyDRI4plbIiXdixANZDWqmkiTYz+geU0yj6Yuo4SzNsNGPwEwpS0QQVzgiqEZRN5MV0pzRUL7U2BZrGrimXjCBZrCpBZsa0j4AQAkkIIFRNDE4WAtTPezGf8XwekSNiMdY2X2nouw/YwLZNFhtCifSVQtbzuh+gfDEsvMbZdDIjRpC1yQi6h/psZ4xFBF2LCXUwy8ADAWMERFms8lUXHv0AEko1j9QwsF4oZ1ANxNVxlmZYayryXlHjHqoEgjSNYS1n3NGbKtYh+H2sk13qaCpTmskAZnkSDvpdMQKVbRTWu03tNIgIfki1xOBKR4HWOlHMej6PyCFZzC4pVOqumskALwYCJdKH5NLFc4HtRHq3Q2k4PDTAHmA5gkK1gVqHBxPXukc3SRryj7CGJ7eMINGIoEkJxIrODAAAKKdRDY4gZ2MYzHJNuYbK4s//T0GQhYS1rHEgYL0XQdZdG+1oKtMEAgCubCaSxVqLEdRLpp8zSQwgXku4mwNtF0qOoFnOXzSMQBLMZ5e4taAGvBgIFK1bIhXPdwtySKLfsD64KVOUak1j7c9ujkDwXo4A6LaZSBVriIkBCIEBf3Rzy0BkEuEI+4wVXDKCbKWJDImD6M0AAIByGg1hBPlqw3JE5GJVWdBK7YxAFkfQhB8XzBhBSWPLIWn8hnTmBk9GBUdVQ6xZTRMIANM8wUSwihAtD5YR+AOQ/SGE4e1RtlpYMgKXQ2kADweCKCoXT6Q3mUlg6R9ilxGo0pB3cgRAt81EurRJXcW5VSA+p+6w3JaQ5ioNZMhIu5yjRTmNZmgUlAIFi5m0akK4QxoiEjOkM5OG0kWNLYdWGtJ0FXNMxURHMwly2hnSUaXnx6RyaKdfyXEMMkcAoBmIIIryRbVhdF1daAHvBYJACDL8iJIydoxeJIFAO8e1A1yb1k0CUWqfEQRCbG6xV6QhbjPRMaksVdwkn6H8ChDfqe6w3DaV5cp1FPyjpoEAoTH1WCOUag2s1JQFrUMa8klTiIoBU2kopQ2gypwEFBNtXcUcTm0meFJ/IqplBMaBYIdPyZcMYjKZBlV/BFHifXsJDslkmqEs87nml0qOgBBUfGFMiw11SpPXIYl+FAw0aFPDuUYFkOv2GAEhbUN9tjv4Yt/ZXbxpgSC3AsRajKDgmhHUUQyMdds+A2yoULMKEmHWDGaVQ8lCDVUIqPsj7YGgmAQiE5iJi5Y5ArXkNiAAoRElEHQ3dk1GRaRLddRtlsymtBPjbEhDMyTT9ZyDQAlhREkFcyPWPRpegKlyUDOZWWIDnlxJiwhjStj+njl2IYnGjIBfeF3KZ9dniMNDA+xVv6FOaWgzLKgbVdUZkzcsumcEDVTEcX1GoDSTBSTWVGVWOcSlmkZovLtqKDKOuZGwoTTUlCky5Xq7pMabytRAMKPexUtIkzZdSFVGIImsH4H4TQPBFJTE+SBzBADyNITxQM1Vp+12BM8l6k0z7MVnCPBoIMhTEePBiycQsPpgI0agSEN6lE/1GbK2IAbgqZkEfIhIp80Es6AesL2EKpdoGIHbQFCpoy6MA9Vst+GfEgiCUabxZ02kIb4oy5HJFiOgVAkEE5iJhwyTxdlyHZSi3aiPN5Vpuoo5nNpMqDOkpSDrcI9Om0pDE3ISBYRbTY4DQqYZwqjf3bS17YiIwKYZ1nSYmuozdKkEgkq9iWwzhBHfxXOBoyZJIFMjqaoLRuARaUgM+CEJ/jYr6nKtiUpd3oRmMsUZM74Dfh9BRPC7loay5ToafJZwqYMVKIEgFGdVPGY5gmSRfd590kTrPNU8kwYjE5gdEbGe13cNVaWbNkag+A1puoo5uM2E3YRxUtu1DFh2F482k7jgcNjPT59N4dh5gxJco9dVFxAjxnKZ12DmVMyr2i6ZHMFSuowCDSF6EV3giBBAud7U/RLzQKA7h7Ri03mUw0MzCYBu4zm+GA6+mawVCAD2BXTDCOpNmZUFR3i5pn4gCPNAYBJs+LCYYGya5QWAFjOQJjEbD6EpUyR1Fu+WUZ+BNNSh1TtmBKUaQkFfy8bBwm9opLGBC/JoV4+IGf7k35/E+7580vbx5VoTybqACMq2H7PdYTa7pBfnUcCDgWAxXUIRYYTpxXOBzaYPmeYInDICIeaZHAHA8gTaHIFqOLdJzWR8gYyG3AUC/uUk2iodLcosaRoZYYzBlBEUapAEPwIxjTRUUnbIijQE6JeQprXJXA5pCqhkgMxCl1bv1GYipU1EA92mdh2I1hJYw7jptK1OXMhVcTZRxGrW3vd+KV1CESGIcsn2c2x3mPmSmW4YbcBzgWApdfFdYO5AqtdUVqg04PcRhIP+7gc6ZQR8jqtHMCYJbQ6kKb2d7SCQWwGCkpp7ccsI+MIeUMdD6jMCvzSOmBgwTRYni1U2hyAyATTKLKBziSgygVmlMkavckjXn4nX+yd+1hUIQkE/YmLANiNIFWsY11pjR6eZ35Css+OnFJFqAut0zHTIihayTFWZ6v7TSYujGfiGMdAo6b8ODyJiKg31liPwnCXfYrqM3SQM31YkPVeOA2N7gHAPw8x1oE1IznTcl6+woTS6g7cdM4Kot6ShSBDPbhSB0/8B/OjPcXUZ+PtgDVfePw+cjAPBCPsnRNjfdvCO/szAza2wGnflPY+KAVc5Ar6wCzwQ6OUIfAFAiCIeDlqWj05EhXaZiTODyDhmA8aMIKUwqa5dO8ByDDrVO05sJlKdlVzRGTYvuJLptpkupeCnDVygo7YZQapUQ0ORTe8/vYE7j+yyfMxiiknIAIB6ceCJ6TYkTrFNV5/nMUdNLOtNi0pswHOBYCFZwt5QDKRWYFUTg7Kx7US9DHz6lcAVrwBe/9m+npprqyWdSJ83NZzjgcDmh1yQPJMsBjTS0PEvAqsnQSP7sYtkEEmkgNVyy0GTKu9b6ixw+0d6f+L8aptuHhUDWCg6Z6B8YY/EJ9iC3yUNpdmmghDEQuaMYKNQxa6xSMseopTUBIIJTAgi/D5iyAhCQR/CgoZVSpqpf9HuQDAZFW0bz6WKNeyd0DRtcS+jwoXuQJBn+Zc1Om7bv2k9x15HVAzgvtMboJTqb4w0WEyVWM8FwD7zmxkI/vl17P39jXv7uj6p64SehFxpgBAgoqcc2IDnpKHFdAlCJM52HI1NrBxaeZRR8ie/DiTP9PXUfKC3nvxgaTgnRAGfzYsvxoBmFWi6883ZbIxLAvKVBujqSWDfi/FPBz+FV9U/DPK7jwLv/RnwgUXgD1PA/0gAs4eA1Ln+PHGOdRVzREMBVxYTfGGPR4JsYdCThhR2yRiBmTRUw1RMYNIQ0AoEviAgxuH3EczERANGUOue3xDVBIIeGQEztOtgBIB+wlgpV71Ax0xncGixnmd/023XziKRr+KZdevNzGK6BFFSyqo3kwVnFlneZfkY8OyP+3rqlnLQHUBzlQaiQgA+vbnmNuC9QJAqIbQVF3jhKLv1B4EHPtbXU0sWkd50TKVdWQjw3rjKSBBhVIDkM8DsIdU4revDHhCAscuA9LO9P6ksM0YQb2cEdhctLXhfwEg4yCSdzkBQyQAh1lUcDwUNq4ZkmSJVrLGGLR4IuDQUmVB3nTMj+r0EXQs10JKGAN0OX7vGc9VGE/lqo72Sy6y7WEnEr2NMlwHrgb+O11zPgvN9zxjYdWiwmCpDio0qL3ITZeTFB9mtXwTu+6u+nppXDRkVlbhtJgM8Fgiy5TpylQakuKLRb/YFnjgAPPeNwPF/aTXi9AFm3av5qonHeDVrP1EMeG8mgSTgKrIIAgrMXqe/oKkH7wUy53tPDJY2GNvUWCTzHIFeR6cZ+A4/Hgp2zwkGOhhBwJARZMt1NGXKcgRaaUixl+CYjYd0paFUSceWQ5CAgOLVFevMTDFGkK80UKmbL9Z8ZoRuIlqPEShzHtbpqANGwALB9bvHsHcigvtPmwcCSikWUyXE41sQCBaOskKDF70POHMvsHqib6c2XScqddeJYsBjgWAxxXTakRElEGzWzlaW2QXefTPw/P/KFoqjn+jb6SWLqqGex1RyeG4mgYCDvvPsh9lD+hKHevAeoFljDVK9oKOHAGDSUEOmqDqoeweYNMQb0iBN6ieLeSAIBQ1zBGr/RFRk19sXZOdS7CU4WHexTh+Bni0HIa0FO6ofCADrpjLe8dzGCMQ4MznUlYZWIYcnUEPQdtVQIl9FLBRAKOjHrZdP4ujZpKkPUrZcR77awOiY8t5s5ud98Siw6wbgpt9k5dr3/5++nVoM+OD3EV0m1YvhHODRQDDGL/Bm7Ww3TjEav/t5wPg+4OAvAw9/mk0H6wMktSzMRY7ACSPw2kyCiICD5DzqwTgwuptVpxjZS4ztZbe9ykNqIGjJJTGXVtS5cgPxkFLxpZsjyLTlCAoGMwl4M9mkJLAFPDLRLg0pmB0JoVBtdO0YDY36pOmurmKOSZtNZWppqjbQ8OE3utLQGqiSnDazVNZiPV9RA9MLLp9EsdbEiUWD0Z9gshAATIwr781mrROVHHDhCWD3LUB4FLjxbcATX2NFDH0AG/PpN8wlum0mA7wWCNIsEExNKPR4syL9opIfmL+F3b7g3WwRfvjTfTk9z/TrVVGYDqR2ygi8NpNACuIa37NIxa4ECFGGn3QvWgCA0b3sNn2+tydVqlrapCGXVtS5Sh3xsBK4pEn2ea0rDVHNBvsMqYwgAErZ9e6EuutWFmfGLlIsEHCpCEwaAtp7CepNGblKQ78Jb/wyYOJy3dfeYgTmnl6pTnsJjuiMQSBYARlhQdYJI5hWXs/z9k+AEOA+E3mIrxOzU3yd2KTP+9JPASoD8zezn295B6sWe+DjfXsKI4NK01yiDXgrEKTKiIcCiG629rfwIEv2TexnP88dBva/lMlD9d6tLgJ+H0JBX5dmWm00UWvIvQ+u5/Basjjkw1VkAavhKyDLFOlS3dhwbnQeAOkDI1hl7pnRVjKVJ/Pt7mDVU5XrrWvXaTPB2WRYSRYrAUMvT9CShpTFNjLOGrbK6TZGMKMTCLiGr/u+3f7nwBs+r/va1e5iC0bgPBCswRebgxjo/rwbYT1fxXSM/W2jEQGHdo6Y5gm4cjA3rUhfm8UIFh8EiA/YdSP7OTYLHH4D8OjnTDutncBovnnukgoE6RIbMrHZC9riURbltTXBt76baaAn9L9ITqE3t7hg5R/iNkfgkWRxKHsOIVLHgrAPee0ULD0ERFby2Q9pKDbbVpLLGYFTaShbriMeVq5dp82E0lWszREA+lbUG4UaCNHIL5FJYOMZALRLGgLam8p0u4o5wqNtjEKLCck6R0ApxfHFDPw+0j1DWs+BtNlgC2Jsju1sbVQNUUqxnmsxAgC49fJJPLqQMWRoC6kSRiNBxGIjbGHerHVi4Sgwc0375uz5v8tyVw/+bV+eQhL8ugG0YFZUYgPeCgQpZWD9Zla/FNaZxrf75vbfX/YiYMdzWSmp7G6wuRYs0rdfYFMjqUaV9QQ4qhriOQJvBAKsPQYA+BnZZ89eYmwPqxzqBfmVrnLKmMi+YM6loQYrHQXaq32ArkDAj9Ozok4WqhiPCPDzsllpkuWsgK6qIQBtJaTqjt2hP5MQ8GE0EjRlBJ/68Tl87dFlvO3Wva3XxhGdYX+rtmelcAEABeJziBgsaJ0oVBso15uYjrcCwQsun0RDpnjonL7dxGJamVO8mcOYmg1g6eGWfMwxeTlw9auBn36q1QDaA/TWiXpTRqUuXxo5AlmmzF5iIqJJem7CBeZ1wZ0XmBDgBb/HgsRT3+j5adgXoz2g8IVHl/KpXcU2ZxEAnpOGsHYCdQTwTHMOKUUeMbWgHtvbH2mowxqglSNw1ojXJg1pB8YD3YxAYQ56NhOqvQSHZvHXVg2FBT9GwsE2aSitZ0FtE5NR45GVXz++jD/91lN41XVz+MBtV3cfIE0BoO0Jcs00NDZ20XoDxUtHpzSM4MieMYgBn6Hv0FKq1JpnvlkzOC48zqwsdt/Sfd8L3s2kwGOf6flp9GaX9DqUBvBQIEgUqqg1ZMyPhQF/gJWnbUaOYOEoaw7Z8Zzu+676RWB8P3DfXzK7ix4Q1Yn0XCYwdR51wgiCYUaVPSINYe0xLAX3IlGWVb8cUwvq0T2sYamXvE1upSsQtDq/nTG/9mSxe2koWayqUg2AjkAw0XbsbDzUJg31YtQ3FdUfYv/A6Q2890sncNNl4/joXYf1u1n1uos185Ejoj1GwAMRzxEAzBTvxr3junkCWaZY4owA2LwZHOqG8ebu+3YeYQrC0b/p2Q0hqvO+qYZzlwIj4AmgXXwQtbBJTpqLDzIJSKfEDj4/cOu7gNXjwLkf9vQ0ETHQzQhUa1kd7Y8nG53kCDhV9gIjoBRYewxrkSuQLtb0rZQ7wUtIMwvunrOaZ7tHI2nIQY6g2mBDdFRbYCHKNhRqsliRdkI2ksWdjECr60faNf7O7mL+vo1GnOvHejYTT63m8Fv/dAyXTUr4+zfdgJCRtw0PBNomOjUQ7GCMwEbV0LoaCNq/f7dePomn1/Kq/YT2+FpTbg2s36zP+8JRlqMande//wW/x/7+k1/o6WkiOsniXI+Gc4CXAoFSEtYW6Qd9getl5jjamR/Q4vCvMNOu+/6yp6fSm1JmGundMALAO4EgtwKUkkjFrkS6VLeZI9jLbt3KQ0rXq9ZnCABCQdbI40QaUv3hOSPo7CXgjECxuo6JARCiP5xmo1BV6/oBGEpDADDbMcQ+XWLutWLAuRlZpzS0nCnjP//fhyCJAXzmrTdhxCy4aI3nOPJKRZY0yaRQGzmXdSWoaRkBwPIEAPCTM+3ykLpOjG8BI9BjAxz7fg6YvQ64v7ecop5ysOXSECHkLkLIE4QQmRByQ8d9HyCEnCaE/IwQ8grN71+p/O40IeT9dp+LN4nsGuPaX2zwF3j5EWbT25kf0CIgAs97B3D2B8yYziUiQgAlg2SxeY7AYSDwyrhKJVFcGL8G2XIdiXwVQsCn+q3oYmwPu3WbMM4ts9t4OyMghDi2os5q7SU4tGMmy2mW3/Gza+vzsefoZAS1BusDaJPEOAsISkzu02A2HkKiUFU7b9NmTXgWmIqJKNWaKFYbyJRqeMunH0Kp2sRn3nYjdoyGzR+sGwjWGFPw+ZV6eOsFkV93tfpKwcEdcYxGgl2+QwtJvmHUrBOD3vhkFtlnRy8/wMFzislngKe/6fqpIoK/a5phr9PJgN4ZweMAXgvgR9pfEkIOAvgVANcAeCWAvyGE+AkhfgB/DeA2AAcBvEE51hILqRKmY2KLim4GI1AbyUwiPQAceSv7UvdgMqU3/MTWdLKLlRGsnQRA0JhkH49zG0VMSIK5/XB0huWO3DKCjslkbacWA7rNXkbIaQ3nOKSp9hyB0kPAoWczwat+JrSMgEtDHfkBgElDlLa09VTJxJbDApOKHLWcKePt/3gMC8kS/u7NR3DVrI3PXDDMvhPa+nk+5wFsQbMzjyCRr2IqKnZdd7+P4Pn7J3C/YkvNsZgugRBgJw8EmzGMySw/oMXBO5g54v1/5TqnqDfN0LSoxCZ6CgSU0qcopT/TuesOAP9KKa1SSs8BOA3gJuXfaUrpWUppDcC/KsdaYjFVwu5xjee5GBv8BV54EJi8gu3kzBCKAzf+OvDU3a4tqtkXo9n2oc5V6hACPn1a75YReGUmwdpJYHwfYoqv1JlEwXpEJSEsYexaGur2Gfr/2zvz8LbKO99/XkmWHC9JnNVOnMQBskLsxIQlJGwBCkzL1kJpoGVJp0yhtNPOMLf0lgudAs+lMxTuLJQ7cGlhgBRoO2UIS1nasiSQlARCNickECcOsR3bJLGdeJP03j/ec6Qj6Ug6ks6xpOR8nseP7XOOpNdH8vt739/y/emYbclTPlXENWT454x3DcU1ODJrTqMHa2NiBPrj4rX+MVQXay6VlEJ9adAzdb674kP+0vw5P/9qA2ccb153YEp8LYGhP3Kywqh49vcMxKSOGll8wjj2HepnV2dUMqXl8z4mVpZG/2eGI5aoC81NPCn1dR4vLP5eThLV0Z4E0XvXkyqpxCJOxQgmAy2G3/dqx5IdT8veA31Rvx84v7INh9P7/YycfrMSA3v33xLPSameLzSksgZMVgPlgURhs97+YETnJoFMu5PpBIZhq2yk/1C0t24mtG6E6nlUaX7ols+PWMt8qarLXmaie5+aZEsS3R6Z9i3uNnMN6RpBUmo6Q/E7gsTmNF3ajmCc0RB4S1SQ2aQYTC8qa9fiBLnsCHRDsL29hzu+OIdLGjLsuBWvN9TTGul9YObiMGN/T39CoFhHjxMYs4daDsQvGIfJc1C7MOLmS0nDNUrjycx7EJknghAcTDJPJPYu0XeqSRUILJB25EKIN4DEzhXwYynlf2f9yulf9ybgJoCpU6fiPdQX9fuB877uiNBcCr+fkYoJMP8aWP8rJVMtw9Ev4t7QhcvhS7HB5XJ/VIFUd3+lFJzr71arECsfPiPD7Rr6r7+BQ3vh5lXWH9N3UPn5T74+sgsIS4u58FV1sPvd7LrX9bTGaAwZqQj4OHgkte6OkUhTmnjXkN5vuO8AjIpdA40cURLJjtPp0ncE8RpLE+bC+NkJr5u4IxhKrPq1SO3oMvw+D9cvmsZfn3lc5k9QMSES62GoT/0/6TsCbWXbNxRK6dve3zPAadPNd+RTx5RRWzWCVTs7+caiOkDVEJx+vOF6f6Wq7A0Oqr4VdqMLzZ31D9auLymF078Nf/wp3D0h9TxxwgXw9d/GHCo36WbY0x/E5xEEfNmv69POIlLK87N43s8AYx5VrXaMFMfjX/cR4BGA+vmNslsaUkfB+SBQvNCcFc79n2pFKcMqX9/45fGqialpperBG4dRgVRf+famEpzLtBeBznAGi0NDsOttVWjT8TGMn2ntce2b1ffq+pjJf4yVFMiqaSoFtO+AqeskJQf3wCjzfrgVpb5IRooVdBdPbLBYry7uNHcNlZYkyFhEBefiJrHrV6rPVRxjyv34vR7auvsZCIboHQgm12dKw6iyEj74XxdkH4SsmAi9f1I/x8VfynTp9RStWAeCIQ4eGYopJjMihGDJCeN4aVMrobAkGA7T2t0fzSyE2Gp6X4afByvEC81Z4bRvK89AcCBxntC/mt9RKemhIbUD1DDrSaALzqVr35kKp3oWvwCsEEI8AEwCZgB/AQQwQwgxHWUAvgZck+7JBrUMiNg3WFvZhsPgccDDFS80Z4WKCXD+XamvER61GohzDUQMgSEI1NM/FMlhT2CgJ3O3EAzvjmDfBmUEALathPF/b+1x+iqyup4xAYMhSKY8aiSSQrorM0MQHISO7TDjAtPTFSZaUKno7h+ixCsoLTF8NvWist6OJDGCxKyhzsMqayZhskyyExRCMGFkgPZD/eZNYzIkl0wUKiaoBctQn6GqOOoaAhJqZ4zoyqfJXEOg4gTPvN/Cps8OMWpECVIS60I2Ku5mujCwQrzQnBX85WrRmIox05Uh6NgO1dHYQ7R3Sew8kUt8AHJPH71CCLEXWAS8JIR4FUBKuQV4DtgK/AH4jpQyJKUMArcCrwJNwHPatSkZ1PzmU8fGxQggOtHYjZnQnB1UN6jv+mSnYdaToKc/jQR1NjsCfwUE+5Uf0ml2a+6gMcdnJsPRtkn5USsnMsLvjWx5La1sR2sppJnGCTq2qVTh6nmmpytKM08fHTWiJHaVpu8IDjSDDJnvCAaCMX7zrt5BxqXLlopDry7OVmfINowtK+MC8WUWFF0jNQRJgsUAZ2huoNU7OyNutRgXstOyKmZCc3ZQXa++t22MOWy6IxgIJl8wWiTXrKHfSylrpZQBKeVEKeWFhnP3SimPl1LOklK+Yjj+spRypnbuXiuvMxgKU+IVEf8n4KzwXDKhOTvQJ5r4N9if2JOgJ12wOJsdQcDhfwwjzatV1lXjdarG4mBL+seAChTX1Ed+1V1l1mIEuiFozmysujtqYhJDoFV+pwtu6sToDOno+f9dO9T30rhg8YjECuau3oHY1FELTBylWlbmojNkC0ZDELcjMMYIkrHfRF4inrEVAebWjGTVjs7EYjJwdp5IJjRnB2OPh5Iy9b9goNwka6g71YLRIkVRWTwYDDNp9IhYhUMnhef2ZBEfsErlRPUPYmFHkLIhdS47AnDeEISCsOc9qFui1BcBtr2Y/nHBQbU6N6zM9YCxpZVtoFJNuJkagrbNqodvEleg/j5Y1dDv7g9SOSLOEOg7gs6P1feEHYEmPGfIHOo6PJgYH0iDviPoStYrYLgwFpX1tKoaD8346TGClDsCE8E5M5bMGMf63QfY0d6L3+uJ9GUADPOEAymkqYTmcsXjVTuNhHki8b719gejUibZvlxOjx4mlNhcWexBJ7tttaxNLjRnB9X1SS297jOVUqYJFue4I3A6YNz2kTI20xaryXXCidbcQx1NCS4avTLW8so2Gznqto0wcW5MHwIjuq/cqntI7Qji3jt/ucr06typfjepI4BYKequ3sHEjKE0VI8spX8ozO4u5TZNW3/hFOVxhqCyJuJqNVvZxtPRM4AQaYQGUXGCwVCYlR/tY3JV3ILRyc+71UKybKmuV4bAkEYadakZPAcDQ7nFcigSQzAUCkdlZXWcXNnuWQOTG82F5uygep5a9RpUMuMtvZ5jnVRIqtB3BM2r1fe6Jer7nEtUWme6Tk2RQHFD5JA+kaWbEKIPqMtsRyClWt2lKAhK1VfajBjl0ZgnGgtd5oZgVJzwnJRS0xnKcEeg1RI0tapFUjaCc7ZQPg4QqpraUEwGhmBxivvZ0dPP2PIAPm/qaeqUuir8Xg9dhwejEjQ6Ts8TqYTmcqV6ngq2Gz7LXo9gREmsAqnKGspjjGC4CIZlrN8PnNsRDPVB60fOWXlQ/m8ZUqtfjfisoZT6IaEhlY+eSS8CneEyBLtXqyCx5hNW7iEJ219O/bjWjWrVPCaat64bAsv5nmSDKwAAIABJREFU8KOnqdoFqwHx7s9UFk+SQDEYupRZNQR9QfMCH72WAEyDxRB1DR0eDDEQDGfuGooYgm5GlvooSTOROoa3RBXR6TsCg4ZTeSD9jiC+M1kyyvw+Gqcpl1PiPKG5hpzaETg9T4Cpe0g3oGk9BxYpCkMAJLqG/A69wbrQnBN+Px09I8DgHgr4PHhEdIVkSXAu2zoCcNY1FA6p1b++GwDl76yant491LZJpcsZUoLPnT2eLy+YjN9qwUxVHYSDURG5dLTpdQvJDUFlBq4hKSXdWtZQAkbZ6PjK4rjmNEmLydKgJ1Xs6jqcv/iATsVE6GlXyq5mO4IUMZdU8hLx6FXGifOEvvCxecFoRWguVybMVWqtcYklZYZU5oFgmKGQzElnCIrJECTbEdj9BlsVmsuFqunKkBneYCFEjP5KSiGpAb0XQWXmrz0cO4K2TSqGYTQEQqhdwadvqRoKM8JhzRDETshLZ0/kgasziNdkKkfdrq24Jp6Y9JJol7L0hmAgGGYwFE5QzASitQS+0gQpi0hPAm1H0JmsmCwN+uQprVZjO0nFBJUlFeyL7g5RCx+vR6TsUra/p5/xFjOmzp6p4hEzJlTEnvD5weu333PgdHwA1Odj3MzEeKKhd0lkwXgsxAimjS1LfIP9Dq1sdaE5J4pPdDweteqN3/IZLH1PqmYT2QrOgeG+OSjEtVuLD0xbHHt8zqVqt7XjNfPHHWxWhr263vy8VTKVo27bpIxzCsOqBzetGAJTnaHIE2k7gji3EKiiNSGij9d3BOMyTB8N+LyRnUDeagh0KiZCp5Yua9gRCCFS9i0OhyWdvYOWdwTzakfx6vfPYunsCYknnehbbFVoLldq6k3mCW/CPHFMuIZGlpZEfIoR/OWAsHdlm6nQXC5U1yuXhKFJRXnAG/GZ9qaKEWQrQQ2GnZRDhXigAsVVdQlaOkw+WU0GTS+YPy4SKE7uorHEyFq1pba6I2jbHFO9aYa+M7PiGjLVGdJJYQg8HkFlwBdRLu06nN2OAIikUBbEjkDX0ImT907Vt/jzI4OEwjJlDUE8s6orzdtmOiE8l4nQXC5U10PPvpjez8YdQcRzkM+CsrwihCZFbeMbnKnQXC5Uz1M5yJ9/GjlUbuhJ4EhTGlBFKsLjnGsoHFY7AqNbSMfjUX2ed7wBgya6Pa0b1QQ+wVKLiuR4fUozyEp18UCveg/S7ELMKjqTcSiiM5TCNWRiCECXoo7dEWTj56/WVtL5jxEYVuiVsdqVqfoW7+82b1GZFXY3sdKF5oZrngCVwKJhDBZHkkqOhR1BUvwV9sYIshGay5aaxBJy1cdVe4NTWfpcdgR632KngsX7tyhjOs3EEICKEwT74JM/Jp5r2wTjZymFxlyxmkK6fysg027xS7weSks81lxDqXYEerA4rqpYx9icprN3kMrS7NpMVo9S8Ye81RDo6NXFYL4jSJI1pPciTldMZolApb3zRDZCc9kSUSKIuofKDd0MUy4YM6C4DYHdSpp71mQuNJct4+eo/gWGQFB5wEvvgB4ESuH7i+wIskgfBeVWc6LSEgz1A4vNz09brFbDZtlDbRtzjw/oWDUEEXdUel9vRSBRHdQM0+5kOilcQ6ALz6nXSOhVnAF65lC2yqO2oe8ISkeBPzbhI1XfYivyEpaxe57IRmguW8rGwKipsQvGGM+BFks8Zl1DYL+S5p41artnt9CcGT4/TJid8AbrO4Le/iBlfm9slaROLjsC0O6bQzGC3atg9FT1ZYbXB7O+CNv/oOQkdHo7VK55rvEBnappSu453QTQtklNUqPSFwVVBKw1XE8dLNZdQ+l3BKqqOLsVffUoZUAKZkdg0v4zVd9ivdWm1WBxSpyYJ5wQmktGTX3CglHvZmhHm0oodkNgp6Xv3a+ki4dju6ejS01oJeRlMVlDyXXa6T+kdHG8Wa4CnOpJEA6rHUEyt5DOnEtUCmyzodW1nsJZY+OOANJnDrVvVkJzFoy/1S5l3am26+XjlM9aH18cMTGCwwNZBYoBpo9TSQGT4ytth5sUhiBV1lBHzwCVpb5oj/JcsPPz7qTQXDKq56lqdG3xVuaPdjNMGSPIoC+ywyFvh/FXwhGLKYI9bWoCTcYubVIajgCQTnU9bHhajW1kjbbijGYDJBecy7IpjY5TPQk6tkHf58ndQjrHnaPG0LQSTtD6HukrHrvS8UbXqe8HdievDwiHVNCv8XpLT1kRsCZF3d03RMDnMZ/EfAG49X3TNpOg7wj0grJBTqnLLo35lLoqXvvBWcycmEWtiZ2UjlYuULMdQYqsoVQtKjMmkyZWhzvhSFfy8107nROaS0Z1PSDVZ3XKqYYG9qrxUGmJx7x6fN0vLb9EcRuCgMXG1Adb4F8alKxDKkrKoKYh9TV2YgwYj6yhzO+LaAx19w9RYeZakFJV7eaSWeOvUBIMdpOsfiCeklKY8QXY9hJ88QEl9Na2Sbln7KrfsFJU9vkuGDpiKT4AKkbw2cG+tNcl1RnSGZk4KUZOjVC7jsFgmM+PDGYsQa0jhMi/EQCVKbbkBzDtjIRT6bKGbAkUQzR9NF370qE++NcFUddrKobTEOjzROtHMOXUGJ2mnv4hKpLFBzLoAVLchsDqynb3amUELrovNp0tnqrpzgnNmaGvfts2wswLDZY+SO9AEmnZjm3w+Sew6DvZv65TDb2bV6kc/iRujxjmXAJb/ksF3qadYW+gGJRB8VemNgSRimKrhsBqjCB7WWA9rrDn8yNIScaCcwXJ0h+bHtazhqSUCY13OnoHaKg1j6NkjL9CZfkMHdHqj5Kw70NlBM68TSnRJqOiOmlLU0cYOVklF2iJDRUGXbKeZBLUfQdUu0uLFLchsLoj2LNG5dyfelNSmeG8UDpSGR/NLRLVaA/R0x+MbcSj07QSEDD7i9m/rhOuISmVwT1+qbVg+4wLlNR300q1C+vcASd+2b7xCJFejrptE3h8pk3gzbAeI0iiM2QBfSfxaYd6fzLVGSomygJeQpqv2+hGk1JaFpyzhFFfK5Uh0PuQnH6LUoktFITQClD1eSKqhJu0i+HHryq9LYsUd7DYXwmhwdjsEzNa1qpUr0IyAjo10TfYaOn1htQJNK2EKacmFOdkhBPB4s6PldxwOrdQZAyVcPy56u9p3wJI+wLFOulSSNs2KzkRi3ULFYESSzGCQ31pXEMp0Fd3uzpVYDDbYHExkKwnQe9AkL6hkD0ZQ2C9iVXLWhg7o7CMgE5NPbRvhdAQFYYFY9JYYtNKqJxk+emL2xBYabvYdxD2Nw2vTy8Tquepyar/UEwf196BYKLv70CzMhqzv5Tba/orVFGXnX2Lm7X+xGYVxcmYcwkcaoEPn1S/25U6qjN6mgoWJ8ueaN+c0WtWlvoYDIUZCKaONZm2qbSIbkB0Q3BUuIaSkKwnga01BGDYEaTw/UupDIET7WntoLoeQgPQuSN2njDLLhw8DDv/CHOszxPFbQisCKjtXQdItYouRCLN7DdHmtP09AfNLf22l9T3DN5gU/T7NmRjLcHu1SozxNBHIC0zL1aSEh8+rbJLLOTyZ0RVnTJ4Zs1wjnyuZIQzyFKy2qWsuz9orjxqAd2l9Km+IziKXUPJehLo8hK2BYutCFR27lB+9eFMH88EQzP7qOcgRE//UKIw5c4/qs99BgvG4jYEAQtbvpY1arKZvHB4xpQphswhfausF9MkGIKmlWriymSyNcPungRSavUDizMrxisfq1JNZUitzO0u5EuVOZRBRbFOtEtZ8h2B3ovAjh2B1yOyjjUUA8l6EnT02qgzBNY8B8MpL5MN42ao2qHWjZH7dmQwSM+AyY5g24squGzVTUvRGwILE9qeNeqfXb+20KiYqKpN2zZFJpq2bqWzEmMIeverv0VvBJ8Ldvck6PoEetvS1w+YMedS9d2JtN1UctS6IZho3TWk/8P1DAwlvaZvKEQwLHOOEXT0DDCm3G+upnmUENkRDMTvCNTn3zbXkJUmVnvWwogxasItRDxelcnUtjFy30w9B8FBVbU/668yUkYtbkMQCQIlcQ2FgvDZ+sK18hDNCGjdGHENtR1S/wgxMYJtLwEy9/gA2N/LYbcWH0hXUWzGnEuUZtL0s+0ZixFd5sJsR9C+WaUBVoy3/HRWpKh1naBsV/Llfh/63J+tvESxkHRH0DOA3+fJ2r2WQCDNPAFR+fnhkJfJFi1zKOAVeD2Cjp4BpIxbMDa/o6r2M1wwFrchSLcjaN+kcocLNT6gU1MPHU2Ue9XKqF1bEcWkhW17UaWapuiiZRkrW+VMaF4N5ROyW01VVsPtu2HmF+wZi5GSEWqyN5OjbsssUAzRyf21re2Ew+YB6IjyaJauIY9HRHy+2QrOFQu6K7QvPkbQM8D4ikBCbUHWpJsnDnepLmrFME/0H0IcaqHc743OE8YFY9NK1TDnuHMzeuriNgTpXBx7tHZyhZoxpFM9D8JByg7uBExcQ/2HVIvHOV+yZ8Vip2tIrx+oyzA+YMTJVZhZCmlwUBXmZRAfAJhbM5IvL5jMY6t2cdOT6yKTvpFDuuBcDqtZ/bFHc+oopN4R2JY6CmpihOSf95ZimSeizezLA77EeSIcUp6DGRdkLOVe3IYgkMb317JWVboOZxVgNmiZQ779mygt8dCuuYYifUg/fk21eNT96bmS7r5lwoFdKvsmg8DUsGJmCDq3q/uZoa6RxyP4+Vcb+MdLT+TN7R1c9u+r2d4W625IqTxqEf2xR3PGEEQLoxJiBHbqDIGSuUjVg6NlrdJDmrTAvtd0gglzlfy1FifQ54mI52Dv+3B4f1ZxxOI2BJGVbRLfX8vawt/ugcoCKimPZA61R7KGtMmk6QXl4rAr80mvrrSjJ0Gk/0AW8YHhoGqaMlTGosNIxlDmBWxCCK4/o45f33Q6vQNBrvjFal7cuC9yPmVTGotEDMFRviMYUWK+I9jfM2BfoFgnVROrlrUqWaEkz0qt6fCXRZrZl/u9kXkiIjHRtBK8fqXjlSHFbQh8fvWHm1n6gy1qAij07R7ENLMvD/gIaf7nilKfEsLa+YaSlPDY9HZFDKgNdQS7V0PZWMsyDcNOVR0gVeGaTttmlYqXQwOiU+rG8NJ3lzC3ZiS3rviQe1/aSjAUjgSLs9Uagqhr6GguJgPwegQjSrwxdQQDwRAHjwzZuyOA5NX0wQH47IPimCdAuZHj54lAiXLRNq1USRdZKBPnNLMIIf5ZCLFNCLFRCPF7IcRow7kfCSF2CiG2CyEuNBy/SDu2Uwhxey6vDyTXzdH9foVaIBJPdb16g0uUv1wIKPd74ZM/qYB3rkVkRvzlgLDHNZRN/cBwMlpLITW6h9o3qVS8HCVHJowsZcW3Tuf6RdN49J1dfP2xtZGKYFt2BEe5awhi++9CtIbGtmIynWTzROtHqmK3mOaJ7r2M90YXcZWlPrXLPbg76/TyXJeYrwMnSSnrgY+BHwEIIeYCXwNOBC4CfiGE8AohvMBDwMXAXGCZdm32JBOea1mr3C126ds7TU09DPZyvK8DUDnrQghoelF10Ko7077X0vsW5xosPrAbDu0pXLcQJBaVSan+aWz6XPh9Hv7xspN44KsNfLjnII+/20yZ32uuD28R3Ygc7a4hUE1WjDsCWzuTGQkkaWBfbAtGrQB1lvw0cqii1KeyCoVH1Q9kQU6GQEr5mpRSN+drAD0qexnwjJRyQEq5C9gJnKp97ZRSfiqlHASe0a7NHn+SN3jPGqg9OaOiiryipTLOEs2AtioMDcH2l5UUQ7bdyJLhL7em3JoKXfLiuHNyHY1zVNYo96FuCLr3KSkBm3WNvtxYy+9uPoPaqhFMNFONzYCRx0j6KCT2LbZdZ0jHn2TBuGeNWixUTrT39ZxCi2vVBQ2GwO9TbqGpizKqizFiZ4xgOfCK9vNkwOCUZa92LNnx7AmYBIEGelTBUCEXksUzYS54fMwIqTe4IuBT/vf+g/ZUE8eTa08CKZVY3OSTYfws+8ZlNx6PKizTq4sjgWKbBe6AkyaP4tXvn8UzN+X2uTtp8kjqxpbZvyouQOL7FkcNgQMxgvh5QheaK6Z5omwMjKxl6qBKNa8I+PAc+BT2b81pnki7XBZCvAGYaR7/WEr539o1PwaCwNNZjyTxdW8CbgKYOjVJI3RQlr7vQOyxz9arRhSFqiRohi8A42dTd/gTQPP7Nf1OBTWPX2r/6+XawP6zD9SH70v/x74xOYUxhTTSjMaGwjwTygO+iARAtpw3ZyLnzSmSFWqOlPm9MT0eOrr7EQLG2F1VbZY+emCXkk4vpnkCoKaemj1NgD5PaJ3IcuhRkvYTK6U8P9V5IcQNwJeA86SM6P1+BhilJGu1Y6Q4Hv+6jwCPACxcuDB5F+ZARWxGCGiFZEL1ICgmqucxectrAFQGPFpxyPkqbcxukvlMrfLhf6rWnid9xb4xOcXoaZoKLSpjqGp6tJbCJa+U+30RtVFQgnNjywP4coixmGK2A95TZPEBnep5VG3/A6UMUBGoUPGBmvlRSZUsyGnpIoS4CPgfwNlSyiOGUy8AK4QQDwCTgBnAXwABzBBCTEcZgK8B1+QyBtMJrWWNcrWUjsrpqYed6noqP/o14znISRyAnlaY7YBbCFSMoNvUBqdn8DBs+h3MvTyrVLVMGBoaYu/evfT392f/JLVXw9gLYesWmHY9HFcCTU32DbLAKS0tpba2lpKSwlMyje9bbGtnMiP+Sgj2K/0xPW7YskbpXI2fY//rOUl1PR7CzBYtjC3xqkKypXfk9JS5RlL/HQgAr2u6IGuklN+WUm4RQjwHbEW5jL4jpeocL4S4FXgV8AK/lFJuyWkE/spYSx8OQcv7UH9VTk+bF7SMgBM9zSzs26vaKM68MM2DsiRVpWU6tjyv/K2N37B3TCbs3buXyspK6urqstee6Tuo3ABjp0HXoNI3qkzeQP5oQkpJV1cXe/fuZfr06fkeTgLl/sQYgSOxEaPw3Igq9fOetTDlFPvqc4YLwzwxM6yyDHNVHcjJEEgpT0hx7l7gXpPjLwMv5/K6MehbPilVWuT+JvVmF1MASEcLYM4VzczrfhemnwUjbGrgHU8ghxjBh0/C2BNUloLD9Pf352YEQGUNgdJsAvA54GorUIQQjB07lo6OjnwPxZSyQHzWUD9zahxw2xmF50ZUqcVBR1NxuDbjGTWFIf8o5gZ3c3L/AdVeM8eEjSIzhSb4K1RgeEjzTEUaTBSBtEQ8paPoLavlMu+7jBnY60y2kE62dQSdO2DPe7DgG8NWRJazCqVPNwQH1fcMBbmKHdtUPB2g3O9jIBgmGAoTCks6ewftLyaDRKHFve+r78U4TwjBkaq5nO7Zyoy+j2yZJ4rfEMRLzO5Zq5q96IVERUbv6DnM8uxFImBW9lkAaQlUKuMZTt1/N4EPn1Qd3xqWOTMuJ/D41JhDg+q7N/+FWjfccAO//e1vbXmu5uZmVqxYYctzDTeRbltDIQ4cGSQUlvbXEECi0OIerXNhbYF2LkzD4PgTOd7TipeQLaoDxW8IIs1ptDe4ZU3hN5hIQd84VfHaVTXf2SIXPcNATz2zQmgINqyAmRcVTwGOjr4rKBlR0J+NUChDw0xxGwJjlzI9e8iZYHGcQGXLWuWK1QUYi4ywVljW458Ikxpzfr7iNwSRHUEPdLfCwT3Flw5mIDRB5bd3TnEoSKwz76tKcfHl21Qjdyt8/KrKux6GILHteLXJJQuFyXvvvZeZM2eyZMkSli1bxv33388555zDunUqJbWzs5O6ujpATcpnnnkmjY2NNDY28u677wIqaHvrrbcya9Yszj//fPbv3x95/rq6On74wx/S2NjIb37zGx599FFOOeUUGhoa+MpXvsKRI8rtecMNN/C9732PM844g+OOOy6yo7j99tt55513mD9/Pg8++GC2dygvGHsS7O/RWlQ6Eiw2eA5CQ1rnwuKdJ7yTlCHYM3GpLQubItFfSIHR99fSrH4uFiVBE6ad8kXWbl/OSefd5OwLeX1w2UPwyDnwhx/Bl/8j/WM+fFLJYZ9wgbNjS8I/rtzC1n3d2T04NKAmAF8PeKJ1J3MnjeSuS5IXl61fv55nnnmGDRs2EAwGaWxs5OSTT056/YQJE3j99dcpLS1lx44dLFu2jHXr1vH73/+e7du3s3XrVtrb25k7dy7Lly+PPG7s2LF88MEHAHR1dfGtb30LgDvuuIPHHnuM7373uwC0trayatUqtm3bxqWXXsqVV17Jfffdx/3338+LL76Y3b3JI3qXsiMDoUhV8fgKB1xDxnmiTetcWGyFZAbG1tXzl+m3ULf0m7Y8X/EbAqOlb1kLvtKsdOYLhZJAGad9c5hWddXzYMnfwdv/pLInUrWL7N4HO16DxX9bPPpNRoS2+RWZKY6+8847XHHFFZSVqUyjSy9NnaY3NDTErbfeyoYNG/B6vXz88ccAvP322yxbtgyv18ukSZNYujS2Wvzqq6+O/Lx582buuOMODh48SG9vLxdeGN0dXn755Xg8HubOnUt7e3tGf0shUhaI7ggcE5yD2BhBRGiueBeMwuPl1Ov/t23PV4T/0XHoMYKBHvUGTz456g92Sc9Zt6k4wYvfh1veS16Et2GFys5akD+3UKqVe1rCIZU+OqLKnq20z0c4HAaIKXZ78MEHmThxIh999BHhcJjSUmur2/LyqK/6hhtu4Pnnn6ehoYHHH3+cN998M3IuEIhOktFC/uIlsiPQDEFlqY/SktzkwU3RdwQD3UqHbNQUGJWbzNnRxNETIzjcobTFizEdLJ/4AspF1NMKr99pfk04DB8+BdOW5NTMJa94vEqwK0MjcNZZZ/H888/T19dHT08PK1eq4HpdXR3r168HiMn+OXToEDU1NXg8Hp588slI8Pess87i2WefJRQK0drayp///Oekr9nT00NNTQ1DQ0M8/XR6+a7Kykp6emzoNpcHyvUdwUDI/haVRnwBlT022KsVkrnzhJHiNwS6pd/1NoSDRb3dyxu1J8Oi78D6x+HTtxLP716tKnOLMUicI42NjVx99dU0NDRw8cUXc8opSr/qtttu4+GHH2bBggV0dnZGrr/lllt44oknaGhoYNu2bZGV/hVXXMGMGTOYO3cu1113HYsWJS/Gu/vuuznttNNYvHgxs2en7/xWX1+P1+uloaGhCIPF0R2BkpdwqMZD78Gxfxv07HPniThEMWwvFy5cKPUMjQTCYfhplWpCM3QY/scutfJzyYyhPnh4sTKmt7wXm1b3XzfB9lfg77c7I4CXgqamJubMKRwtmJ/85CdUVFRw22235XsoGVFo91Hn0JEhGn76Gnd+aS6Pv9vM/Cmj+ddlDjWRf/AklSE3dBj+5m2VNXeUI4RYL6VMWyxR/DsCj0dZ+qHDqrGzawSyo2QEXPpvSrf/T/dEj/cdhK3/DfOuHHYj4HL0M0IvKNNiBI65hkArojysFo0TnJEhL1aKP1gMUbmEIs4LLgjqFsMp34I1Dytl0amnwabfKNXGPAaJC4mf/OQn+R7CUYXf58Hv9bC/Z4C+oZCzzXh0N3LtwuLMfHOQ4t8RQDRgXMT1AwXD+XfBqFp44VYY6le1AxNPgkkObdddjnnKAl52dSoBRMdiBODOEyk4OgyBbundAFDuBCrhkn+Bzo/hd99UmViN1xW0LINLcVNW4qW5SxkCRwTndCLzhOs5iOfoMASBSigbW7ypjYXGCefB/K+rzkfeAMwrwt4OLkVDWcDHZwf6AId0hnQClajOhcUpNOckR4ejbP61qqDMXbXax4X3wKd/Vj0R3AC8i4OU+72EteRFR11DJ35ZuT2LrXPhMHCUGIIikkQuFkZUwXfWRsXaXBI455xzuP/++1m40F1h5oJeS+D3eRg5wsEpacb56sslgaPDELg4g9vgPa+EQiG8XgfkFgoMvbp4QmWgoJvoHM0cHTECFxcHaW5uZvbs2Vx77bXMmTOHK6+8MiINrXPzzTezcOFCTjzxRO666y4A/vSnP3H55ZdHrnn99de54oorAHjttddYtGgRjY2NXHXVVfT2qn4aRknqZ599lvnz50e+vF4vu3fvHqa/evjQdwSOBopdUuLuCFyKh1duVxLCdlI9Dy6+L+1l27dv57HHHmPx4sUsX76cX/ziFzHn7733XsaMGUMoFOK8885j48aNnHvuudxyyy10dHQwfvx4fvWrX7F8+XI6Ozu55557eOONNygvL+dnP/sZDzzwAHfeqbSejJLU11xzDQAPPfQQb731FtOmTbP37y8AjDsCl/zg7ghcXCwwZcoUFi9eDMDXv/51Vq1aFXP+ueeeo7GxkQULFrBlyxa2bt2KEIJvfOMbPPXUUxw8eJD33nuPiy++mDVr1rB161YWL17M/PnzeeKJJ2JW+kZJaoDVq1fz6KOP8stf/tL5PzQP6DsCRwPFLilxdwQuxYOFlbtTxPuujb/v2rWL+++/n/fff5+qqipuuOGGiDT1jTfeyCWXXEJpaSlXXXUVPp8PKSUXXHABv/71r01fyyhJ3drayje/+U1eeOEFKioqHPjL8k+5390R5Bt3R+DiYoE9e/bw3nvvAbBixQqWLFkSOdfd3U15eTmjRo2ivb2dV155JXJu0qRJTJo0iXvuuYcbb7wRgNNPP53Vq1ezc+dOAA4fPhxpYGNkaGiIq666ip/97GfMnDnTyT8vr5QF3BhBvnENgYuLBWbNmsVDDz3EnDlzOHDgADfffHPkXENDAwsWLGD27Nlcc801EReSzrXXXsuUKVMi6p/jx4/n8ccfZ9myZdTX17No0SK2bduW8Jrvvvsu69at46677ooEjPft2+fsH5oHIjsCJ3WGXFLiuoZcXCzg8/l46qmnYo4ZO4c9/vjjSR+7atWqSA9inaVLl/L+++8nXNvc3Bz5+eyzz47pfna04sYI8o9rCFxcHOTkk0+mvLycn//85/keSsFy9qzx3HzO8cyudutW8oW1wUzPAAAGQUlEQVRrCFxc0lBXV8fmzZuzeqzeztIlOeMqAvzwovSd2Fycw40RuLi4uBzj5GQIhBB3CyE2CiE2CCFeE0JM0o4LIcS/CiF2aucbDY+5XgixQ/u6Ptc/wOXopxjaqRYy7v1zSUeuO4J/llLWSynnAy8Cd2rHLwZmaF83AQ8DCCHGAHcBpwGnAncJIapyHIPLUUxpaSldXV3uZJYlUkq6urooLXUDsS7JySlGIKXsNvxaDuj/rZcB/ynVf+8aIcRoIUQNcA7wupTycwAhxOvARYB5ZY3LMU9tbS179+6lo6Mj30MpWkpLS6mtrc33MFwKmJyDxUKIe4HrgEPAudrhyUCL4bK92rFkx82e9ybUboKpU6fmOkyXIqWkpITp06fnexguLkc1aV1DQog3hBCbTb4uA5BS/lhKOQV4GrjVroFJKR+RUi6UUi4cP368XU/r4uLi4hJH2h2BlNJqJ4engZdRMYDPgCmGc7Xasc9Q7iHj8TctPr+Li4uLiwPkmjU0w/DrZYBeJ/8CcJ2WPXQ6cEhK2Qq8CnxBCFGlBYm/oB1zcXFxcckTucYI7hNCzALCwG7g29rxl4G/AnYCR4AbAaSUnwsh7gb02vqf6oHjVKxfv75XCLE9x7EOF+OAznwPwiLuWJ3BHaszuGPNHEsNLEQxpOUJIdZJKYuiMaw7Vmdwx+oM7lidoZjGCm5lsYuLi8sxj2sIXFxcXI5xisUQPJLvAWSAO1ZncMfqDO5YnaGYxlocMQIXFxcXF+colh2Bi4uLi4tDFIwh0PSIfiuE2CaEaBJCLNKOf1c7tkUI8U+G63+kqZtuF0JcWKhjFULUCSH6NIXWDUKI/5vvsQohnjWMp1kIscFwfV7uaybjLNB7Ol8IsUYbzzohxKnatUmVeAtwrOcIIQ4Z7uud6Z5/GMbaIIR4TwixSQixUggx0nB9Qc0Bycaa78+rJaSUBfEFPAH8tfazHxiN0i56Awhoxydo3+cCHwEBYDrwCeAt0LHWAZsL6b7Gnf85cGe+72uG4yy4ewq8BlysHfsr4E3Dz68AAjgdWFvAYz0HeLHA7uv7wNnaseXA3fn+rGYx1rx+Xq18FcSOQAgxCjgLeAxASjkopTwI3AzcJ6Uc0I7v1x5yGfCMlHJASrkLVbh2aoGONW+kGKt+XgBfJar+mpf7msU480aKsUpAX62OAvQu8xElXinlGkBX4i3EseaNFGOdCbytXfY68BXt50KcA5KNteApCEOAsugdwK+EEB8KIf6fEKIcdWPPFEKsFUK8JYQ4RbvesoppAYwVYLp27VtCiDOHaZypxqpzJtAupdyh/Z6v+5rpOKHw7un3gX8WQrQA9wM/0q4vxM9qsrECLBJCfCSEeEUIceIwjTPVWLegJn2Aq4hqmBXifU02Vsjf59UShWIIfEAj8LCUcgFwGLhdOz4GtaX+B+A5bXWYTzIdayswVbv274AVRj9nnsaqs4wCWGWT+TgL8Z7eDPxAKiXeH6CtFvNMpmP9AJgmpWwA/g14vgDGuhy4RQixHqgEBodxTMnIdKz5/LxaI9++Kc2HVg00G34/E3gJ+ANwruH4J8B41ArmR4bjrwKLCnGsJo9/E1iYz7FqP/uAdqDWcD4v9zXTcRbiPUX149DTsQXQrf38H8Ayw/XbgZpCHKvJ45uBcfkca9w1M4G/aD8X3ByQbKwmjx+2z6vVr4LYEUgp24AWoQTsAM4DtqJWJOcCCCFmooIynSh1068JIQJCiOmolph/KcSxCiHGCyG82vHjtLF+muexApwPbJNS7jU8JC/3NdNxFug93QecrR1bCuhurGRKvAU3ViFEtb7jFiqTyAN05XOsQogJ2ng8wB2AnnFTcHNAsrHm8/NqmXxbIoOVnA+sAzaiJtUq1GT6FLAZtW1darj+x6hV93a0DIhCHCsqYLQF2KAdvyTfY9WOPw582+T6vNzXTMZZiPcUWAKsR2WyrAVO1q4VwEPaPd3EMK8EMxzrrdp9/QhYA5xRAGP9W+Bj7es+tJ1MPj+rmY41359XK19uZbGLi4vLMU5BuIZcXFxcXPKHawhcXFxcjnFcQ+Di4uJyjOMaAhcXF5djHNcQuLi4uBzjuIbAxcXF5RjHNQQuLi4uxziuIXBxcXE5xvn/YJvR88uLcvkAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "features_of_players[660:700].plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Removing playerguid before training. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "features_of_players[328800:328816]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Training\n", "\n", "***\n", "\n", "Next, we configure a SageMaker training job to train the Random Cut Forest (RCF) algorithm on the taxi cab data." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Hyperparameters\n", "\n", "Particular to a SageMaker RCF training job are the following hyperparameters:\n", "\n", "* **`num_samples_per_tree`** - the number randomly sampled data points sent to each tree. As a general rule, `1/num_samples_per_tree` should approximate the the estimated ratio of anomalies to normal points in the dataset.\n", "* **`num_trees`** - the number of trees to create in the forest. Each tree learns a separate model from different samples of data. The full forest model uses the mean predicted anomaly score from each constituent tree.\n", "* **`feature_dim`** - the dimension of each data point.\n", "\n", "In addition to these RCF model hyperparameters, we provide additional parameters defining things like the EC2 instance type on which training will run, the S3 bucket containing the data, and the AWS access role. Note that,\n", "\n", "* Recommended instance type: `ml.m4`, `ml.c4`, or `ml.c5`\n", "* Current limitations:\n", " * The RCF algorithm does not take advantage of GPU hardware." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[ 199.0827 49.08284 3. 25. 7. ]\n", " [ 178.2321 146.2268 3. 21. 7. ]\n", " [ 186.6547 -115.9491 1. 21. 3. ]\n", " ...\n", " [-107.2072 107.2072 2. 18. 1. ]\n", " [-140.7918 125. 2. 18. 1. ]\n", " [-150.0805 117.0529 2. 0. 1. ]]\n" ] } ], "source": [ "print(player_data.values.reshape(-1,5))\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2020-04-22 04:44:48 Starting - Starting the training job...\n", "2020-04-22 04:44:50 Starting - Launching requested ML instances......\n", "2020-04-22 04:45:52 Starting - Preparing the instances for training...\n", "2020-04-22 04:46:39 Downloading - Downloading input data............\n", "2020-04-22 04:48:37 Training - Downloading the training image..\u001b[34mDocker entrypoint called with argument(s): train\u001b[0m\n", "\u001b[34m/opt/amazon/lib/python2.7/site-packages/scipy/_lib/_numpy_compat.py:10: DeprecationWarning: Importing from numpy.testing.nosetester is deprecated, import from numpy.testing instead.\n", " from numpy.testing.nosetester import import_nose\u001b[0m\n", "\u001b[34m/opt/amazon/lib/python2.7/site-packages/scipy/stats/morestats.py:12: DeprecationWarning: Importing from numpy.testing.decorators is deprecated, import from numpy.testing instead.\n", " from numpy.testing.decorators import setastest\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Reading default configuration from /opt/amazon/lib/python2.7/site-packages/algorithm/resources/default-conf.json: {u'_ftp_port': 8999, u'num_samples_per_tree': 256, u'_tuning_objective_metric': u'', u'_num_gpus': u'auto', u'_log_level': u'info', u'_kvstore': u'dist_async', u'force_dense': u'true', u'epochs': 1, u'num_trees': 100, u'eval_metrics': [u'accuracy', u'precision_recall_fscore'], u'_num_kv_servers': u'auto', u'mini_batch_size': 1000}\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Reading provided configuration from /opt/ml/input/config/hyperparameters.json: {u'mini_batch_size': u'1000', u'feature_dim': u'5', u'num_samples_per_tree': u'512', u'num_trees': u'50'}\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Final configuration: {u'_ftp_port': 8999, u'num_samples_per_tree': u'512', u'_tuning_objective_metric': u'', u'_num_gpus': u'auto', u'_log_level': u'info', u'_kvstore': u'dist_async', u'force_dense': u'true', u'epochs': 1, u'feature_dim': u'5', u'num_trees': u'50', u'eval_metrics': [u'accuracy', u'precision_recall_fscore'], u'_num_kv_servers': u'auto', u'mini_batch_size': u'1000'}\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 WARNING 139928461305664] Loggers have already been setup.\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Launching parameter server for role scheduler\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] {'ECS_CONTAINER_METADATA_URI': 'http://169.254.170.2/v3/8c0ee340-1be8-42c6-b03e-049664aa12b9', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION': '2', 'PATH': '/opt/amazon/bin:/usr/local/nvidia/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/amazon/bin:/opt/amazon/bin', 'SAGEMAKER_HTTP_PORT': '8080', 'HOME': '/root', 'PYTHONUNBUFFERED': 'TRUE', 'CANONICAL_ENVROOT': '/opt/amazon', 'LD_LIBRARY_PATH': '/opt/amazon/lib/python2.7/site-packages/cv2/../../../../lib:/usr/local/nvidia/lib64:/opt/amazon/lib', 'MXNET_KVSTORE_BIGARRAY_BOUND': '400000000', 'LANG': 'en_US.utf8', 'DMLC_INTERFACE': 'eth0', 'SHLVL': '1', 'AWS_REGION': 'us-west-2', 'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility', 'NVIDIA_VISIBLE_DEVICES': 'void', 'TRAINING_JOB_NAME': 'randomcutforest-2020-04-22-04-44-48-664', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION': 'cpp', 'ENVROOT': '/opt/amazon', 'SAGEMAKER_DATA_PATH': '/opt/ml', 'SAGEMAKER_METRICS_DIRECTORY': '/opt/ml/output/metrics/sagemaker', 'NVIDIA_REQUIRE_CUDA': 'cuda>=9.0', 'OMP_NUM_THREADS': '2', 'HOSTNAME': 'ip-10-0-231-18.us-west-2.compute.internal', 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI': '/v2/credentials/dfdc32af-218b-465d-a79f-42969f0d8f7b', 'PWD': '/', 'TRAINING_JOB_ARN': 'arn:aws:sagemaker:us-west-2:356566070122:training-job/randomcutforest-2020-04-22-04-44-48-664', 'AWS_EXECUTION_ENV': 'AWS_ECS_EC2'}\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] envs={'ECS_CONTAINER_METADATA_URI': 'http://169.254.170.2/v3/8c0ee340-1be8-42c6-b03e-049664aa12b9', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION': '2', 'DMLC_NUM_WORKER': '1', 'DMLC_PS_ROOT_PORT': '9000', 'PATH': '/opt/amazon/bin:/usr/local/nvidia/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/amazon/bin:/opt/amazon/bin', 'SAGEMAKER_HTTP_PORT': '8080', 'HOME': '/root', 'PYTHONUNBUFFERED': 'TRUE', 'CANONICAL_ENVROOT': '/opt/amazon', 'LD_LIBRARY_PATH': '/opt/amazon/lib/python2.7/site-packages/cv2/../../../../lib:/usr/local/nvidia/lib64:/opt/amazon/lib', 'MXNET_KVSTORE_BIGARRAY_BOUND': '400000000', 'LANG': 'en_US.utf8', 'DMLC_INTERFACE': 'eth0', 'SHLVL': '1', 'DMLC_PS_ROOT_URI': '10.0.231.18', 'AWS_REGION': 'us-west-2', 'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility', 'NVIDIA_VISIBLE_DEVICES': 'void', 'TRAINING_JOB_NAME': 'randomcutforest-2020-04-22-04-44-48-664', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION': 'cpp', 'ENVROOT': '/opt/amazon', 'SAGEMAKER_DATA_PATH': '/opt/ml', 'SAGEMAKER_METRICS_DIRECTORY': '/opt/ml/output/metrics/sagemaker', 'NVIDIA_REQUIRE_CUDA': 'cuda>=9.0', 'OMP_NUM_THREADS': '2', 'HOSTNAME': 'ip-10-0-231-18.us-west-2.compute.internal', 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI': '/v2/credentials/dfdc32af-218b-465d-a79f-42969f0d8f7b', 'DMLC_ROLE': 'scheduler', 'PWD': '/', 'DMLC_NUM_SERVER': '1', 'TRAINING_JOB_ARN': 'arn:aws:sagemaker:us-west-2:356566070122:training-job/randomcutforest-2020-04-22-04-44-48-664', 'AWS_EXECUTION_ENV': 'AWS_ECS_EC2'}\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Launching parameter server for role server\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] {'ECS_CONTAINER_METADATA_URI': 'http://169.254.170.2/v3/8c0ee340-1be8-42c6-b03e-049664aa12b9', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION': '2', 'PATH': '/opt/amazon/bin:/usr/local/nvidia/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/amazon/bin:/opt/amazon/bin', 'SAGEMAKER_HTTP_PORT': '8080', 'HOME': '/root', 'PYTHONUNBUFFERED': 'TRUE', 'CANONICAL_ENVROOT': '/opt/amazon', 'LD_LIBRARY_PATH': '/opt/amazon/lib/python2.7/site-packages/cv2/../../../../lib:/usr/local/nvidia/lib64:/opt/amazon/lib', 'MXNET_KVSTORE_BIGARRAY_BOUND': '400000000', 'LANG': 'en_US.utf8', 'DMLC_INTERFACE': 'eth0', 'SHLVL': '1', 'AWS_REGION': 'us-west-2', 'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility', 'NVIDIA_VISIBLE_DEVICES': 'void', 'TRAINING_JOB_NAME': 'randomcutforest-2020-04-22-04-44-48-664', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION': 'cpp', 'ENVROOT': '/opt/amazon', 'SAGEMAKER_DATA_PATH': '/opt/ml', 'SAGEMAKER_METRICS_DIRECTORY': '/opt/ml/output/metrics/sagemaker', 'NVIDIA_REQUIRE_CUDA': 'cuda>=9.0', 'OMP_NUM_THREADS': '2', 'HOSTNAME': 'ip-10-0-231-18.us-west-2.compute.internal', 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI': '/v2/credentials/dfdc32af-218b-465d-a79f-42969f0d8f7b', 'PWD': '/', 'TRAINING_JOB_ARN': 'arn:aws:sagemaker:us-west-2:356566070122:training-job/randomcutforest-2020-04-22-04-44-48-664', 'AWS_EXECUTION_ENV': 'AWS_ECS_EC2'}\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] envs={'ECS_CONTAINER_METADATA_URI': 'http://169.254.170.2/v3/8c0ee340-1be8-42c6-b03e-049664aa12b9', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION': '2', 'DMLC_NUM_WORKER': '1', 'DMLC_PS_ROOT_PORT': '9000', 'PATH': '/opt/amazon/bin:/usr/local/nvidia/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/amazon/bin:/opt/amazon/bin', 'SAGEMAKER_HTTP_PORT': '8080', 'HOME': '/root', 'PYTHONUNBUFFERED': 'TRUE', 'CANONICAL_ENVROOT': '/opt/amazon', 'LD_LIBRARY_PATH': '/opt/amazon/lib/python2.7/site-packages/cv2/../../../../lib:/usr/local/nvidia/lib64:/opt/amazon/lib', 'MXNET_KVSTORE_BIGARRAY_BOUND': '400000000', 'LANG': 'en_US.utf8', 'DMLC_INTERFACE': 'eth0', 'SHLVL': '1', 'DMLC_PS_ROOT_URI': '10.0.231.18', 'AWS_REGION': 'us-west-2', 'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility', 'NVIDIA_VISIBLE_DEVICES': 'void', 'TRAINING_JOB_NAME': 'randomcutforest-2020-04-22-04-44-48-664', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION': 'cpp', 'ENVROOT': '/opt/amazon', 'SAGEMAKER_DATA_PATH': '/opt/ml', 'SAGEMAKER_METRICS_DIRECTORY': '/opt/ml/output/metrics/sagemaker', 'NVIDIA_REQUIRE_CUDA': 'cuda>=9.0', 'OMP_NUM_THREADS': '2', 'HOSTNAME': 'ip-10-0-231-18.us-west-2.compute.internal', 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI': '/v2/credentials/dfdc32af-218b-465d-a79f-42969f0d8f7b', 'DMLC_ROLE': 'server', 'PWD': '/', 'DMLC_NUM_SERVER': '1', 'TRAINING_JOB_ARN': 'arn:aws:sagemaker:us-west-2:356566070122:training-job/randomcutforest-2020-04-22-04-44-48-664', 'AWS_EXECUTION_ENV': 'AWS_ECS_EC2'}\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Environment: {'ECS_CONTAINER_METADATA_URI': 'http://169.254.170.2/v3/8c0ee340-1be8-42c6-b03e-049664aa12b9', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION_VERSION': '2', 'DMLC_PS_ROOT_PORT': '9000', 'DMLC_NUM_WORKER': '1', 'SAGEMAKER_HTTP_PORT': '8080', 'PATH': '/opt/amazon/bin:/usr/local/nvidia/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/amazon/bin:/opt/amazon/bin', 'PYTHONUNBUFFERED': 'TRUE', 'CANONICAL_ENVROOT': '/opt/amazon', 'LD_LIBRARY_PATH': '/opt/amazon/lib/python2.7/site-packages/cv2/../../../../lib:/usr/local/nvidia/lib64:/opt/amazon/lib', 'MXNET_KVSTORE_BIGARRAY_BOUND': '400000000', 'LANG': 'en_US.utf8', 'DMLC_INTERFACE': 'eth0', 'SHLVL': '1', 'DMLC_PS_ROOT_URI': '10.0.231.18', 'AWS_REGION': 'us-west-2', 'SAGEMAKER_METRICS_DIRECTORY': '/opt/ml/output/metrics/sagemaker', 'NVIDIA_VISIBLE_DEVICES': 'void', 'TRAINING_JOB_NAME': 'randomcutforest-2020-04-22-04-44-48-664', 'HOME': '/root', 'PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION': 'cpp', 'ENVROOT': '/opt/amazon', 'SAGEMAKER_DATA_PATH': '/opt/ml', 'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility', 'NVIDIA_REQUIRE_CUDA': 'cuda>=9.0', 'OMP_NUM_THREADS': '2', 'HOSTNAME': 'ip-10-0-231-18.us-west-2.compute.internal', 'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI': '/v2/credentials/dfdc32af-218b-465d-a79f-42969f0d8f7b', 'DMLC_ROLE': 'worker', 'PWD': '/', 'DMLC_NUM_SERVER': '1', 'TRAINING_JOB_ARN': 'arn:aws:sagemaker:us-west-2:356566070122:training-job/randomcutforest-2020-04-22-04-44-48-664', 'AWS_EXECUTION_ENV': 'AWS_ECS_EC2'}\u001b[0m\n", "\u001b[34mProcess 32 is a shell:scheduler.\u001b[0m\n", "\u001b[34mProcess 33 is a shell:server.\u001b[0m\n", "\u001b[34mProcess 1 is a worker.\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Using default worker.\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Loaded iterator creator application/x-recordio-protobuf for content type ('application/x-recordio-protobuf', '1.0')\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Verifying hyperparamemters...\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Hyperparameters are correct.\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Validating that feature_dim agrees with dimensions in training data...\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] feature_dim is correct.\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Validating memory limits...\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Available memory in bytes: 15277408256\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Estimated sample size in bytes: 2048000\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Estimated memory needed to build the forest in bytes: 5120000\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Memory limits validated.\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Starting cluster sharing facilities...\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139928461305664] Create Store: dist_async\u001b[0m\n", "\u001b[34m[I 20-04-22 04:49:00] >>> starting FTP server on 0.0.0.0:8999, pid=1 <<<\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139926989424384] >>> starting FTP server on 0.0.0.0:8999, pid=1 <<<\u001b[0m\n", "\u001b[34m[I 20-04-22 04:49:00] poller: \u001b[0m\n", "\u001b[34m[I 20-04-22 04:49:00] masquerade (NAT) address: None\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139926989424384] poller: \u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139926989424384] masquerade (NAT) address: None\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139926989424384] passive ports: None\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:00 INFO 139926989424384] use sendfile(2): False\u001b[0m\n", "\u001b[34m[I 20-04-22 04:49:00] passive ports: None\u001b[0m\n", "\u001b[34m[I 20-04-22 04:49:00] use sendfile(2): False\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:01 INFO 139928461305664] Cluster sharing facilities started.\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:01 INFO 139928461305664] Verifying all workers are accessible...\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:01 INFO 139928461305664] All workers accessible.\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:01 INFO 139928461305664] Initializing Sampler...\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:01 INFO 139928461305664] Sampler correctly initialized.\u001b[0m\n", "\u001b[34m#metrics {\"Metrics\": {\"initialize.time\": {\"count\": 1, \"max\": 789.4580364227295, \"sum\": 789.4580364227295, \"min\": 789.4580364227295}}, \"EndTime\": 1587530941.168406, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"RandomCutForest\"}, \"StartTime\": 1587530940.355911}\n", "\u001b[0m\n", "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}, \"Total Batches Seen\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}, \"Total Records Seen\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}, \"Reset Count\": {\"count\": 1, \"max\": 0, \"sum\": 0.0, \"min\": 0}}, \"EndTime\": 1587530941.168693, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"init_train_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"RandomCutForest\"}, \"StartTime\": 1587530941.168586}\n", "\u001b[0m\n", "\u001b[34m[2020-04-22 04:49:01.177] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 0, \"duration\": 820, \"num_examples\": 1, \"num_bytes\": 64000}\u001b[0m\n", "\u001b[34m[04/22/2020 04:49:01 INFO 139928461305664] Sampling training data...\u001b[0m\n", "\n", "2020-04-22 04:48:56 Training - Training image download completed. Training in progress.\n", "2020-04-22 04:50:49 Uploading - Uploading generated training model\u001b[34m[2020-04-22 04:50:48.607] [tensorio] [info] epoch_stats={\"data_pipeline\": \"/opt/ml/input/data/train\", \"epoch\": 1, \"duration\": 107430, \"num_examples\": 65327, \"num_bytes\": 4180896000}\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] Sampling training data completed.\u001b[0m\n", "\u001b[34m#metrics {\"Metrics\": {\"epochs\": {\"count\": 1, \"max\": 1, \"sum\": 1.0, \"min\": 1}, \"update.time\": {\"count\": 1, \"max\": 107443.8591003418, \"sum\": 107443.8591003418, \"min\": 107443.8591003418}}, \"EndTime\": 1587531048.621054, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"RandomCutForest\"}, \"StartTime\": 1587530941.168522}\n", "\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] Early stop condition met. Stopping training.\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] #progress_metric: host=algo-1, completed 100 % epochs\u001b[0m\n", "\u001b[34m#metrics {\"Metrics\": {\"Max Batches Seen Between Resets\": {\"count\": 1, \"max\": 65327, \"sum\": 65327.0, \"min\": 65327}, \"Number of Batches Since Last Reset\": {\"count\": 1, \"max\": 65327, \"sum\": 65327.0, \"min\": 65327}, \"Number of Records Since Last Reset\": {\"count\": 1, \"max\": 65326500, \"sum\": 65326500.0, \"min\": 65326500}, \"Total Batches Seen\": {\"count\": 1, \"max\": 65327, \"sum\": 65327.0, \"min\": 65327}, \"Total Records Seen\": {\"count\": 1, \"max\": 65326500, \"sum\": 65326500.0, \"min\": 65326500}, \"Max Records Seen Between Resets\": {\"count\": 1, \"max\": 65326500, \"sum\": 65326500.0, \"min\": 65326500}, \"Reset Count\": {\"count\": 1, \"max\": 1, \"sum\": 1.0, \"min\": 1}}, \"EndTime\": 1587531048.621461, \"Dimensions\": {\"Host\": \"algo-1\", \"Meta\": \"training_data_iter\", \"Operation\": \"training\", \"Algorithm\": \"RandomCutForest\", \"epoch\": 0}, \"StartTime\": 1587530941.177153}\n", "\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] #throughput_metric: host=algo-1, train throughput=608002.316984 records/second\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] Master node: building Random Cut Forest...\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] Gathering samples...\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] 25600 samples gathered\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] Building Random Cut Forest...\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] Random Cut Forest built: \n", "\u001b[0m\n", "\u001b[34mForestInfo{num_trees: 50, num_samples_in_forest: 25600, num_samples_per_tree: 512, sample_dim: 5, shingle_size: 1, trees_num_nodes: [1019, 1005, 1019, 1015, 1015, 1015, 1013, 1017, 1019, 1019, 1019, 1009, 1017, 1013, 1007, 1017, 1023, 1013, 1013, 1009, 1011, 1019, 1021, 1015, 1017, 1009, 1017, 1015, 1013, 1017, 1015, 1017, 1019, 1011, 1017, 1017, 1017, 1005, 1011, 1021, 1021, 1017, 1011, 1007, 1013, 1009, 1013, 1013, 1015, 1019, ], trees_depth: [18, 27, 20, 18, 23, 21, 26, 23, 21, 24, 22, 23, 23, 22, 20, 23, 20, 25, 23, 22, 21, 21, 22, 21, 22, 21, 20, 22, 19, 21, 22, 23, 22, 27, 26, 23, 24, 22, 22, 22, 20, 23, 22, 23, 22, 20, 22, 24, 22, 21, ], max_num_nodes: 1023, min_num_nodes: 1005, avg_num_nodes: 1014, max_tree_depth: 27, min_tree_depth: 18, avg_tree_depth: 22, mem_size: 8524432}\u001b[0m\n", "\u001b[34m#metrics {\"Metrics\": {\"finalize.time\": {\"count\": 1, \"max\": 53.192138671875, \"sum\": 53.192138671875, \"min\": 53.192138671875}, \"model.bytes\": {\"count\": 1, \"max\": 8524432, \"sum\": 8524432.0, \"min\": 8524432}, \"fit_model.time\": {\"count\": 1, \"max\": 25.33888816833496, \"sum\": 25.33888816833496, \"min\": 25.33888816833496}}, \"EndTime\": 1587531048.674985, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"RandomCutForest\"}, \"StartTime\": 1587531048.621164}\n", "\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] Master node: Serializing the RandomCutForest model\u001b[0m\n", "\u001b[34m#metrics {\"Metrics\": {\"serialize_model.time\": {\"count\": 1, \"max\": 97.0149040222168, \"sum\": 97.0149040222168, \"min\": 97.0149040222168}}, \"EndTime\": 1587531048.772133, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"RandomCutForest\"}, \"StartTime\": 1587531048.675066}\n", "\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139928461305664] Test data is not provided.\u001b[0m\n", "\u001b[34m[I 20-04-22 04:50:48] >>> shutting down FTP server (0 active fds) <<<\u001b[0m\n", "\u001b[34m[04/22/2020 04:50:48 INFO 139926989424384] >>> shutting down FTP server (0 active fds) <<<\u001b[0m\n", "\u001b[34m#metrics {\"Metrics\": {\"totaltime\": {\"count\": 1, \"max\": 108752.5041103363, \"sum\": 108752.5041103363, \"min\": 108752.5041103363}, \"setuptime\": {\"count\": 1, \"max\": 201.57313346862793, \"sum\": 201.57313346862793, \"min\": 201.57313346862793}}, \"EndTime\": 1587531048.898565, \"Dimensions\": {\"Host\": \"algo-1\", \"Operation\": \"training\", \"Algorithm\": \"RandomCutForest\"}, \"StartTime\": 1587531048.772215}\n", "\u001b[0m\n", "\n", "2020-04-22 04:50:56 Completed - Training job completed\n", "Training seconds: 257\n", "Billable seconds: 257\n", "Training job: randomcutforest-2020-04-22-04-44-48-664\n" ] } ], "source": [ "from sagemaker import RandomCutForest\n", "\n", "session = sagemaker.Session()\n", "\n", "# specify general training job information\n", "rcf = RandomCutForest(role=execution_role,\n", " train_instance_count=1,\n", " train_instance_type='ml.m4.xlarge',\n", " data_location='s3://{}/{}/'.format(bucket, prefix),\n", " output_path='s3://{}/{}/output'.format(bucket, prefix),\n", " num_samples_per_tree=512,\n", " num_trees=50)\n", "\n", "# automatically upload the training data to S3 and run the training job\n", "#rcf.fit(rcf.record_set(player_data.value.as_matrix().reshape(-1,1)))\n", "\n", "rcf.fit(rcf.record_set(player_data.values.reshape(-1,5)))\n", "job_name = rcf.latest_training_job.job_name\n", "print(\"Training job: %s\" % job_name)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Inference\n", "\n", "***\n", "\n", "A trained Random Cut Forest model does nothing on its own. We now want to use the model we computed to perform inference on data. In this case, it means computing anomaly scores from input time series data points.\n", "\n", "We create an inference endpoint using the SageMaker Python SDK `deploy()` function from the job we defined above. We specify the instance type where inference is computed as well as an initial number of instances to spin up. We recommend using the `ml.c5` instance type as it provides the fastest inference time at the lowest cost." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "---------------!" ] } ], "source": [ "rcf_inference = rcf.deploy(\n", " initial_instance_count=1,\n", " instance_type='ml.m4.xlarge',\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Congratulations! You now have a functioning SageMaker RCF inference endpoint. You can confirm the endpoint configuration and status by navigating to the \"Endpoints\" tab in the AWS SageMaker console and selecting the endpoint matching the endpoint name, below: " ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Endpoint name: randomcutforest-2020-04-22-04-44-48-664\n" ] } ], "source": [ "print('Endpoint name: {}'.format(rcf_inference.endpoint))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data Serialization/Deserialization\n", "\n", "We can pass data in a variety of formats to our inference endpoint. In this example we will demonstrate passing CSV-formatted data. Other available formats are JSON-formatted and RecordIO Protobuf. We make use of the SageMaker Python SDK utilities `csv_serializer` and `json_deserializer` when configuring the inference endpoint." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "from sagemaker.predictor import csv_serializer, json_deserializer\n", "\n", "rcf_inference.content_type = 'text/csv'\n", "rcf_inference.serializer = csv_serializer\n", "rcf_inference.accept = 'application/json'\n", "rcf_inference.deserializer = json_deserializer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's pass the training dataset, in CSV format, to the inference endpoint so we can automatically detect the anomalies we saw with our eyes in the plots, above. Note that the serializer and deserializer will automatically take care of the datatype conversion from Numpy NDArrays.\n", "\n", "For starters, let's only pass in the first six datapoints so we can see what the output looks like." ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " playerx playerz quadrant sector1 sector2 eventname_encoded\n", "0 100.2216 -151.1082 1 -2 0 7\n", "1 100.2742 -151.3711 1 -1 0 7\n", "[[ 100.2216 -151.1082 1. -2. 0. 7. ]\n", " [ 100.2742 -151.3711 1. -1. 0. 7. ]]\n", "{'scores': [{'score': 0.6370093232}, {'score': 0.6410381801}]}\n" ] } ], "source": [ "print(player_data[:2])\n", "player_data_numpy = player_data.values.reshape(-1,6)\n", "print(player_data_numpy[:2])\n", "results = rcf_inference.predict(player_data_numpy[:2])\n", "print(results)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Computing Anomaly Scores\n", "\n", "Now, let's compute and plot the anomaly scores from the entire taxi dataset." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "results = rcf_inference.predict(player_data_numpy[:100])\n", "scores = [datum['score'] for datum in results['scores']]\n", "print(scores)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Stop and Delete the Endpoint\n", "\n", "Finally, we should delete the endpoint before we close the notebook.\n", "\n", "To do so execute the cell below. Alternately, you can navigate to the \"Endpoints\" tab in the SageMaker console, select the endpoint with the name stored in the variable `endpoint_name`, and select \"Delete\" from the \"Actions\" dropdown menu. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sagemaker.Session().delete_endpoint(rcf_inference.endpoint)" ] } ], "metadata": { "celltoolbar": "Tags", "kernelspec": { "display_name": "conda_amazonei_mxnet_p36", "language": "python", "name": "conda_amazonei_mxnet_p36" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" }, "notice": "Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License." }, "nbformat": 4, "nbformat_minor": 4 }