{ "cells": [ { "cell_type": "markdown", "id": "d8c854cb", "metadata": {}, "source": [ "# Data Preprocessing using RAPIDS and Training XGBoost for Fraud Detection" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n", "\n", "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-2/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "---" ] }, { "cell_type": "markdown", "id": "1c585524", "metadata": {}, "source": [ "\n", "\n", "\"rapids\"\n", "\n", "In this notebook we will walk through using [RAPIDS](https://rapids.ai/about.html) for GPU-accelerated data preprocessing and training of XGBoost model for a Fraud Detection use-case. This is the first notebook in a two notebook series. In the [second notebook](2_triton_xgb_fil_ensemble.ipynb) we will show how to deploy the trained XGBoost model in Triton on SageMaker. The RAPIDS suite of open source software libraries and APIs gives you the ability to execute end-to-end data science and analytics pipelines entirely on GPUs. \n" ] }, { "cell_type": "markdown", "id": "a15d9b9e", "metadata": {}, "source": [ "**Note:** Since the primary goal of this example is to get a trained XGBoost model to illustrate deployment of Tree-based ML models on Triton in SageMaker we don't perform any in-depth feature engineering or hyperparameter optimization. RAPIDS on SageMaker however is excellent for running cost-effective HPO in minimal amount of time as shown in the blog post [RAPIDS and Amazon SageMaker: Scale up and scale out to tackle ML challenges](https://aws.amazon.com/blogs/machine-learning/rapids-and-amazon-sagemaker-scale-up-and-scale-out-to-tackle-ml-challenges/). \n", "\n", "## To Run This Notebook Please Select RAPIDS 2106 Kernel from the Kernel Dropdown menu" ] }, { "cell_type": "markdown", "id": "bdf4884f", "metadata": {}, "source": [ "This notebook was tested with the `rapids-2106` kernel on an Amazon SageMaker notebook instance of type `g4dn`." ] }, { "cell_type": "markdown", "id": "0f87fb40", "metadata": {}, "source": [ "## Get Data" ] }, { "cell_type": "markdown", "id": "b379efd7", "metadata": {}, "source": [ "For this example, we use the Tabformer [synthetic credit card transactions dataset](https://arxiv.org/abs/1910.03033) from IBM available on [Kaggle](https://www.kaggle.com/datasets/ealtman2019/credit-card-transactions). The origin of this dataset along with its licensing terms can be found at: [Kaggle link](https://www.kaggle.com/datasets/ealtman2019/credit-card-transactions).\n" ] }, { "cell_type": "markdown", "id": "d82b7ea5", "metadata": {}, "source": [ "### Download Dataset" ] }, { "cell_type": "markdown", "id": "4588fb76", "metadata": {}, "source": [ "First we download the dataset from our Amazon S3 bucket." ] }, { "cell_type": "code", "execution_count": null, "id": "4f4f19b0", "metadata": {}, "outputs": [], "source": [ "!python -m pip install --upgrade pip --quiet\n", "!pip install -U awscli --quiet" ] }, { "cell_type": "code", "execution_count": null, "id": "58a250bc", "metadata": {}, "outputs": [], "source": [ "!aws s3 cp s3://sagemaker-sample-files/datasets/tabular/synthetic_credit_card_transactions/credit_card_transactions-ibm_v2.csv ./" ] }, { "cell_type": "markdown", "id": "f00abe8b", "metadata": {}, "source": [ "## Check on our GPU" ] }, { "cell_type": "markdown", "id": "96e7edd8", "metadata": {}, "source": [ "Next, let's check the GPU resources we have by using the terminal command `nvidia-smi`." ] }, { "cell_type": "code", "execution_count": null, "id": "7c3aeafc", "metadata": {}, "outputs": [], "source": [ "!nvidia-smi\n", "!nvidia-smi -L" ] }, { "cell_type": "markdown", "id": "8f5e8810", "metadata": {}, "source": [ "Awesome, we have powerful NVIDIA GPU at our disposal. Let's get started with using it for Data Preprocessing." ] }, { "cell_type": "markdown", "id": "0fb68f21", "metadata": {}, "source": [ "## Data Preprocessing" ] }, { "cell_type": "code", "execution_count": null, "id": "9894de9e", "metadata": {}, "outputs": [], "source": [ "import cudf\n", "import cuml\n", "import numpy as np\n", "import pickle\n", "import os" ] }, { "cell_type": "markdown", "id": "ef59ddd2", "metadata": {}, "source": [ "We read in the data and begin our data preprocessing." ] }, { "cell_type": "code", "execution_count": null, "id": "7adb6815", "metadata": {}, "outputs": [], "source": [ "data_path = \"./\"\n", "data_csv = \"credit_card_transactions-ibm_v2.csv\"\n", "full_data = cudf.read_csv(os.path.join(data_path, data_csv))\n", "full_data.head()" ] }, { "cell_type": "markdown", "id": "4ffea2c4", "metadata": {}, "source": [ "Each row here is a credit card transaction with attributes like time and amount of transaction along with merchant attributes like Name, City, State, Zipcode and Merchant Category Code (MCC) and finally whether the transaction was fraudulent or legitimate (`Is Fraud?`). \n", "\n", "**Note:** `Merchant Name` is hashed so that's why we see integers instead of strings.\n", "\n", "The full dataset has about 24 million rows but in this example we use random subset of about ~5 million transactions." ] }, { "cell_type": "code", "execution_count": null, "id": "aaa12ce5", "metadata": {}, "outputs": [], "source": [ "SEED = 42\n", "data = full_data.sample(frac=0.2, random_state=SEED)\n", "data = data.reset_index(drop=True)\n", "print(data.shape)" ] }, { "cell_type": "markdown", "id": "aff03966", "metadata": {}, "source": [ "We convert some categorical features to dtype objects." ] }, { "cell_type": "code", "execution_count": null, "id": "937f07a6", "metadata": {}, "outputs": [], "source": [ "data[\"Zip\"] = data[\"Zip\"].astype(\"object\")\n", "data[\"MCC\"] = data[\"MCC\"].astype(\"object\")\n", "data[\"Merchant Name\"] = data[\"Merchant Name\"].astype(\"object\")" ] }, { "cell_type": "markdown", "id": "e60bbfcd", "metadata": {}, "source": [ "### Encode labels\n" ] }, { "cell_type": "markdown", "id": "1a5b1c65", "metadata": {}, "source": [ "Next we perform encoding on our binary labels `Is Fraud?` which indicate whether a transaction is fraudulent or not. After encoding, `1` will denote fraud and `0` will denote legitimate transaction." ] }, { "cell_type": "code", "execution_count": null, "id": "89dd9c48", "metadata": {}, "outputs": [], "source": [ "y = data[\"Is Fraud?\"]\n", "data.drop(columns=[\"Is Fraud?\"], inplace=True)\n", "y = (y == \"Yes\").astype(int)" ] }, { "cell_type": "markdown", "id": "91889ae3", "metadata": {}, "source": [ "### Save subset for inference" ] }, { "cell_type": "markdown", "id": "8fdac51b", "metadata": {}, "source": [ "We will also save a small subset of the data to submit Triton inference requests for later on in the [second notebook](2_triton_xgb_fil_ensemble.ipynb)." ] }, { "cell_type": "code", "execution_count": null, "id": "588dcba3", "metadata": {}, "outputs": [], "source": [ "data_infer = data.iloc[625:630]\n", "data_infer.to_csv(\"data_infer.csv\", index=False)" ] }, { "cell_type": "markdown", "id": "89151f4b", "metadata": {}, "source": [ "### Handle Missing Values" ] }, { "cell_type": "markdown", "id": "d57baac4", "metadata": {}, "source": [ "Next let's handle the missing values in our data." ] }, { "cell_type": "code", "execution_count": null, "id": "626f87dc", "metadata": {}, "outputs": [], "source": [ "data.isna().sum() / len(data) * 100" ] }, { "cell_type": "markdown", "id": "66b2518f", "metadata": {}, "source": [ "We have some missing values in `Merchant State` and `Zip` columns. Turns out these correspond to ONLINE transactions so we will set those missing values to `ONLINE`." ] }, { "cell_type": "code", "execution_count": null, "id": "48e9d1b1", "metadata": {}, "outputs": [], "source": [ "data.loc[data[\"Merchant City\"] == \"ONLINE\", \"Merchant State\"] = \"ONLINE\"\n", "data.loc[data[\"Merchant City\"] == \"ONLINE\", \"Zip\"] = \"ONLINE\"" ] }, { "cell_type": "markdown", "id": "aa13e333", "metadata": {}, "source": [ "We also have some foreign transactions where `Merchant City` and `Merchant State` is a foreign city and country and the Zipcode is missing. For those transactions we will set the Zipcode to `FOREIGN`." ] }, { "cell_type": "code", "execution_count": null, "id": "926ac124", "metadata": {}, "outputs": [], "source": [ "us_states_plus_online = [\n", " \"AK\",\n", " \"AL\",\n", " \"AR\",\n", " \"AZ\",\n", " \"CA\",\n", " \"CO\",\n", " \"CT\",\n", " \"DC\",\n", " \"DE\",\n", " \"FL\",\n", " \"GA\",\n", " \"HI\",\n", " \"IA\",\n", " \"ID\",\n", " \"IL\",\n", " \"IN\",\n", " \"KS\",\n", " \"KY\",\n", " \"LA\",\n", " \"MA\",\n", " \"MD\",\n", " \"ME\",\n", " \"MI\",\n", " \"MN\",\n", " \"MO\",\n", " \"MS\",\n", " \"MT\",\n", " \"NC\",\n", " \"ND\",\n", " \"NE\",\n", " \"NH\",\n", " \"NJ\",\n", " \"NM\",\n", " \"NV\",\n", " \"NY\",\n", " \"OH\",\n", " \"OK\",\n", " \"OR\",\n", " \"PA\",\n", " \"RI\",\n", " \"SC\",\n", " \"SD\",\n", " \"TN\",\n", " \"TX\",\n", " \"UT\",\n", " \"VA\",\n", " \"VT\",\n", " \"WA\",\n", " \"WI\",\n", " \"WV\",\n", " \"WY\",\n", " \"ONLINE\",\n", "]\n", "\n", "# set zip of all transactions that are not in US States or Online to Foreign\n", "data.loc[~data[\"Merchant State\"].isin(us_states_plus_online), \"Zip\"] = \"FOREIGN\"" ] }, { "cell_type": "markdown", "id": "9ff624ef", "metadata": {}, "source": [ "The `Errors?` column indicates whether or not the transaction had any errors like an Incorrect Pin associated with it. We make this a boolean indicator feature." ] }, { "cell_type": "code", "execution_count": null, "id": "068800f0", "metadata": {}, "outputs": [], "source": [ "data[\"Errors?\"] = data[\"Errors?\"].notna()" ] }, { "cell_type": "code", "execution_count": null, "id": "0ae0b332", "metadata": {}, "outputs": [], "source": [ "data.isna().sum() / len(data) * 100" ] }, { "cell_type": "markdown", "id": "5162efe1", "metadata": {}, "source": [ "So now we have handled all the missing values in our data." ] }, { "cell_type": "markdown", "id": "e80a3fb2", "metadata": {}, "source": [ "### Handle Amount and Time" ] }, { "cell_type": "markdown", "id": "9f8ea1d0", "metadata": {}, "source": [ "Next, for the `Amount` column we remove the dollar symbol prefix and for `Time` column we extract out the Hour and Minute." ] }, { "cell_type": "code", "execution_count": null, "id": "8350762d", "metadata": {}, "outputs": [], "source": [ "data[\"Amount\"] = data[\"Amount\"].str.slice(1)\n", "data[\"Hour\"] = data[\"Time\"].str.slice(stop=2)\n", "data[\"Minute\"] = data[\"Time\"].str.slice(start=3)\n", "data.drop(columns=[\"Time\"], inplace=True)" ] }, { "cell_type": "markdown", "id": "a2934011", "metadata": {}, "source": [ "### Train-Test Split" ] }, { "cell_type": "markdown", "id": "25acf723", "metadata": {}, "source": [ "Before doing any further preprocessing let's perform the train-test split. Here we use 70-30 train-test split." ] }, { "cell_type": "code", "execution_count": null, "id": "1aa71566", "metadata": {}, "outputs": [], "source": [ "from cuml.model_selection import train_test_split\n", "\n", "X_train, X_test, y_train, y_test = train_test_split(\n", " data, y, test_size=0.3, random_state=SEED, stratify=y\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "5c957d39", "metadata": {}, "outputs": [], "source": [ "# Free up some room on the GPU by explicitly deleting dataframes\n", "import gc\n", "\n", "del data\n", "del y\n", "gc.collect()" ] }, { "cell_type": "markdown", "id": "8dfa111b", "metadata": {}, "source": [ "### Encoding Categorical Columns" ] }, { "cell_type": "markdown", "id": "08c0cb62", "metadata": {}, "source": [ "Next, we handle categorical columns in our dataset by performing [label encoding](https://docs.rapids.ai/api/cuml/stable/api.html?highlight=label%20encoder#feature-and-label-encoding-single-gpu) on them which convert categorical values into numerical values. For some of these columns we have some unseen values which are present in test data but not train data. We handle those values by setting them to `UNKNOWN` before doing the label encoding so that at test time we have an encoding for these unseen values.\n", "\n", "We also serialize the encodings for all categorical columns so that we can later use them for doing data preprocessing at inference time in the [second notebook](2_triton_xgb_fil_ensemble.ipynb)." ] }, { "cell_type": "code", "execution_count": null, "id": "27bcaf30", "metadata": {}, "outputs": [], "source": [ "from cuml.preprocessing import LabelEncoder\n", "\n", "categorial_columns = [\"Zip\", \"MCC\", \"Merchant Name\", \"Use Chip\", \"Merchant City\", \"Merchant State\"]\n", "encoders = {}\n", "\n", "# handle unknown values present in test data but not in training data\n", "for col in categorial_columns:\n", " # convert cudf series to numpy array with .values_host\n", " unique_values = X_train[col].unique().values_host\n", " X_test.loc[~X_test[col].isin(unique_values), col] = \"UNKNOWN\"\n", " unique_values = np.append(unique_values, [\"UNKNOWN\"])\n", " # convert numpy array to cudf series\n", " unique_values = cudf.Series(unique_values)\n", " le = LabelEncoder().fit(unique_values)\n", " X_train[col] = le.transform(X_train[col])\n", " X_test[col] = le.transform(X_test[col])\n", " encoders[col] = le.classes_.values_host\n", "\n", "with open(\"label_encoders.pkl\", \"wb\") as f:\n", " pickle.dump(encoders, f)" ] }, { "cell_type": "code", "execution_count": null, "id": "97eb4855", "metadata": {}, "outputs": [], "source": [ "# convert all dtypes to fp32 for xgboost training\n", "X_train = X_train.astype(\"float32\")\n", "X_test = X_test.astype(\"float32\")" ] }, { "cell_type": "markdown", "id": "28e78093", "metadata": {}, "source": [ "Let's look at our preprocessed data." ] }, { "cell_type": "code", "execution_count": null, "id": "46c42639", "metadata": {}, "outputs": [], "source": [ "X_train.head()" ] }, { "cell_type": "markdown", "id": "fbf7bb71", "metadata": {}, "source": [ "## Train XGBoost" ] }, { "cell_type": "markdown", "id": "8df4e196", "metadata": {}, "source": [ "Now we train the XGBoost fraud detection model on our GPU. This will take about 2-3 minutes on `g4dn.xlarge` instance." ] }, { "cell_type": "code", "execution_count": null, "id": "e6be5d5e", "metadata": {}, "outputs": [], "source": [ "import xgboost as xgb\n", "import time\n", "\n", "dtrain = xgb.DMatrix(X_train, y_train)\n", "\n", "dtest = xgb.DMatrix(X_test, y_test)\n", "\n", "max_depth = 8\n", "num_trees = 2000\n", "xgb_params = {\n", " \"max_depth\": max_depth,\n", " \"tree_method\": \"gpu_hist\",\n", " \"objective\": \"binary:logistic\",\n", " \"eval_metric\": \"aucpr\",\n", " \"predictor\": \"gpu_predictor\",\n", "}\n", "model = xgb.train(params=xgb_params, dtrain=dtrain, num_boost_round=num_trees)" ] }, { "cell_type": "markdown", "id": "e26dbd34", "metadata": {}, "source": [ "We quickly evaluate our trained model's predictions on the test set using F1-score." ] }, { "cell_type": "code", "execution_count": null, "id": "0121077c", "metadata": {}, "outputs": [], "source": [ "from sklearn.metrics import f1_score\n", "\n", "y_score = model.predict(dtest)\n", "threshold = 0.5\n", "y_pred = (y_score >= 0.5).astype(int)\n", "y_true = y_test.values_host\n", "f1 = f1_score(y_true, y_pred)\n", "print(f\"Test F1-Score: {f1: 0.4f}\")" ] }, { "cell_type": "markdown", "id": "2672cf3a", "metadata": {}, "source": [ "We can do further Hyperparameter tuning/Feature Engineering to improve the model accuracy but since the primary goal of this example is to walkthrough deployment of decision tree-based ML models like XGBoost on Triton in SageMaker we save our trained model and move on to the [second notebook](2_triton_xgb_fil_ensemble.ipynb)." ] }, { "cell_type": "markdown", "id": "b0962df8", "metadata": {}, "source": [ "### Save Trained Model" ] }, { "cell_type": "code", "execution_count": null, "id": "c5f2f476", "metadata": {}, "outputs": [], "source": [ "model_path = \"./xgboost.json\"\n", "model.save_model(model_path)" ] }, { "cell_type": "markdown", "id": "3236c5c1", "metadata": {}, "source": [ "## Next Step" ] }, { "cell_type": "markdown", "id": "fdb0297e", "metadata": {}, "source": [ "Please open the [second notebook](2_triton_xgb_fil_ensemble.ipynb) to learn how to deploy this XGBoost model and other similar decision tree-based ML models on Triton in SageMaker." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Notebook CI Test Results\n", "\n", "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n", "\n", "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-2/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ca-central-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/sa-east-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-2/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-3/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-central-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-north-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-2/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-2/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n", "\n", "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-south-1/sagemaker-triton|fil_ensemble|1_prep_rapids_train_xgb.ipynb)\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3.9.13 64-bit", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" }, "vscode": { "interpreter": { "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e" } } }, "nbformat": 4, "nbformat_minor": 5 }