{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d9f690a0",
   "metadata": {},
   "source": [
    "# Detect Social Media Fake News using Amazon Neptune ML"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ae453fc",
   "metadata": {},
   "source": [
    "In this notebook we demonstrate how to use Graph Machine Learning from Amazon Neptune ML to identify fake news on social media. We created a graph dataset based on the [BuzzFeed data](https://github.com/KaiDMML/FakeNewsNet/tree/old-version/Data/BuzzFeed) from the 2018 version of FakeNewsNet in the `create-graph-dataset.ipynb` notebook. \n",
    "\n",
    "Note: Use [these CloudFormation templates](https://docs.aws.amazon.com/neptune/latest/userguide/machine-learning-quick-start.html) to quickly spin up a `graph-notebook`, an associted Neptune cluster, and set up all the configurations needed to work with Neptune ML in a `graph-notebook`. You can use the `%graph_notebook_config` magic command to see information about the Neptune cluster associated with your graph-notebook, and `%status` magic command to see the status of your Neptune cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7094cf6d",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8e71304",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import required libraries\n",
    "import boto3\n",
    "import sagemaker\n",
    "import pandas as pd\n",
    "import utils.neptune_ml_utils as neptune_ml\n",
    "# Check to make sure your Neptune cluster is configured to run Neptune ML.\n",
    "neptune_ml.check_ml_enabled()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61fbd6d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "sess = sagemaker.Session()\n",
    "bucket = sess.default_bucket()\n",
    "\n",
    "# S3 location that will be used to store data, processing results and model artifacts\n",
    "bucket = bucket #'<bucket-name>'\n",
    "prefix = 'fake-news-detection'\n",
    "s3_uri = f\"s3://{bucket}/{prefix}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "724711ec",
   "metadata": {},
   "source": [
    "## Checking Neptune DB"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b631760b",
   "metadata": {},
   "source": [
    "Check the status of the Neptune cluster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8eb943d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "%status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a494fcab",
   "metadata": {},
   "source": [
    "To verify that the graph dataset is loaded in the Neptune cluster, we run the following Gremlin traversals to see the count of nodes and edges by label:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57b96e80",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "g.V().groupCount().by(label).unfold().order().by(keys)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "017211d9",
   "metadata": {},
   "source": [
    "If nodes are loaded correctly, the output would be:\n",
    "\n",
    "* 126 `author` nodes\n",
    "* 182 `news` nodes\n",
    "* 28 `publisher` nodes\n",
    "* 15,257 `user` nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d52d36d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "g.E().groupCount().by(label).unfold().order().by(keys)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a14eae40",
   "metadata": {},
   "source": [
    "If edges are loaded correctly, then the output would be:\n",
    "\n",
    "* 634,750 `follows` edges\n",
    "* 174 `published` edges\n",
    "* 250 `wrote` edges\n",
    "* 250 `wrote_for` edges"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a13f307f",
   "metadata": {},
   "source": [
    "## Preparing for Export\n",
    "With our data validated, let's simulate new `news` being added into our graph by removing the `news_type` property (i.e. the target variable for machine learning) from two of the `news` nodes. We will treat these nodes as testing nodes later (i.e. will run inference on them at the end to determine whether they're `real` or `fake`)\n",
    "\n",
    "Let's begin by taking a look at the current value of the `news_type` property for those two `news` nodes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a95c7e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "\n",
    "g.V().has('news', 'news_title', within(\"Jeb Bush to lecture at Harvard this fall\", \"BREAKING: Steps to FORCE FBI Director Comey to Resign In Process – Hearing Decides His Fate Sept 28\")).\n",
    "    valueMap('news_title', 'news_type')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30b1cd37",
   "metadata": {},
   "source": [
    "Now let's remove these `news_type` property values from the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cae94561",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "\n",
    "g.V().has('news', 'news_title', within(\"Jeb Bush to lecture at Harvard this fall\", \"BREAKING: Steps to FORCE FBI Director Comey to Resign In Process – Hearing Decides His Fate Sept 28\")).\n",
    "    properties('news_type').drop()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "600ae785",
   "metadata": {},
   "source": [
    "Let's check those two `news` nodes again to verify that they no longer have `news_type` values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d809d07",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "\n",
    "g.V().has('news', 'news_title', within(\"Jeb Bush to lecture at Harvard this fall\", \"BREAKING: Steps to FORCE FBI Director Comey to Resign In Process – Hearing Decides His Fate Sept 28\")).\n",
    "    valueMap('news_title', 'news_type')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "afdc81ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -g T.label -p v,inE,outV,outE,inV,outE,inV,oute,inv,ine,outv,ine,outv,ine,outv\n",
    "\n",
    "g.V().has('news', 'news_title', within(\"BREAKING: Steps to FORCE FBI Director Comey to Resign In Process – Hearing Decides His Fate Sept 28\"))\n",
    ".inE(\"wrote\")\n",
    ".outV()\n",
    ".outE(\"wrote_for\")\n",
    ".inV()\n",
    ".outE(\"published\")\n",
    ".inV().has('news', 'news_title', within(\"BREAKING: Steps to FORCE FBI Director Comey to Resign In Process – Hearing Decides His Fate Sept 28\"))\n",
    ".outE(\"spread_by\")\n",
    ".inV()\n",
    ".inE(\"follows\")\n",
    ".outV()\n",
    ".inE(\"follows\")\n",
    ".outV()\n",
    ".inE(\"follows\")\n",
    ".outV()\n",
    ".path()\n",
    ".by(valueMap(true))\n",
    ".limit(100)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17b2ffc3",
   "metadata": {},
   "source": [
    "## Exporting Data and Model Configuration\n",
    "\n",
    "The export process is triggered by calling to the Neptune Export service endpoint. This call contains a configuration object which specifies the type of machine learning model to build, in our case `node classification`, as well as any feature configurations required.\n",
    "\n",
    "The configuration options provided to the Neptune Export service are broken into two main sections, selecting the target and configuring features. Here we want to classify `news` nodes according to the `news_type` property. \n",
    "\n",
    "The second section of the configuration, configuring features, is where we specify details about the types of data stored in our graph and how the machine learning model should interpret that data. When data is exported from Neptune, all properties of all nodes are included. Each property is treated as a separate feature for the ML model. Neptune ML does its best to infer the correct type of feature for a property, in many cases, the accuracy of the model can be improved by specifying information about the property used for a feature. We use [word2vec](https://en.wikipedia.org/wiki/Word2vec) for encoding `news_title` property of `news` nodes, and `numerical` type for `user_features` property of `user` nodes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d9f8987a",
   "metadata": {},
   "outputs": [],
   "source": [
    "export_params={ \n",
    "\"command\": \"export-pg\", \n",
    "\"params\": { \"endpoint\": neptune_ml.get_host(),\n",
    "            \"profile\": \"neptune_ml\",\n",
    "            \"useIamAuth\": neptune_ml.get_iam(),\n",
    "            \"cloneCluster\": False\n",
    "            }, \n",
    "\"outputS3Path\": f\"{s3_uri}/neptune-export\",\n",
    "\"additionalParams\": {\n",
    "        \"neptune_ml\": {\n",
    "          \"version\": \"v2.0\",\n",
    "          \"targets\": [\n",
    "            {\n",
    "              \"node\": \"news\",\n",
    "              \"property\": \"news_type\",\n",
    "              \"type\": \"classification\"\n",
    "            }\n",
    "          ],\n",
    "         \"features\": [\n",
    "            {\n",
    "                \"node\": \"news\",\n",
    "                \"property\": \"news_title\",\n",
    "                \"type\": \"text_word2vec\"\n",
    "            },\n",
    "            {\n",
    "                \"node\": \"user\",\n",
    "                \"property\": \"user_features\",\n",
    "                \"type\": \"numerical\"\n",
    "            }\n",
    "         ]\n",
    "        }\n",
    "      },\n",
    "\"jobSize\": \"medium\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "909b6aa9",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%neptune_ml export start --export-url {neptune_ml.get_export_service_host()} --export-iam --wait --store-to export_results\n",
    "${export_params}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c3254c9",
   "metadata": {},
   "source": [
    "## ML Data Processing\n",
    "\n",
    "Once the export job is completed we are ready to train our machine learning model and create the inference endpoint. There are three machine learning steps in Neptune ML. The first step (data processing) processes the exported graph dataset using standard feature preprocessing techniques to prepare it for use by [Deep Graph Library (DGL)](https://www.dgl.ai/). This step performs functions such as feature normalization for numeric data and encoding text features using word2vec. At the conclusion of this step the dataset is formatted for model training.\n",
    "\n",
    "This step is implemented using a SageMaker Processing Job and data artifacts are stored in a pre-specified S3 location once the job is completed. Running the cells below will create the data processing configuration and begin the processing job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f084a487",
   "metadata": {},
   "outputs": [],
   "source": [
    "# The training_job_name can be set to a unique value below, otherwise one will be auto generated\n",
    "training_job_name=neptune_ml.get_training_job_name('fake-news-detection')\n",
    "\n",
    "processing_params = f\"\"\"\n",
    "--config-file-name training-data-configuration.json\n",
    "--job-id {training_job_name} \n",
    "--s3-input-uri {export_results['outputS3Uri']} \n",
    "--s3-processed-uri {str(s3_uri)}/preloading\n",
    "--instance-type ml.c5.9xlarge \n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27c1a331",
   "metadata": {},
   "outputs": [],
   "source": [
    "%neptune_ml dataprocessing start --wait --store-to processing_results {processing_params}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d477acd",
   "metadata": {},
   "source": [
    "## Model Training\n",
    "The second step (model training) trains the ML model that will be used for predictions. The model training is done in two stages. The first stage uses a SageMaker Processing job to generate a model training strategy. A model training strategy is a configuration set that specifies what type of model and model hyperparameter ranges will be used for the model training. Once the first stage is complete, the SageMaker Processing job launches a SageMaker Hyperparameter tuning job. The SageMaker Hyperparameter tuning job runs a pre-specified number of model training job trials on the processed data, and stores the model artifacts generated by the training in the output S3 location. Once all the training jobs are complete, the Hyperparameter tuning job also notes the training job that produced the best performing model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4588fd01",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_params=f\"\"\"\n",
    "--job-id {training_job_name}\n",
    "--data-processing-id {training_job_name} \n",
    "--instance-type ml.c5.18xlarge\n",
    "--s3-output-uri {str(s3_uri)}/training\n",
    "--max-hpo-number 30\n",
    "--max-hpo-parallel 3 \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1af31b2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "%neptune_ml training start --wait --store-to training_results {training_params}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37f9304b",
   "metadata": {},
   "source": [
    "### Evaluating HPO Job"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1ae7c64",
   "metadata": {},
   "source": [
    "In this section we retrieve the results of Hyperparameter Tuning job and summarize hyperparameters of the five best training jobs and their respective model performance.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44b4435c",
   "metadata": {},
   "outputs": [],
   "source": [
    "tuning_job_name = training_results['hpoJob']['name']\n",
    "tuner = sagemaker.HyperparameterTuningJobAnalytics(tuning_job_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd233cbf",
   "metadata": {},
   "outputs": [],
   "source": [
    "full_df = tuner.dataframe()\n",
    "\n",
    "if len(full_df) > 0:\n",
    "    df = full_df[full_df[\"FinalObjectiveValue\"] > -float(\"inf\")]\n",
    "    if len(df) > 0:\n",
    "        df = df.sort_values(\"FinalObjectiveValue\", ascending=False)\n",
    "        print(\"Number of training jobs with valid objective: %d\" % len(df))\n",
    "        print({\"lowest\": min(df[\"FinalObjectiveValue\"]), \"highest\": max(df[\"FinalObjectiveValue\"])})\n",
    "        pd.set_option(\"display.max_colwidth\", None)  # Don't truncate TrainingJobName\n",
    "    else:\n",
    "        print(\"No training jobs have reported valid results yet.\")\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "262a6a53",
   "metadata": {},
   "source": [
    "We can see that the best performing training job has acheived an accuracy of ~89%. This training job will be automatically selected by Neptune ML for creating an endpoint in the next step."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c20e3c9",
   "metadata": {},
   "source": [
    "## Endpoint Creation\n",
    "The final step of machine learning is to create an inference endpoint which is an Amazon SageMaker endpoint instance that is launched with the model artifacts produced by the best training job. This endpoint will be used by our graph queries to return the model predictions for the inputs in the request. Once the endpoint is created, it stays active until it is manually deleted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b43f3a68",
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint_params=f\"\"\"\n",
    "--id {training_job_name}\n",
    "--model-training-job-id {training_job_name} \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d566043",
   "metadata": {},
   "outputs": [],
   "source": [
    "%neptune_ml endpoint create --wait --store-to endpoint_results {endpoint_params}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17fa8f92",
   "metadata": {},
   "source": [
    "Once this has completed we get the endpoint name for our newly created inference endpoint. The cell below will set the endpoint name which will be used in the Gremlin queries below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da7647d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint=endpoint_results['endpoint']['name']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "755293ac",
   "metadata": {},
   "source": [
    "## Predicting Values using Gremlin Queries\n",
    "Now that we have our inference endpoint setup let's query our graph to see how the model predicts `news_type` for our new `news` nodes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69f1d04d",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "g.with(\"Neptune#ml.endpoint\", \"${endpoint}\").\n",
    "  V().has('news_title', \"Jeb Bush to lecture at Harvard this fall\").\n",
    "  properties(\"news_type\").with(\"Neptune#ml.classification\").value()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6fb74d25",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "g.with(\"Neptune#ml.endpoint\", \"${endpoint}\").\n",
    "  V().has('news_title', \"BREAKING: Steps to FORCE FBI Director Comey to Resign In Process – Hearing Decides His Fate Sept 28\")\n",
    "  .properties(\"news_type\").with(\"Neptune#ml.classification\").value()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c6acbed",
   "metadata": {},
   "source": [
    "We see that the model correctly predicts `news_type` for both test nodes!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eec0de8e",
   "metadata": {},
   "source": [
    "## Cleaning Up"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09d92399",
   "metadata": {},
   "source": [
    "Now that we can delete the inference endpoint to avoid recurring costs!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84f68410",
   "metadata": {},
   "outputs": [],
   "source": [
    "neptune_ml.delete_endpoint(training_job_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02f7f0b4",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}