{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Amanzon SageMaker Ground Truth Demonstration for Text Classification\n",
    "\n",
    "1. [Introduction](#introduction)\n",
    "2. [Run a Ground Truth labeling job](#run-a-ground-truth-labeling-job)\n",
    "    1. [Prepare the data](#prepare-the-data)\n",
    "    2. [Specify the categories](#specify-the-categories)\n",
    "    3. [Create the instruction template](#create-the-instruction-template)\n",
    "    4. [Assign private team to test your task](#assign-private-team-to-test-your-task)\n",
    "    5. [Define pre-built lambda functions for use in the labeling job](#define-pre-built-lambda-functions-for-use-in-the-labeling-job)\n",
    "    6. [Submit the Ground Truth job request](#submit-the-ground-truth-job-request)\n",
    "    7. [Monitor job progress](#monitor-job-progress)\n",
    "    8. [View Task Results](#view-task-results)\n",
    "3. [Clean Up](#Clean-Up)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Introduction\n",
    "\n",
    "\n",
    "This sample notebook takes you through an end-to-end workflow to demonstrate the functionality of SageMaker Ground Truth. We'll start with an unlabeled review text data set, acquire labels for the sentiment like Positive, or Negative  using SageMaker Ground Truth, and analyze the results of the labeling job. Before you begin, we highly recommend you start a Ground Truth labeling job through the AWS Console first to familiarize yourself with the workflow. The AWS Console offers less flexibility than the API, but is simple to use.\n",
    "\n",
    "#### Cost and runtime\n",
    "You can run this demo in two modes:\n",
    "1. Set `RUN_FULL_AL_DEMO = True` in the next cell to label 10k reviews(text). This should cost about \\$800 given current [Ground Truth pricing scheme](https://aws.amazon.com/sagemaker/groundtruth/pricing/). In order to reduce the cost, we will use Ground Truth's auto-labeling feature. Auto-labeling uses text classification to learn from human responses and automatically create labels for the easiest review text at a cheap price. The total end-to-end runtime should be about 20h.\n",
    "1. Set `RUN_FULL_AL_DEMO = False` in the next cell to label only 10 reviews. This should cost about \\$.80. **Since Ground Truth's auto-labeling feature only kicks in for datasets of 1000 images or more, this cheaper version of the demo will not use it. Some of the analysis plots might look awkward, but you should still be able to see good results on the human-annotated 100 images.**\n",
    "\n",
    "#### Prerequisites\n",
    "To run this notebook, you can simply execute each cell one-by-one. To understand what's happening, you'll need:\n",
    "* An S3 bucket you can write to -- please provide its name in the following cell. The bucket must be in the same region as this SageMaker Notebook instance. You can also change the `EXP_NAME` to any valid S3 prefix. All the files related to this experiment will be stored in that prefix of your bucket.\n",
    "* Familiarity with Python and [numpy](http://www.numpy.org/).\n",
    "* Basic familiarity with [AWS S3](https://docs.aws.amazon.com/s3/index.html),\n",
    "* Basic understanding of [AWS Sagemaker](https://aws.amazon.com/sagemaker/),\n",
    "* Basic familiarity with [AWS Command Line Interface (CLI)](https://aws.amazon.com/cli/) -- set it up with credentials to access the AWS account you're running this notebook from. This should work out-of-the-box on SageMaker Jupyter Notebook instances.\n",
    "\n",
    "This notebook is only tested on a SageMaker notebook instance. The runtimes given are approximate, we used an `ml.m4.xlarge` instance in our tests. However, you can likely run it on a local instance by first executing the cell below on SageMaker, and then copying the `role` string to your local copy of the notebook.\n",
    "\n",
    "NOTE: This notebook will create/remove subdirectories in its working directory. We recommend to place this notebook in its own directory before running it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# cell 01\n",
    "%matplotlib inline\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "import os\n",
    "from collections import namedtuple\n",
    "from collections import defaultdict\n",
    "from collections import Counter\n",
    "import itertools\n",
    "import json\n",
    "import time\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.backends.backend_pdf import PdfPages\n",
    "from sklearn.metrics import confusion_matrix\n",
    "import boto3\n",
    "import sagemaker\n",
    "from urllib.parse import urlparse\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\", category=DeprecationWarning)\n",
    "\n",
    "sess = sagemaker.Session()\n",
    "BUCKET = sess.default_bucket()\n",
    "\n",
    "EXP_NAME = \"label-text/text-classification\"  # Any valid S3 prefix.\n",
    "RUN_FULL_AL_DEMO = False  # See 'Cost and Runtime' in the Markdown cell above!\n",
    "VERIFY_USING_PRIVATE_WORKFORCE = True # private team leveraged for labelling job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# cell 02\n",
    "# Make sure the bucket is in the same region as this notebook.\n",
    "role = sagemaker.get_execution_role()\n",
    "region = boto3.session.Session().region_name\n",
    "\n",
    "s3 = boto3.client(\"s3\")\n",
    "bucket_region = s3.head_bucket(Bucket=BUCKET)[\"ResponseMetadata\"][\"HTTPHeaders\"][\n",
    "    \"x-amz-bucket-region\"\n",
    "]\n",
    "assert (\n",
    "    bucket_region == region\n",
    "), \"You S3 bucket {} and this notebook need to be in the same region.\".format(BUCKET)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run a Ground Truth labeling job\n",
    "**This section should take about 20h to complete.**\n",
    "\n",
    "We will first run a labeling job. This involves several steps: collecting the reviews text we want labeled, specifying the possible label categories, creating instructions, and writing a labeling job specification. In addition, we highly recommend to run a (free) mock job using a private workforce before you submit any job to the public workforce. This notebook will explain how to do that as an optional step. Without using a private workforce, this section until completion of your labeling job should take about 20h. However, this may vary depending on the availability of the public annotation workforce.\n",
    "\n",
    "## Prepare the data\n",
    "We will first download IMDB reviews subset of the [IMDB Movie Review Dataset](https://huggingface.co/datasets/pietrolesci/imdb). Later, will compare Ground Truth annotations to these labels. Our dataset will include labels in the following categories:\n",
    "\n",
    "* Positive\n",
    "* Negative\n",
    "\n",
    "\n",
    "If you chose `RUN_FULL_AL_DEMO = False`, then we will choose a subset of 10 reviews text in this dataset. This is a diverse dataset of interesting reviews about movies, and should be fun for the human annotators to work with. You are free to ask the annotators to annotate the sentiments.\n",
    "\n",
    "We will copy these images to our local `BUCKET`, and will create the corresponding *input manifest*. The input manifest is a formatted list of the S3 locations of the images we want Ground Truth to annotate. We will upload this manifest to our S3 `BUCKET`.\n",
    "\n",
    "#### Disclosure regarding the Open IMDB Review Dataset V1.0:\n",
    "Movie Review Dataset V1 is created by Stanford AI Lab. We have not modified the text or the accompanying annotations. You can obtain the text and the annotations [here](http://ai.stanford.edu/~amaas/data/sentiment/).  The following paper describes the learning of word vectors for Sentiment Analysis.\n",
    "\n",
    "Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. (2011). \n",
    "*Learning Word Vectors for Sentiment Analysis. The 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011).*([link to PDF](https://aclanthology.org/P11-1015/))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# cell 03\n",
    "# Download the data if not exists\n",
    "if not os.path.isfile('tc_dataset.json'):\n",
    "    !curl -X GET \\\n",
    "     \"https://datasets-server.huggingface.co/rows?dataset=pietrolesci%2Fimdb&config=pietrolesci--imdb&split=train&offset=0&limit=100\"  >> tc_dataset.json\n",
    "\n",
    "\n",
    "read_file = pd.DataFrame(columns = ['source'])\n",
    "text_list = []\n",
    "# Read and store content of an json file \n",
    "with open ('tc_dataset.json', \"r\") as f:\n",
    "    # Reading from file\n",
    "    data = json.loads(f.read())\n",
    "\n",
    "    # Iterating through the json list\n",
    "    for i in data['rows']:\n",
    "        text_list.append(i['row']['text'])\n",
    "# remove html tags\n",
    "text_list = [i.replace('<br>', '').replace('</br>', '').replace('<br />','') for i in text_list]\n",
    "\n",
    "read_file['source'] = text_list\n",
    "\n",
    "\n",
    "# If running the short version of the demo, reduce each class count 10 times.\n",
    "if RUN_FULL_AL_DEMO is False:\n",
    "    text_data = read_file.sample(n=10)\n",
    "else:\n",
    "    text_data = read_file\n",
    "\n",
    "# Create a manifest (jsonline) file\n",
    "text_data.to_json(r'./input.manifest', orient='records', lines=True)\n",
    "\n",
    "# Upload the input manifest file to s3\n",
    "manifest_name = \"input/input.manifest\"\n",
    "s3.upload_file(r'./input.manifest', BUCKET, EXP_NAME + \"/\" + manifest_name)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After running the cell above, you should be able to go to `s3://BUCKET/EXP_NAME/input` in [S3 console](https://console.aws.amazon.com/s3/) and see a 10k reviews. We recommend you inspect the contents of these content! You can download them all to a local machine using the AWS CLI."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Specify the categories\n",
    "To run the text classification labeling job, you need to decide on a set of classes the annotators can choose from. \n",
    "In our case, this list is `[\"Positive\", \"Negative\"]`. In your own job you can choose any list of up to 3 classes. We recommend the classes to be as unambiguous and concrete as possible. The categories should be mutually exclusive, with only one correct label per text. \n",
    "\n",
    "To work with Ground Truth, this list needs to be converted to a .json file and uploaded to the S3 `BUCKET`.\n",
    "\n",
    "*Note: The ordering of the labels or classes in the template governs the class indices that you will see downstream in the output manifest (this numbering is zero-indexed). In other words, the class that appears second in the template will correspond to class \"1\" in the output. At the end of this demonstration, we will train a model and make predictions, and this class ordering is instrumental to interpreting the results.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# cell 04\n",
    "# deine the classes\n",
    "import json\n",
    "json_body = {\"document-version\":\"2021-05-13\",\n",
    "            \"labels\":[{\"label\":\"Positive\"},\n",
    "                     {\"label\":\"Negative\"}]\n",
    "            }\n",
    "\n",
    "with open(\"class_labels.json\", \"w\") as f:\n",
    "    json.dump(json_body, f)\n",
    "    \n",
    "# upload the json file to s3\n",
    "s3.upload_file(\"class_labels.json\", BUCKET, EXP_NAME + \"/input/class_labels.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should now see `class_labels.json` in `s3://BUCKET/EXP_NAME/input`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the instruction template\n",
    "Part or all of your revviews will be annotated by human annotators. It is **essential** to provide good instructions that help the annotators give you the annotations you want. Good instructions are:\n",
    "1. Concise. We recommend limiting verbal/textual instruction to two sentences, and focusing on clear visuals.\n",
    "2. Visual. In the case of text classification, we recommend providing one example in each of the classes as part of the instruction.\n",
    "\n",
    "When used through the AWS Console, Ground Truth helps you create the instructions using a visual wizard. When using the API, you need to create an HTML template for your instructions. Below, we prepare a very simple but effective template and upload it to your S3 bucket.\n",
    "\n",
    "#### Testing your instructions\n",
    "It is very easy to create broken instructions. This might cause your labeling job to fail. However, it might also cause your job to complete with meaningless results (when the annotators have no idea what to do, or the instructions are plain wrong). We *highly recommend* that you verify that your task is correct in two ways:\n",
    "1. The following cell creates and uploads a file called `template.liquid` to S3. It also creates `instructions.html` that you can open in a local browser window. Please do so and inspect the resulting web page; it should correspond to what you want your annotators to see (except the actual image to annotate will not be visible).\n",
    "2. Run your job in a private workforce, which is a way to run a mock labeling job. We describe how to do it in [Verify your task using a private team [OPTIONAL]](#Verify-your-task-using-a-private-team-[OPTIONAL])."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<script src=\"https://assets.crowd.aws/crowd-html-elements.js\"></script>\n",
      "        <crowd-form>\n",
      "          <crowd-classifier\n",
      "            name=\"crowd-classifier\"\n",
      "            categories=\"{{ task.input.labels | to_json | escape }}\"\n",
      "            header=\"TEXT CLASSIFICATION (SINGLE LABEL)\"\n",
      "          >\n",
      "            <classification-target style=\"white-space: pre-wrap\">\n",
      "              {{ task.input.taskObject }}\n",
      "            </classification-target>\n",
      "            <full-instructions header=\"Classifier instructions\">\n",
      "              <ol><li><strong>Read</strong> the text carefully.</li>\n",
      "              <li><strong>Read</strong> the examples to understand more about the options.</li>\n",
      "              <li><strong>Choose</strong> the appropriate labels that best suit the text.</li></ol>\n",
      "            </full-instructions>\n",
      "            <short-instructions>\n",
      "              <p>Enter description of the labels/sentiments that workers have to choose from</p>\n",
      "              <p>Positive : If you are positive about things, you are hopeful and confident, and think of the good aspects of a situation rather than the bad ones.\". </p>\n",
      "              <p>Negative : A fact, situation, or experience that is negative is unpleasant, depressing, or harmful.\". </p>\n",
      "                 \n",
      "              <p><br></p><p><br></p><p>Add examples to help workers understand the label</p>\n",
      "              <p> Positive : “good”, “great”, “wonderful”, “fantastic”.</p>\n",
      "              <p> Negative : “bad”, “terrible”, “awful”, “disgusting”.</p>\n",
      "              <p><br></p><p><br></p><p><br></p><p><br></p><p><br></p>\n",
      "            </short-instructions>\n",
      "          </crowd-classifier>\n",
      "          </crowd-form>\n"
     ]
    }
   ],
   "source": [
    "# cell 05\n",
    "# a template for text classification\n",
    "def make_template(test_template=False, save_fname=\"instructions.template\"):\n",
    "    template = r\"\"\"<script src=\"https://assets.crowd.aws/crowd-html-elements.js\"></script>\n",
    "        <crowd-form>\n",
    "          <crowd-classifier\n",
    "            name=\"crowd-classifier\"\n",
    "            categories=\"{{ task.input.labels | to_json | escape }}\"\n",
    "            header=\"TEXT CLASSIFICATION (SINGLE LABEL)\"\n",
    "          >\n",
    "            <classification-target style=\"white-space: pre-wrap\">\n",
    "              {{ task.input.taskObject }}\n",
    "            </classification-target>\n",
    "            <full-instructions header=\"Classifier instructions\">\n",
    "              <ol><li><strong>Read</strong> the text carefully.</li>\n",
    "              <li><strong>Read</strong> the examples to understand more about the options.</li>\n",
    "              <li><strong>Choose</strong> the appropriate labels that best suit the text.</li></ol>\n",
    "            </full-instructions>\n",
    "            <short-instructions>\n",
    "              <p>Enter description of the labels/sentiments that workers have to choose from</p>\n",
    "              <p>Positive : If you are positive about things, you are hopeful and confident, and think of the good aspects of a situation rather than the bad ones.\". </p>\n",
    "              <p>Negative : A fact, situation, or experience that is negative is unpleasant, depressing, or harmful.\". </p>\n",
    "                 \n",
    "              <p><br></p><p><br></p><p>Add examples to help workers understand the label</p>\n",
    "              <p> Positive : “good”, “great”, “wonderful”, “fantastic”.</p>\n",
    "              <p> Negative : “bad”, “terrible”, “awful”, “disgusting”.</p>\n",
    "              <p><br></p><p><br></p><p><br></p><p><br></p><p><br></p>\n",
    "            </short-instructions>\n",
    "          </crowd-classifier>\n",
    "          </crowd-form>\"\"\"\n",
    "    \n",
    "          \n",
    "    with open(save_fname, \"w\") as f:\n",
    "        f.write(template)\n",
    "    if test_template is False:\n",
    "        print(template)\n",
    "\n",
    "\n",
    "#make_template(test_template=True, save_fname=\"instructions.html\")\n",
    "make_template(test_template=False, save_fname=\"template.liquid\")\n",
    "s3.upload_file(\"template.liquid\", BUCKET, EXP_NAME + \"/input/template.liquid\")\n",
    "#s3.upload_file(\"instructions.html\", BUCKET, EXP_NAME + \"/instructions.html\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should now be able to find your template in `s3://BUCKET/EXP_NAME/instructions.template`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Assign private team to test your task\n",
    "\n",
    "Refer to Prerequisites to setup private workforce team. \n",
    "\n",
    "Copy private_workteam_arn, from Amazon SageMaker console > Ground Truth > Labeling workforces > Private Teams\n",
    "\n",
    "The [SageMaker Ground Truth documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-workforce-management-private.html) has more details on the management of private workteams. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WORKTEAM_ARN : arn:aws:sagemaker:us-east-1:370501389570:workteam/private-crowd/smgt-immersionday\n"
     ]
    }
   ],
   "source": [
    "# cell 06\n",
    "# choose the workforce team\n",
    "if VERIFY_USING_PRIVATE_WORKFORCE:\n",
    "    private_workteam_arn = \"<< your private workteam ARN here >>\"\n",
    "    WORKTEAM_ARN = private_workteam_arn\n",
    "else:\n",
    "    workteam_arn = \"arn:aws:sagemaker:{}:394669845002:workteam/public-crowd/default\".format(region)\n",
    "    WORKTEAM_ARN = workteam_arn\n",
    "\n",
    "print(\"WORKTEAM_ARN : {}\".format(WORKTEAM_ARN))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define pre-built lambda functions for use in the labeling job \n",
    "\n",
    "Before we submit the request, we need to define the ARNs for four key components of the labeling job: 1) the workteam, 2) the annotation consolidation Lambda function, 3) the pre-labeling task Lambda function, and 4) the machine learning algorithm to perform auto-annotation. These functions are defined by strings with region names and AWS service account numbers, so we will define a mapping below that will enable you to run this notebook in any of our supported regions. \n",
    "\n",
    "See the official documentation for the available ARNs:\n",
    "* [Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-workforce-management-public.html) for a discussion of the workteam ARN definition. There is only one valid selection if you choose to use the public workfofce; if you elect to use a private workteam, you should check the corresponding ARN for the workteam.\n",
    "* [Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/API_HumanTaskConfig.html#SageMaker-Type-HumanTaskConfig-PreHumanTaskLambdaArn) for available pre-human ARNs for other workflows.\n",
    "* [Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/API_AnnotationConsolidationConfig.html#SageMaker-Type-AnnotationConsolidationConfig-AnnotationConsolidationLambdaArn) for available annotation consolidation ANRs for other workflows.\n",
    "* [Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/API_LabelingJobAlgorithmsConfig.html#SageMaker-Type-LabelingJobAlgorithmsConfig-LabelingJobAlgorithmSpecificationArn) for available auto-labeling ARNs for other workflows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# cell 07\n",
    "# Specify ARNs for resources needed to run an text classification job.\n",
    "ac_arn_map = {\n",
    "    \"us-west-2\": \"081040173940\",\n",
    "    \"us-east-1\": \"432418664414\",\n",
    "    \"us-east-2\": \"266458841044\",\n",
    "    \"eu-west-1\": \"568282634449\",\n",
    "    \"ap-northeast-1\": \"477331159723\",\n",
    "}\n",
    "# PreHumanTaskLambdaArn for text classification(single)\n",
    "prehuman_arn = \"arn:aws:lambda:{}:{}:function:PRE-TextMultiClass\".format(\n",
    "    region, ac_arn_map[region]\n",
    ")\n",
    "\n",
    "# AnnotationConsolidationConfig for text classification(single)\n",
    "acs_arn = \"arn:aws:lambda:{}:{}:function:ACS-TextMultiClass\".format(region, ac_arn_map[region])\n",
    "\n",
    "# auto-labelling job\n",
    "labeling_algorithm_specification_arn = \"arn:aws:sagemaker:{}:027400017018:labeling-job-algorithm-specification/text-classification\".format(\n",
    "    region\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submit the Ground Truth job request\n",
    "The API starts a Ground Truth job by submitting a request. The request contains the \n",
    "full configuration of the annotation task, and allows you to modify the fine details of\n",
    "the job that are fixed to default values when you use the AWS Console. The parameters that make up the request are described in more detail in the [SageMaker Ground Truth documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateLabelingJob.html).\n",
    "\n",
    "After you submit the request, you should be able to see the job in your AWS Console, at `Amazon SageMaker > Labeling Jobs`.\n",
    "You can track the progress of the job there. This job will take several hours to complete. If your job\n",
    "is larger (say 10,000 review text), the speed and cost benefit of auto-labeling should be larger.\n",
    "\n",
    "### Verify your task using a private team [OPTIONAL]\n",
    "If you chose to follow the steps in [Create a private team](#Create-a-private-team-to-test-your-task-[OPTIONAL]), then you can first verify that your task runs as expected. To do this:\n",
    "1. Set VERIFY_USING_PRIVATE_WORKFORCE to True in the cell below.\n",
    "2. Run the next two cells. This will define the task and submit it to the private workforce (to you).\n",
    "3. After a few minutes, you should be able to see your task in your private workforce interface [Create a private team](#Create-a-private-team-to-test-your-task-[OPTIONAL]).\n",
    "Please verify that the task appears as you want it to appear.\n",
    "4. If everything is in order, change `VERIFY_USING_PRIVATE_WORKFORCE` to `False` and rerun the cell below to start the real annotation task!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'LabelingJobArn': 'arn:aws:sagemaker:us-east-1:370501389570:labeling-job/ground-truth-text-classification-1686338403',\n",
       " 'ResponseMetadata': {'RequestId': '29a05986-717d-4ef2-9357-3de1e32c0b39',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'x-amzn-requestid': '29a05986-717d-4ef2-9357-3de1e32c0b39',\n",
       "   'content-type': 'application/x-amz-json-1.1',\n",
       "   'content-length': '118',\n",
       "   'date': 'Fri, 09 Jun 2023 19:20:03 GMT'},\n",
       "  'RetryAttempts': 0}}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# cell 08\n",
    "# task definitions\n",
    "task_description = 'Carefully read and classify this text using the categories provided.'\n",
    "task_keywords = ['Text classification']\n",
    "task_title = 'Text Classification (Single Label)'\n",
    "job_name = \"ground-truth-text-classification-\" + str(int(time.time()))\n",
    "USE_AUTO_LABELING = False\n",
    "\n",
    "# define human task config\n",
    "human_task_config={\n",
    "        'PreHumanTaskLambdaArn': prehuman_arn,\n",
    "        'TaskKeywords': task_keywords,\n",
    "        'TaskTitle': task_title,\n",
    "        'TaskDescription': task_description ,\n",
    "        'NumberOfHumanWorkersPerDataObject': 1,  # number of  workers required to label each text.\n",
    "        'TaskTimeLimitInSeconds': 300,  # Each text must be labeled within 5 minutes.\n",
    "        'TaskAvailabilityLifetimeInSeconds': 21600,  # Your worteam has 6 hours to complete all pending tasks.\n",
    "        'MaxConcurrentTaskCount': 100,   # 100 text sentences will be sent at a time to the workteam.\n",
    "        'AnnotationConsolidationConfig': {\n",
    "        'AnnotationConsolidationLambdaArn': acs_arn,\n",
    "        },\n",
    "        'UiConfig': {\n",
    "            'UiTemplateS3Uri': \"s3://{}/{}/input/template.liquid\".format(BUCKET, EXP_NAME),\n",
    "        },\n",
    "\n",
    "    }\n",
    "    \n",
    "\n",
    "if not VERIFY_USING_PRIVATE_WORKFORCE:\n",
    "    human_task_config[\"PublicWorkforceTaskPrice\"] = {\n",
    "        \"AmountInUsd\": {\n",
    "            \"Dollars\": 0,\n",
    "            \"Cents\": 1,\n",
    "            \"TenthFractionsOfACent\": 2,\n",
    "        }\n",
    "    }\n",
    "    human_task_config[\"WorkteamArn\"] = workteam_arn\n",
    "else:\n",
    "    human_task_config[\"WorkteamArn\"] = private_workteam_arn\n",
    "\n",
    "ground_truth_request = {\n",
    "    \"InputConfig\":{\n",
    "        'DataSource': {\n",
    "            'S3DataSource': {\n",
    "                'ManifestS3Uri': \"s3://{}/{}/{}\".format(BUCKET, EXP_NAME, manifest_name),\n",
    "            }\n",
    "        },\n",
    "        'DataAttributes': {\n",
    "            'ContentClassifiers': [\n",
    "                'FreeOfPersonallyIdentifiableInformation','FreeOfAdultContent',\n",
    "            ]\n",
    "        }\n",
    "    },\n",
    "    \"OutputConfig\":{\n",
    "        'S3OutputPath': \"s3://{}/{}/output/\".format(BUCKET, EXP_NAME),\n",
    "    },\n",
    "    \n",
    "    \"HumanTaskConfig\": human_task_config,\n",
    "    \"LabelingJobName\": job_name,\n",
    "    \"RoleArn\": role,\n",
    "    \"LabelAttributeName\": \"label\",\n",
    "    \"LabelCategoryConfigS3Uri\": \"s3://{}/{}/input/class_labels.json\".format(BUCKET, EXP_NAME),\n",
    "    \"Tags\":[\n",
    "        {\n",
    "            'Key': 'text',\n",
    "            'Value': 'text_classification'\n",
    "        },\n",
    "    ]\n",
    "\n",
    "\n",
    "}\n",
    "\n",
    "if USE_AUTO_LABELING and RUN_FULL_AL_DEMO:\n",
    "    ground_truth_request[\"LabelingJobAlgorithmsConfig\"] = {\n",
    "        \"LabelingJobAlgorithmSpecificationArn\": labeling_algorithm_specification_arn\n",
    "    }\n",
    "sagemaker_client = boto3.client(\"sagemaker\")\n",
    "sagemaker_client.create_labeling_job(**ground_truth_request)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Navigate to the private worker portal and do the tasks. Make sure you've invited yourself to your workteam!\n",
      "https://so3zqudq9a.labeling.us-east-1.sagemaker.aws\n"
     ]
    }
   ],
   "source": [
    "# cell 09\n",
    "# navigate to the private worker portal and do the tasks\n",
    "workteamName = WORKTEAM_ARN[WORKTEAM_ARN.rfind('/') + 1:]\n",
    "print(\"Navigate to the private worker portal and do the tasks. Make sure you've invited yourself to your workteam!\")\n",
    "print('https://' + sagemaker_client.describe_workteam(WorkteamName=workteamName)['Workteam']['SubDomain'])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Monitor job progress\n",
    "\n",
    "A Ground Truth job can take a few hours to complete (if your dataset is larger than 1000 reviews, it can take much longer than that!). One way to monitor the job's progress is through AWS Console. In this notebook, we will use Ground Truth output files and Cloud Watch logs in order to monitor the progress. You can re-evaluate the next two cells repeatedly.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can re-evaluate the next cell repeatedly. It sends a `describe_labelging_job` request which should tell you whether the job is completed or not. If it is, then 'LabelingJobStatus' will be 'Completed'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : InProgress\n",
      "Labelling job : ground-truth-text-classification-1686338403, status : Completed\n"
     ]
    }
   ],
   "source": [
    "# cell 10\n",
    "# re-evaluate repeatedly. It sends a `describe_labelging_job` request which should tell you whether the job is completed or not. If it is, then 'LabelingJobStatus' will be 'Completed'.\n",
    "while sagemaker_client.describe_labeling_job(LabelingJobName=job_name)['LabelingJobStatus'] == 'InProgress':\n",
    "    job_status = sagemaker_client.describe_labeling_job(LabelingJobName=job_name)['LabelingJobStatus']\n",
    "    print('Labelling job : {}, status : {}'.format(job_name, job_status))\n",
    "    time.sleep(30)\n",
    "print('Labelling job : {}, status : {}'.format(job_name, sagemaker_client.describe_labeling_job(LabelingJobName=job_name)['LabelingJobStatus']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Wait For Workers to Complete Task!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View Task Results\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once work is completed, SagMaker GroundTruth stores results in your S3 bucket and sends a Cloudwatch event. Your results should be available in the S3 OUTPUT_PATH when all work is completed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "S3 OUPUT_PATH : s3://sagemaker-us-east-1-370501389570/label-text/text-classification/output/ground-truth-text-classification-1686338403\n",
      "download: s3://sagemaker-us-east-1-370501389570/label-text/text-classification/output/ground-truth-text-classification-1686338403/manifests/output/output.manifest to ./output.manifest\n"
     ]
    }
   ],
   "source": [
    "# cell 11\n",
    "# ouput path\n",
    "S3_OUTPUT = boto3.client('sagemaker').describe_labeling_job(LabelingJobName=job_name)['OutputConfig']['S3OutputPath'] + job_name\n",
    "print('S3 OUPUT_PATH : {}'.format(S3_OUTPUT))\n",
    "\n",
    "# Download human annotation data.\n",
    "!aws s3 cp {S3_OUTPUT + '/manifests/output/output.manifest'} ./output.manifest #--recursive --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>label</th>\n",
       "      <th>label-metadata</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I rented I AM CURIOUS-YELLOW from my video sto...</td>\n",
       "      <td>1</td>\n",
       "      <td>{'class-name': 'Negative', 'job-name': 'labeli...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>\"I Am Curious: Yellow\" is a risible and preten...</td>\n",
       "      <td>0</td>\n",
       "      <td>{'class-name': 'Positive', 'job-name': 'labeli...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>If only to avoid making this type of film in t...</td>\n",
       "      <td>0</td>\n",
       "      <td>{'class-name': 'Positive', 'job-name': 'labeli...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>This film was probably inspired by Godard's Ma...</td>\n",
       "      <td>1</td>\n",
       "      <td>{'class-name': 'Negative', 'job-name': 'labeli...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Oh, brother...after hearing about this ridicul...</td>\n",
       "      <td>0</td>\n",
       "      <td>{'class-name': 'Positive', 'job-name': 'labeli...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              source  label  \\\n",
       "0  I rented I AM CURIOUS-YELLOW from my video sto...      1   \n",
       "1  \"I Am Curious: Yellow\" is a risible and preten...      0   \n",
       "2  If only to avoid making this type of film in t...      0   \n",
       "3  This film was probably inspired by Godard's Ma...      1   \n",
       "4  Oh, brother...after hearing about this ridicul...      0   \n",
       "\n",
       "                                      label-metadata  \n",
       "0  {'class-name': 'Negative', 'job-name': 'labeli...  \n",
       "1  {'class-name': 'Positive', 'job-name': 'labeli...  \n",
       "2  {'class-name': 'Positive', 'job-name': 'labeli...  \n",
       "3  {'class-name': 'Negative', 'job-name': 'labeli...  \n",
       "4  {'class-name': 'Positive', 'job-name': 'labeli...  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# cell 12\n",
    "data = []\n",
    "with open('./output.manifest') as f:\n",
    "    for line in f:\n",
    "         data.append(json.loads(line))\n",
    "pd.DataFrame(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clean Up [OPTIONAL]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's clean up and delete this labeling job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# cell 16\n",
    "if sagemaker_client.describe_labeling_job(LabelingJobName=job_name)['LabelingJobStatus'] == 'InProgress':\n",
    "    sagemaker_client.stop_labeling_job(LabelingJobName=job_name) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The End!"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}