{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sign & Speak ML Instructions\n",
    "\n",
    "This notebook shows how to use Amazon SageMaker to run the training and inference scripts for the Sign & Speak project.\n",
    "\n",
    "Use the `conda_pytorch_p36` kernel to run the cells in this notebook.\n",
    "\n",
    "## Training\n",
    "\n",
    "The following cell defines the training job to be run by Amazon SageMaker. It points to the `grid_train.py` training script, defines the number and types of instances used for training, sets the hyperparameter values, and defines regular expressions which Amazon SageMaker uses to track the training metrics.\n",
    "\n",
    "Before running this cell, you must provide a descriptive name for the training job and specify the Amazon S3 URI where the output should be stored. The URI should look like `s3://bucket-name/output-folder/`.\n",
    "\n",
    "*Note: If you are using a new AWS account, you may not have access to p2 instance types yet. The code should run fine on a CPU instance type, but it will require more time to complete. Submit a limit increase request to use p2 instances.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "from sagemaker.pytorch import PyTorch\n",
    "\n",
    "# Replace the following variables with a descriptive name for the\n",
    "# training job and an S3 URI where to store the output\n",
    "JOB_NAME = 'INSERT_A_NAME_HERE'\n",
    "OUTPUT_PATH = 'INSERT_AN_S3_URI_HERE'\n",
    "\n",
    "role = sagemaker.get_execution_role()\n",
    "\n",
    "estimator = PyTorch(entry_point='grid_train.py',\n",
    "                    role=role,\n",
    "                    base_job_name=JOB_NAME,\n",
    "                    output_path=OUTPUT_PATH,\n",
    "                    framework_version='1.1.0',\n",
    "                    train_instance_count=1,\n",
    "                    train_instance_type='ml.p2.xlarge',\n",
    "                    hyperparameters={\n",
    "                        \"epochs\": 10,\n",
    "                        \"batch-size\": 4,\n",
    "                        \"gamma\": 0.1,\n",
    "                        \"lr\": 0.001,\n",
    "                        \"momentum\": 0.9,\n",
    "                        \"step-size\": 7\n",
    "                    },\n",
    "                    metric_definitions=[\n",
    "                        {'Name': 'train:loss', 'Regex': 'train Loss: (.*?) '},\n",
    "                        {'Name': 'train:acc', 'Regex': 'train Loss: .*? Acc: (.*?)$'},\n",
    "                        {'Name': 'val:loss', 'Regex': 'val Loss: (.*?) '},\n",
    "                        {'Name': 'val:acc', 'Regex': 'val Loss: .*? Acc: (.*?)$'}\n",
    "                    ]\n",
    "                   )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the training job has been defined, pass in the Amazon S3 URI for the training data to start the training job. The URI should look like `s3://bucket-name/training-data-folder/`, where `training-data-folder` contains one folder per label containing the training images for that label. \n",
    "\n",
    "This cell will output the logs of the training job, but you can also view the logs and visualize the metrics in the Amazon SageMaker console."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "estimator.fit({'training': 'INSERT_AN_S3_URI_HERE'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparameter Tuning\n",
    "This section shows how to run a hyperparameter tuning job using Amazon SageMaker. First, define the range of values for the hyperparameters which you want to tune."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.tuner import IntegerParameter, CategoricalParameter, ContinuousParameter\n",
    "\n",
    "hyperparameter_ranges = {\n",
    "    'batch-size': IntegerParameter(3,30,scaling_type='Auto'),\n",
    "    'momentum': ContinuousParameter(0.1, 0.9, scaling_type='Auto'),\n",
    "    'step-size': IntegerParameter(3, 12, scaling_type='Auto'),\n",
    "    'gamma': ContinuousParameter(0.01, 0.9, scaling_type='Auto')\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, define the training jobs which will be run during hyperparameter tuning. This is the same as in the above section on training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "from sagemaker.pytorch import PyTorch\n",
    "\n",
    "# Replace the following variables with a descriptive name for the\n",
    "# training job and an S3 URI where to store the output\n",
    "JOB_NAME = 'INSERT_A_NAME_HERE'\n",
    "OUTPUT_PATH = 'INSERT_AN_S3_URI_HERE'\n",
    "\n",
    "role = sagemaker.get_execution_role()\n",
    "\n",
    "estimator = PyTorch(entry_point='grid_train.py',\n",
    "                    role=role,\n",
    "                    base_job_name=JOB_NAME\n",
    "                    output_path=OUTPUT_PATH,\n",
    "                    framework_version='1.1.0',\n",
    "                    train_instance_count=1,\n",
    "                    train_instance_type='ml.p2.xlarge',\n",
    "                    hyperparameters={\n",
    "                        \"epochs\": 20,\n",
    "                        \"lr\": 0.001\n",
    "                    },\n",
    "                    metric_definitions=[\n",
    "                        {'Name': 'train:loss', 'Regex': 'train Loss: (.*?) '},\n",
    "                        {'Name': 'train:acc', 'Regex': 'train Loss: .*? Acc: (.*?)$'},\n",
    "                        {'Name': 'val:loss', 'Regex': 'val Loss: (.*?) '},\n",
    "                        {'Name': 'val:acc', 'Regex': 'val Loss: .*? Acc: (.*?)$'}\n",
    "                    ]\n",
    "                   )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, define the hyperparameter tuning job based on the defined hyperparameter ranges. Set the objective metric, the maximum number of training jobs, and the maximum number of parallel training jobs. \n",
    "\n",
    "*Note: make sure your AWS account limits allow for the number of parallel training jobs for the instance type defined in the training job.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.tuner import HyperparameterTuner\n",
    "\n",
    "TUNING_JOB_NAME = 'INSERT_A_NAME_HERE'\n",
    "\n",
    "tuner = HyperparameterTuner(\n",
    "    estimator=estimator,\n",
    "    objective_metric_name='val:acc',\n",
    "    hyperparameter_ranges=hyperparameter_ranges,\n",
    "    metric_definitions=[\n",
    "                        {'Name': 'train:loss', 'Regex': 'train Loss: (.*?) '},\n",
    "                        {'Name': 'train:acc', 'Regex': 'train Loss: .*? Acc: (.*?)$'},\n",
    "                        {'Name': 'val:loss', 'Regex': 'val Loss: (.*?) '},\n",
    "                        {'Name': 'val:acc', 'Regex': 'val Loss: .*? Acc: (.*?)$'}\n",
    "                    ],\n",
    "    strategy='Bayesian',\n",
    "    objective_type='Maximize',\n",
    "    max_jobs=30,\n",
    "    max_parallel_jobs=3,\n",
    "    base_tuning_job_name=TUNING_JOB_NAME\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the tuning job has been defined, pass in the Amazon S3 URI for the training data to start the tuning job. The URI should look like `s3://bucket-name/training-data-folder/`, where `training-data-folder` contains one folder per label containing the training images for that label.\n",
    "\n",
    "View the logs and visualize the metrics for the training jobs linked to this tuning job in the Amazon SageMaker console."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tuner.fit(inputs='INSERT_AN_S3_URI_HERE')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploying\n",
    "\n",
    "After running some training jobs and/or hyperparameter tuning jobs, decide on which training job you want to base your deployment. Find the Amazon S3 URI of the model package, which should look like `s3://bucket-name/training-job-name/output/model.tar.gz`. Insert the URI in the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.predictor import RealTimePredictor, json_serializer, json_deserializer\n",
    "\n",
    "class JSONPredictor(RealTimePredictor):\n",
    "    def __init__(self, endpoint_name, sagemaker_session):\n",
    "        super(JSONPredictor, self).__init__(endpoint_name, sagemaker_session, json_serializer, json_deserializer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.pytorch import PyTorchModel\n",
    "import sagemaker\n",
    "\n",
    "role = sagemaker.get_execution_role()\n",
    "model = PyTorchModel(model_data='INSERT_S3_URI_OF_MODEL_PACKAGE',\n",
    "                     role=role,\n",
    "                     framework_version='1.1.0',\n",
    "                     entry_point='grid_serve.py',\n",
    "                     predictor_cls=JSONPredictor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After defining the model and predictor type, we specify the number and type of instances for running the endpoint. \n",
    "\n",
    "*Note: An endpoint takes several minutes to start up.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = model.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once your endpoint is complete, note down the name to link it up to the Sign & Speak user interface."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_p36",
   "language": "python",
   "name": "conda_pytorch_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}