{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "greater-dispatch",
   "metadata": {},
   "outputs": [],
   "source": [
    "# install dependencies\n",
    "!pip install smdebug"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cutting-singapore",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Profiling PyTorch Single GPU Single Node Training Job with Amazon SageMaker Debugger\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "befba7c5",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-2/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6555e555",
   "metadata": {
    "tags": []
   },
   "source": [
    "\n",
    "This notebook will walk you through creating a PyTorch training job with the SageMaker Debugger profiling feature enabled. It will create a single GPU single node training.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "plain-interaction",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Install sagemaker and smdebug\n",
    "To use the new Debugger profiling features, ensure that you have the latest versions of SageMaker and SMDebug SDKs installed. The following cell updates the libraries and restarts the Jupyter kernel to apply the updates."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fatty-benefit",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 1. Create a Training Job with Profiling Enabled<a class=\"anchor\" id=\"option-1\"></a>\n",
    "\n",
    "You will use the standard [SageMaker Estimator API for PyTorch ](https://sagemaker.readthedocs.io/en/stable/frameworks/tensorflow/sagemaker.pytorch.html) to create training jobs. To enable profiling, create a `ProfilerConfig` object and pass it to the `profiler_config` parameter of the `PyTorch` estimator."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "suspected-principle",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Define hyperparameters\n",
    "\n",
    "Define hyperparameters such as number of epochs, batch size, and data augmentation. You can increase batch size to increases system utilization, but it may result in CPU bottlneck problems. Data preprocessing of a large batch size with augmentation requires a heavy computation. You can disable data_augmentation to see the impact on the system utilization. \n",
    "\n",
    "For demonstration purpose, the following hyperparameters are prepared to increase CPU usage, leading to GPU starvation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ready-pasta",
   "metadata": {
    "execution": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "hyperparameters = {\n",
    "    \"batch_size\": 2048,\n",
    "    \"gpu\": True,\n",
    "    \"pin_memory\": True,\n",
    "    \"workers\": 4,\n",
    "    \"epoch\": 1,\n",
    "    \"model\": \"resnext50_32x4d\",\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "noble-currency",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Configure rules\n",
    "We specify the following rules:\n",
    "- loss_not_decreasing: checks if loss is decreasing and triggers if the loss has not decreased by a certain persentage in the last few iterations\n",
    "- LowGPUUtilization: checks if GPU is under-utilizated \n",
    "- ProfilerReport: runs the entire set of performance rules and create a final output report with further insights and recommendations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "indirect-jesus",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.debugger import Rule, ProfilerRule, rule_configs\n",
    "\n",
    "rules = [\n",
    "    Rule.sagemaker(rule_configs.loss_not_decreasing()),\n",
    "    ProfilerRule.sagemaker(rule_configs.LowGPUUtilization()),\n",
    "    ProfilerRule.sagemaker(rule_configs.ProfilerReport()),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "frozen-stuart",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Specify a profiler configuration\n",
    "The following configuration will capture system metrics at 500 milliseconds. The system metrics include utilization per CPU, GPU, memory utilization per CPU, GPU as well I/O and network.\n",
    "\n",
    "Debugger will capture detailed profiling information from step 5 to step 15. This information includes Horovod metrics, dataloading, preprocessing, operators running on CPU and GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ranking-merchant",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.debugger import ProfilerConfig, FrameworkProfile\n",
    "\n",
    "profiler_config = ProfilerConfig(\n",
    "    system_monitor_interval_millis=500, framework_profile_params=FrameworkProfile(num_steps=10)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "convinced-computer",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Get the image URI\n",
    "The image that we will is dependent on the region that you are running this notebook in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "found-distance",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "\n",
    "session = boto3.session.Session()\n",
    "region = session.region_name\n",
    "\n",
    "image_uri = (\n",
    "    f\"763104351884.dkr.ecr.{region}.amazonaws.com/pytorch-training:1.6.0-gpu-py36-cu110-ubuntu18.04\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mediterranean-founder",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Define estimator\n",
    "\n",
    "To enable profiling, you need to pass the Debugger profiling configuration (`profiler_config`), a list of Debugger rules (`rules`), and the image URI (`image_uri`) to the estimator. Debugger enables monitoring and profiling while the SageMaker estimator requests a training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "corporate-spell",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "from sagemaker.pytorch import PyTorch\n",
    "\n",
    "estimator = PyTorch(\n",
    "    role=sagemaker.get_execution_role(),\n",
    "    image_uri=image_uri,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.p3.2xlarge\",\n",
    "    source_dir=\"entry_point\",\n",
    "    entry_point=\"pytorch_res50_cifar10_dataloader.py\",\n",
    "    hyperparameters=hyperparameters,\n",
    "    profiler_config=profiler_config,\n",
    "    rules=rules,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "honey-studio",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "estimator.fit()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "critical-future",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 2.  Analyze Profiling Data\n",
    "\n",
    "Copy outputs of the following cell (`training_job_name` and `region`) to run the analysis notebooks `profiling_generic_dashboard.ipynb`, `analyze_performance_bottlenecks.ipynb`, and `profiling_interactive_analysis.ipynb`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "competitive-efficiency",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "\n",
    "session = boto3.session.Session()\n",
    "region = session.region_name\n",
    "\n",
    "training_job_name = estimator.latest_training_job.name\n",
    "print(f\"Training jobname: {training_job_name}\")\n",
    "print(f\"Region: {region}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "little-liquid",
   "metadata": {
    "tags": []
   },
   "source": [
    "While the training is still in progress you can visualize the performance data in SageMaker Studio or in the notebook.\n",
    "Debugger provides utilities to plot system metrics in form of timeline charts or heatmaps. Checkout out the notebook \n",
    "[profiling_interactive_analysis.ipynb](analysis_tools/profiling_interactive_analysis.ipynb) for more details. In the following code cell we plot the total CPU and GPU utilization as timeseries charts. To visualize other metrics such as I/O, memory, network you simply need to extend the list passed to `select_dimension` and `select_events`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "selective-bolivia",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from smdebug.profiler.analysis.notebook_utils.training_job import TrainingJob\n",
    "\n",
    "tj = TrainingJob(training_job_name, region)\n",
    "tj.wait_for_sys_profiling_data_to_be_available()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "immune-georgia",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from smdebug.profiler.analysis.notebook_utils.timeline_charts import TimelineCharts\n",
    "\n",
    "system_metrics_reader = tj.get_systems_metrics_reader()\n",
    "system_metrics_reader.refresh_event_file_list()\n",
    "\n",
    "view_timeline_charts = TimelineCharts(\n",
    "    system_metrics_reader,\n",
    "    framework_metrics_reader=None,\n",
    "    select_dimensions=[\"CPU\", \"GPU\"],\n",
    "    select_events=[\"total\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "functional-smart",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 3. Download Debugger Profiling Report\n",
    "The profiling report rule will create an html report `profiler-report.html` with a summary of builtin rules and recommenades of next steps. You can find this report in your S3 bucket.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acute-memorabilia",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "rule_output_path = estimator.output_path + estimator.latest_training_job.job_name + \"/rule-output\"\n",
    "print(f\"You will find the profiler report in {rule_output_path}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rental-vatican",
   "metadata": {
    "tags": []
   },
   "source": [
    "For more information about how to download and open the Debugger profiling report, see [SageMaker Debugger Profiling Report](https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-profiling-report.html) in the SageMaker developer guide."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "89d46997",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-east-2/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/us-west-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ca-central-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/sa-east-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-2/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-west-3/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-central-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/eu-north-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-southeast-2/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-northeast-2/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://h75twx4l60.execute-api.us-west-2.amazonaws.com/sagemaker-nb/ap-south-1/sagemaker-debugger|pytorch_profiling|pt-resnet-profiling-single-gpu-single-node.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (Data Science 2.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}