{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "74995353-48f2-492b-9d00-0979c75204f9",
   "metadata": {},
   "source": [
    "# OpenCALM SageMaker Inference with CTranslate2\n",
    "\n",
    "[Open CALM](https://huggingface.co/cyberagent/open-calm-7b) を CTranslate2 で高速化し SageMaker でデプロイするサンプルコード。\n",
    "\n",
    "検証は SageMaker Studio Notebook で ml.m5.4xlarge 上で PyTorch 2.0.0 Python 3.10 CPU Optimized コンテナで行いました。このノートブックは十分なメモリが必要なため ml.m5.4xlarge 以上のインスタンスタイプを推奨します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf5c5d23-324f-431a-8446-b5410dbc169b",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install \"sagemaker>=2.143.0\" -U\n",
    "!pip install ctranslate2 transformers torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a26cc29b-db3a-4121-bef7-00ba8d14363b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sagemaker, boto3, json\n",
    "from sagemaker import get_execution_role\n",
    "from sagemaker.pytorch.model import PyTorchModel\n",
    "from sagemaker.huggingface import HuggingFace\n",
    "\n",
    "role = get_execution_role()\n",
    "region = boto3.Session().region_name\n",
    "sess = sagemaker.Session()\n",
    "bucket = sess.default_bucket()\n",
    "\n",
    "sagemaker.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07ef5c9b-79a1-4e4e-a6f1-c8e32cf33a8a",
   "metadata": {},
   "source": [
    "## Convert Model\n",
    "\n",
    "モデルを CTranslate2 に最適化された形式に変換します。この処理はメモリを大きく利用するため十分なインスタンスサイズを選択してください。検証は m5.4xlarge で行いました。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01207fbf-74f3-42e9-8cf2-6a96aee44c99",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!ct2-transformers-converter -h"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be3f70d0-c2e7-4c56-b458-d30998d26bca",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!rm -rf scripts/model\n",
    "!ct2-transformers-converter --low_cpu_mem_usage --model cyberagent/open-calm-7b --quantization int8 --output_dir scripts/model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "017f1a87-56d8-482a-a5b2-14a9bba854d1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!ls -l scripts/model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16926e12-8afc-485b-9995-2a7ba17a662e",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Package and Upload Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73df606c-9a8a-4745-8cac-e96d9d58e422",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "!apt update -y\n",
    "!apt install pigz -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83ae32c7-55e5-460f-b615-5f26994c8d35",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%cd scripts\n",
    "# !tar -czvf ../package.tar.gz *\n",
    "!tar cv ./ | pigz -p 8 > ../package.tar.gz # 8 並列でアーカイブ\n",
    "%cd -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4c2c87c-0e01-4bab-a88e-6d3ecd963671",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_path = sess.upload_data('package.tar.gz', bucket=bucket, key_prefix=f\"OpenCALM-Inference-CTranslate2\")\n",
    "model_path"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "111978ac-8f1e-4d96-93d4-1c5cb41e6c4a",
   "metadata": {},
   "source": [
    "## Deploy Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c65d380c-abfc-411d-a3ca-3618bff5287e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.serializers import JSONSerializer\n",
    "\n",
    "endpoint_name = \"OpenCALM-Inference-CTranslate\"\n",
    "\n",
    "huggingface_model = PyTorchModel(\n",
    "    model_data=model_path,\n",
    "    framework_version=\"2.0\",\n",
    "    py_version='py310',\n",
    "    role=role,\n",
    "    name=endpoint_name,\n",
    "    env={\n",
    "        \"model_params\": json.dumps({\n",
    "            \"tokenizer\": \"cyberagent/open-calm-1b\",\n",
    "            \"model\": \"model\",\n",
    "            \"prompt_input\": \"システム: {input}ユーザー: {instruction}<NL>システム: \",\n",
    "            \"prompt_no_input\": \"ユーザー: {instruction}<NL>システム: \"\n",
    "        }),\n",
    "        \"SAGEMAKER_MODEL_SERVER_TIMEOUT\": \"3600\"\n",
    "    }\n",
    ")\n",
    "\n",
    "# deploy model to SageMaker Inference\n",
    "predictor = huggingface_model.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type='ml.g5.xlarge',\n",
    "    endpoint_name=endpoint_name,\n",
    "    serializer=JSONSerializer(),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "914536c7-03ee-42f2-a2c8-196f48403e52",
   "metadata": {},
   "source": [
    "## Inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e927e58-cf32-4d08-bb90-f42d3b9c3471",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.predictor import Predictor\n",
    "from sagemaker.serializers import JSONSerializer\n",
    "from sagemaker.deserializers import JSONDeserializer\n",
    "\n",
    "endpoint_name = \"OpenCALM-Inference-CTranslate\"\n",
    "\n",
    "predictor_client=Predictor(\n",
    "    endpoint_name=endpoint_name,\n",
    "    sagemaker_session=sess,\n",
    "    serializer=JSONSerializer(),\n",
    "    deserializer=JSONDeserializer()\n",
    ")\n",
    "data = {\n",
    "    \"instruction\": \"\"\"ヴァージン・オーストラリアはいつから運航を開始したのですか？完結に答えてください。\"\"\".replace(\"\\n\", \"<NL>\"),  # システム\n",
    "    \"input\": \"\"\"ヴァージン・オーストラリア航空（Virgin Australia Airlines Pty Ltd）の商号で、オーストラリアを拠点とする航空会社です。ヴァージン・ブランドを使用する航空会社の中で、保有機材数では最大の航空会社である。2000年8月31日にヴァージン・ブルーとして、2機の航空機で単一路線で運航を開始した[3]。2001年9月のアンセット・オーストラリアの破綻後、突然オーストラリア国内市場の大手航空会社としての地位を確立した。その後、ブリスベン、メルボルン、シドニーをハブとして、オーストラリア国内の32都市に直接乗り入れるまでに成長した[4]。\"\"\".replace(\"\\n\", \"<NL>\"),  # ユーザー\n",
    "    \"max_new_tokens\": 64,\n",
    "    \"sampling_temperature\": 0.3,\n",
    "    \"stop_ids\": [0, 1],\n",
    "}\n",
    "response = predictor_client.predict(\n",
    "    data=data\n",
    ")\n",
    "print(response.replace(\"<NL>\", \"\\n\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6176ef59-f62f-4908-93cc-043f6cd1b714",
   "metadata": {},
   "source": [
    "## Benchmark\n",
    "\n",
    "1.36 s ± 320 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87230e15-a782-40e0-90a2-0a15d827f72a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%timeit response = predictor_client.predict(data=data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90d5879e-62aa-4a0b-8e1a-57c83a90feea",
   "metadata": {},
   "source": [
    "## JAQUET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f635b12a-a67b-4a1d-b2e5-ff60d49fa4da",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!wget -P data https://jaqket.s3.ap-northeast-1.amazonaws.com/data/aio_02/aio_02_dev_v1.0.jsonl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ffd3d2c-f523-407b-b4cf-46c2e4539997",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from tqdm import tqdm\n",
    "import pandas as pd\n",
    "import re\n",
    "df = pd.read_json('./data/aio_02_dev_v1.0.jsonl', orient='records', lines=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "403e1718-8b26-4011-a16b-e80dd930a8d8",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "def inference(instruction, input):\n",
    "    data = {\n",
    "        \"instruction\": instruction,\n",
    "        \"input\": input,\n",
    "        \"max_new_tokens\": 8,\n",
    "        \"sampling_temperature\": 0,\n",
    "        \"repetition_penalty\": 1.05,\n",
    "        \"stop_ids\": [1, 0],\n",
    "    }\n",
    "    response = predictor_client.predict(\n",
    "        data=data\n",
    "    )\n",
    "    return response\n",
    "\n",
    "\n",
    "# Zero Shot\n",
    "correct = 0\n",
    "for idx, row in df.iterrows():\n",
    "    prompt = \"日本語のクイズに答えてください。\" + row['question'] + \"答えは「\"\n",
    "    # print(prompt)\n",
    "    result = inference(\"\", prompt)\n",
    "    # print(result)\n",
    "    result = prompt + result\n",
    "    try:\n",
    "        result = re.findall(\"「(.*?)」\", result)[-1]\n",
    "    except IndexError:\n",
    "        result = result\n",
    "        # print(\"longer output:\", result)\n",
    "    result = re.sub(r'[(].*[)]', \"\", result)\n",
    "    if result in row['answers']:\n",
    "        correct += 1\n",
    "    else:\n",
    "        print(result, row['answers'])\n",
    "print(correct, \"/\", len(df))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82532107-6b06-4d78-8009-7e0e7c8b8dc7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "prompts = [\"日本語のクイズに答えてください。\" + question + \"答えは「\" for question in df['question']]\n",
    "\n",
    "def inference(prompts):\n",
    "    data = {\n",
    "        \"instruction\": \"\",\n",
    "        \"input\": prompts,\n",
    "        \"max_new_tokens\": 8,\n",
    "        \"sampling_temperature\": 0,\n",
    "        \"repetition_penalty\": 1.05,\n",
    "        \"stop_ids\": [1, 0],\n",
    "    }\n",
    "    response = predictor_client.predict(\n",
    "        data=data\n",
    "    )\n",
    "    return response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "900103e6-7608-4025-a37f-d96f02403f57",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "# Batch Inference\n",
    "\n",
    "# Zero Shot\n",
    "correct = 0\n",
    "batch_size = 250\n",
    "for idx in range(0, len(prompts), batch_size):\n",
    "    results = inference(prompts[idx:idx+batch_size])\n",
    "    # print(result)\n",
    "    for j in range(len(results)):\n",
    "        result = prompts[idx + j] + results[j]\n",
    "        try:\n",
    "            result = re.findall(\"「(.*?)」\", result)[-1]\n",
    "        except IndexError:\n",
    "            result = result\n",
    "            # print(\"longer output:\", result)\n",
    "        result = re.sub(r'[(].*[)]', \"\", result)\n",
    "        if result in df['answers'][idx + j]:\n",
    "            correct += 1\n",
    "        else:\n",
    "            print(result, df['answers'][idx + j])\n",
    "print(correct, \"/\", len(df))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcc6a60f-772b-4b47-9c43-2f5870299078",
   "metadata": {},
   "source": [
    "## Delete Endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e63bdfc-664f-4cc8-9410-079386a2b8b8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictor_client.delete_model()\n",
    "predictor_client.delete_endpoint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b64add43-d565-4eb1-9ec3-bbb1b7c56ec8",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (PyTorch 2.0.0 Python 3.10 CPU Optimized)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/pytorch-2.0.0-cpu-py310"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}