{ "cells": [ { "cell_type": "markdown", "id": "9b262721-9ba7-40c2-acc4-96c41cf9230a", "metadata": {}, "source": [ "# OpenCALM SageMaker Finetuning by JAQKET dataset\n", "\n", "[OpenCALM](https://huggingface.co/spaces/kyo-takano/OpenCALM-7B) を SageMaker の Training Job を使用し Fine Tuning し、学習済みモデルを推論エンドポイントに Hosting する Notebook です。[JAQKET](https://www.nlp.ecei.tohoku.ac.jp/projects/jaqket/) データセットから、 Fine Tuning 用に学習データ、 推論用に評価データセットを使用しています。\n", "\n", "以下の環境で Training Job / Hosting を行い動作確認を行ってます。\n", "\n", "* `ml.g5.2xlarge(NVIDIA A10G Tensor Core GPU 搭載 VRAM 24GB, RAM 32GB, vCPU 8)` : `PyTorch 1.13 Python 3.9 GPU Optimized`\n", " \n", "[各インスタンスの料金についてはこちら](https://aws.amazon.com/jp/sagemaker/pricing/)をご確認ください。" ] }, { "cell_type": "code", "execution_count": null, "id": "32904a97-bedb-451a-b2cc-ac691f7708de", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "!pip install -U \"sagemaker>=2.143.0\"" ] }, { "cell_type": "code", "execution_count": null, "id": "34747940-eb9c-4428-9646-fa0fc9f169cf", "metadata": {}, "outputs": [], "source": [ "!pip install tqdm" ] }, { "cell_type": "code", "execution_count": null, "id": "2edc8a17-9a6b-4dd6-823e-878f6b950ea2", "metadata": { "tags": [] }, "outputs": [], "source": [ "import sagemaker, boto3, json\n", "from sagemaker import get_execution_role\n", "from sagemaker.pytorch.model import PyTorchModel\n", "from sagemaker.huggingface import HuggingFace\n", "\n", "role = get_execution_role()\n", "region = boto3.Session().region_name\n", "sess = sagemaker.Session()\n", "bucket = sess.default_bucket()\n", "\n", "sagemaker.__version__" ] }, { "cell_type": "markdown", "id": "bc1b5fe7-9ed9-4d24-b344-a40a781a24a1", "metadata": { "tags": [] }, "source": [ "## Upload Data\n", "\n", "Fine Tuning 用の日本語データをフォルダに配置してアップロードする。" ] }, { "cell_type": "markdown", "id": "2ae9977e-6be2-44a2-b1be-3b5d8e66df92", "metadata": {}, "source": [ "### Prepare JAQKET dataset" ] }, { "cell_type": "code", "execution_count": null, "id": "951e92f0-acce-4c68-bc43-4fc5dafdae93", "metadata": { "tags": [] }, "outputs": [], "source": [ "!wget -P data https://jaqket.s3.ap-northeast-1.amazonaws.com/data/aio_02/aio_02_train.jsonl" ] }, { "cell_type": "code", "execution_count": null, "id": "fd856b83-5a09-4db4-b33d-790f1b9ecffe", "metadata": { "tags": [] }, "outputs": [], "source": [ "!head -n 2 data/aio_02_train.jsonl" ] }, { "cell_type": "code", "execution_count": null, "id": "65efe8ce-09ed-4cc8-94e3-4198c49762ec", "metadata": {}, "outputs": [], "source": [ "# Convet .jsonl to .json\n", "import pandas as pd\n", "\n", "df = pd.read_json(\"data/aio_02_train.jsonl\", orient=\"records\", lines=True)\n", "df = df.rename(columns={\"question\": \"instruction\", \"answers\": \"output\"})\n", "df = df[[\"instruction\", \"output\"]]\n", "df[\"output\"] = df[\"output\"].apply(lambda x: f\"{x[0]}」\")\n", "df[\"input\"] = \"\"\n", "print(df.shape)\n", "df.to_json(\n", " \"data/aio_02_train_formatted.jsonl\", orient=\"records\", force_ascii=False, lines=True\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "afbbbac7-e783-4a22-a220-3a6e5c268340", "metadata": { "tags": [] }, "outputs": [], "source": [ "df.head(2)" ] }, { "cell_type": "code", "execution_count": null, "id": "dfa10b80-b583-499b-ae1f-d03cd7dd1435", "metadata": { "tags": [] }, "outputs": [], "source": [ "input_train = sess.upload_data(\n", " path=\"./data/aio_02_train_formatted.jsonl\", key_prefix=\"OpenCALM\"\n", ")\n", "input_train" ] }, { "cell_type": "markdown", "id": "07b5098b-1ea9-4b85-88ee-9297d5e00c09", "metadata": {}, "source": [ "## Fine-tuning" ] }, { "cell_type": "code", "execution_count": null, "id": "0f58c68a-704d-42fa-b2a4-5befb1377fe9", "metadata": { "tags": [] }, "outputs": [], "source": [ "model_name = \"cyberagent/open-calm-7b\"\n", "model_name_base = model_name.split(\"/\")[-1]" ] }, { "cell_type": "code", "execution_count": null, "id": "1cb98557-3a3f-40ef-baf3-538c4ee4aa8a", "metadata": { "tags": [] }, "outputs": [], "source": [ "hyperparameters = {\n", " \"base_model\": model_name,\n", " # 'load_in_8bit': True,\n", " # 'load_in_4bit': True,\n", " \"pad_token_id\": 1,\n", " \"data_path\": \"/opt/ml/input/data/train/aio_02_train_formatted.jsonl\",\n", " \"num_epochs\": 2, # default 3\n", " \"cutoff_len\": 256,\n", " \"group_by_length\": False,\n", " \"output_dir\": \"/opt/ml/model\",\n", " # 'resume_from_checkpoint': '/opt/ml/checkpoints',\n", " \"lora_target_modules\": \"[query_key_value]\",\n", " \"lora_r\": 16,\n", " \"batch_size\": 32,\n", " \"micro_batch_size\": 4,\n", " # 'val_set_size': 200,\n", " \"prompt_template_name\": \"simple_qa_ja\",\n", "}" ] }, { "cell_type": "code", "execution_count": null, "id": "e4a8d836-d44f-4bdc-8d87-cff90f9f6ea7", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "huggingface_estimator = HuggingFace(\n", " base_job_name=model_name_base,\n", " role=role,\n", " entry_point=\"finetune.py\",\n", " source_dir=\"./scripts/code\",\n", " instance_type=\"ml.g5.2xlarge\",\n", " instance_count=1,\n", " volume_size=200,\n", " transformers_version=\"4.26\",\n", " pytorch_version=\"1.13\",\n", " py_version=\"py39\",\n", " use_spot_instances=True,\n", " max_wait=86400,\n", " hyperparameters=hyperparameters,\n", " metric_definitions=[\n", " {\"Name\": \"eval_loss\", \"Regex\": \"'eval_loss': (\\d\\.\\d+)\"},\n", " {\"Name\": \"train_loss\", \"Regex\": \"'loss': (\\d\\.\\d+)\"},\n", " ],\n", " # checkpoint_s3_uri=f\"s3://{bucket}/{base_job_name}/checkpoint/\",\n", ")\n", "huggingface_estimator.fit({\"train\": input_train})" ] }, { "cell_type": "markdown", "id": "e8df141b-86f8-4eff-8ba5-129b78d3efd4", "metadata": { "tags": [] }, "source": [ "## Download and Extract Model" ] }, { "cell_type": "code", "execution_count": null, "id": "a02eb6aa-b4c3-4d60-a3fc-94f394df7ac1", "metadata": { "tags": [] }, "outputs": [], "source": [ "import boto3\n", "import sagemaker\n", "\n", "\n", "def get_latest_training_job_artifact(base_job_name):\n", " sagemaker_client = boto3.client(\"sagemaker\")\n", " response = sagemaker_client.list_training_jobs(\n", " NameContains=base_job_name, SortBy=\"CreationTime\", SortOrder=\"Descending\"\n", " )\n", " training_job_arn = response[\"TrainingJobSummaries\"][0][\"TrainingJobArn\"]\n", " training_job_description = sagemaker_client.describe_training_job(\n", " TrainingJobName=training_job_arn.split(\"/\")[-1]\n", " )\n", " return training_job_description[\"ModelArtifacts\"][\"S3ModelArtifacts\"]\n", "\n", "\n", "try:\n", " model_data = huggingface_estimator.model_data\n", "except:\n", " # Retrieve artifact url when kernel is restarted\n", " model_data = get_latest_training_job_artifact(model_name_base)\n", "\n", "print(model_data)" ] }, { "cell_type": "code", "execution_count": null, "id": "21529d11-eb97-4e50-bf07-980a2e26ae24", "metadata": { "tags": [] }, "outputs": [], "source": [ "!aws s3 cp {model_data} {model_name_base}.tar.gz" ] }, { "cell_type": "code", "execution_count": null, "id": "8e23bbc7-6038-4dfd-a632-21b38be59e78", "metadata": { "tags": [] }, "outputs": [], "source": [ "!rm -rf scripts/{model_name_base} && mkdir scripts/{model_name_base}\n", "!tar -xvf {model_name_base}.tar.gz -C scripts/{model_name_base} --no-same-owner --wildcards adapter_*\n", "!ls -l scripts/{model_name_base}" ] }, { "cell_type": "markdown", "id": "44bf1cde-57d4-43f6-be6b-b09bd942219f", "metadata": {}, "source": [ "## Package and Upload Model" ] }, { "cell_type": "code", "execution_count": null, "id": "916cee87-d530-4e67-8fa6-18550ff92532", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "%cd scripts\n", "!tar -czvf ../{model_name_base}.tar.gz *\n", "%cd -" ] }, { "cell_type": "code", "execution_count": null, "id": "e252d1ba-c402-4352-a0f5-c59dc51d6fc9", "metadata": { "tags": [] }, "outputs": [], "source": [ "model_path = sess.upload_data(\n", " f\"{model_name_base}.tar.gz\", bucket=bucket, key_prefix=f\"OpenCALM\"\n", ")\n", "model_path" ] }, { "cell_type": "markdown", "id": "cee0a96e-b185-4e2f-a8dd-75cbb086abfa", "metadata": {}, "source": [ "## Deploy Model" ] }, { "cell_type": "code", "execution_count": null, "id": "e8232665-f10e-4a74-84d6-d5c87640d1f2", "metadata": { "tags": [] }, "outputs": [], "source": [ "from sagemaker.async_inference import AsyncInferenceConfig\n", "from sagemaker.serializers import JSONSerializer\n", "\n", "\n", "huggingface_model = PyTorchModel(\n", " model_data=model_path,\n", " framework_version=\"1.13\",\n", " py_version=\"py39\",\n", " role=role,\n", " name=model_name_base,\n", " env={\n", " \"model_params\": json.dumps(\n", " {\n", " \"base_model\": model_name,\n", " \"lora_weights\": model_name_base, # path relative to model package\n", " \"peft\": True,\n", " \"load_8bit\": False,\n", " \"prompt_template\": \"simple_qa_ja\",\n", " }\n", " ),\n", " \"SAGEMAKER_MODEL_SERVER_TIMEOUT\": \"3600\",\n", " },\n", ")\n", "\n", "# deploy model to SageMaker Inference\n", "predictor = huggingface_model.deploy(\n", " initial_instance_count=1,\n", " instance_type=\"ml.g5.2xlarge\",\n", " endpoint_name=model_name_base,\n", " serializer=JSONSerializer(),\n", " # async_inference_config=AsyncInferenceConfig()\n", ")" ] }, { "cell_type": "markdown", "id": "fb4f6d9f-666d-40d6-b2c3-93ec553dded1", "metadata": {}, "source": [ "## Run Inference" ] }, { "cell_type": "code", "execution_count": null, "id": "be982e42-166c-47fb-bc35-cc20ee6a5c4e", "metadata": { "tags": [] }, "outputs": [], "source": [ "# With SageMaker SDK\n", "\n", "from sagemaker.predictor import Predictor\n", "from sagemaker.predictor_async import AsyncPredictor\n", "from sagemaker.serializers import JSONSerializer\n", "from sagemaker.deserializers import JSONDeserializer\n", "\n", "predictor_client = Predictor(\n", " endpoint_name=model_name_base,\n", " sagemaker_session=sess,\n", " serializer=JSONSerializer(),\n", " deserializer=JSONDeserializer(),\n", ")\n", "# predictor_client = AsyncPredictor(\n", "# predictor=predictor_client,\n", "# name=endpoint_name\n", "# )\n", "data = {\n", " \"instruction\": \"映画『ウエスト・サイド物語』に登場する2つの少年グループといえば、シャーク団と何団?\",\n", " \"max_new_tokens\": 64,\n", " \"temperature\": 0.1,\n", " \"do_sample\": True,\n", " \"pad_token_id\": 1,\n", " \"bos_token_id\": 0,\n", " \"eos_token_is\": 0,\n", " # \"repetition_penalty\": 1.05,\n", " # \"top_p\": 0.75,\n", " # \"top_k\": 40,\n", " # \"no_repeat_ngram_size\": 2,\n", " \"stop_ids\": [1, 0],\n", "}\n", "response = predictor_client.predict(data=data)\n", "print(response)" ] }, { "cell_type": "markdown", "id": "4290a244-a43a-41c0-ab72-6a778da58ce0", "metadata": {}, "source": [ "### Inference for dev data" ] }, { "cell_type": "code", "execution_count": null, "id": "c3f78389-f7aa-4655-820c-05bdda306710", "metadata": { "tags": [] }, "outputs": [], "source": [ "!wget -P data https://jaqket.s3.ap-northeast-1.amazonaws.com/data/aio_02/aio_02_dev_v1.0.jsonl" ] }, { "cell_type": "code", "execution_count": null, "id": "a40e3b01-726d-4137-b78f-e5de5cb5f313", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "import re\n", "\n", "\n", "def inference(instruction):\n", " data = {\n", " \"instruction\": instruction,\n", " \"input\": \"\",\n", " \"max_new_tokens\": 64,\n", " \"temperature\": 0.1,\n", " \"do_sample\": False,\n", " \"num_beams\": 5,\n", " \"pad_token_id\": 1,\n", " \"bos_token_id\": 0,\n", " \"eos_token_is\": 0,\n", " # \"repetition_penalty\": 1.05,\n", " \"stop_ids\": [1, 0],\n", " }\n", " response = predictor_client.predict(data=data)\n", " answer = \"\"\n", " try:\n", " answer = re.findall(\"「(.*?)」\", f\"「{response}\")[-1]\n", " except IndexError:\n", " answer = response\n", " return answer" ] }, { "cell_type": "code", "execution_count": null, "id": "3a19f8c0-1877-423b-b79f-e38181c97451", "metadata": { "tags": [] }, "outputs": [], "source": [ "import pandas as pd\n", "from tqdm import tqdm\n", "\n", "\n", "df = pd.read_json(\"data/aio_02_dev_v1.0.jsonl\", orient=\"records\", lines=True)\n", "\n", "llm_answers = []\n", "matches = []\n", "for idx, row in tqdm(df.iterrows()):\n", " llm_answer = inference(row[\"question\"])\n", " llm_answers += [llm_answer]\n", " matches += [llm_answer in row[\"answers\"]]\n", "\n", "\n", "df[\"llm_answers\"] = pd.Series(llm_answers)\n", "df[\"match\"] = pd.Series(matches)" ] }, { "cell_type": "code", "execution_count": null, "id": "6b2ebd61-9365-4780-9b82-81764003e714", "metadata": {}, "outputs": [], "source": [ "print(df.match.sum(), \"/\", len(df))" ] }, { "cell_type": "code", "execution_count": null, "id": "21fc487b-fd12-4730-8242-144026c31ada", "metadata": { "tags": [] }, "outputs": [], "source": [ "df.to_csv(f\"data/{model_name_base}.csv\", index=False)" ] }, { "cell_type": "markdown", "id": "ae16c211-5f6a-4dc4-bbc3-b8f9b07666a0", "metadata": {}, "source": [ "## Benchmark Speed" ] }, { "cell_type": "code", "execution_count": null, "id": "d1c9ddd3-76ee-4f71-8888-4ae51069a967", "metadata": { "tags": [] }, "outputs": [], "source": [ "%timeit response = predictor_client.predict(data=data)" ] }, { "cell_type": "markdown", "id": "39737593-7053-4dbb-b39b-112e27d228a1", "metadata": {}, "source": [ "## Delete Endpoint" ] }, { "cell_type": "code", "execution_count": null, "id": "82559f16-10f8-4573-91d7-e0b85cb7d0de", "metadata": { "tags": [] }, "outputs": [], "source": [ "predictor.delete_model()\n", "predictor.delete_endpoint()" ] }, { "cell_type": "code", "execution_count": null, "id": "c75d2062-a88e-4c2f-97e9-8778ee231d29", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 }, { "_defaultOrder": 55, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 56, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4de.24xlarge", "vcpuNum": 96 } ], "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Base Python 3.0)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-base-python-310-v1" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.8" } }, "nbformat": 4, "nbformat_minor": 5 }