{ "cells": [ { "cell_type": "markdown", "id": "bb62997e-7212-47c7-8e5a-de72b0cb8903", "metadata": {}, "source": [ "# Korean Embedding 모델을 SageMaker 배포 및 추론\n", "- 이 노트북은 SageMaker Notebook Instance 의 conday_pytorch_p39 에서 테스트 되었습니다. " ] }, { "cell_type": "markdown", "id": "3f7f589c-04d8-4d46-894d-9b12e84f4f8d", "metadata": {}, "source": [ "Model Ref:\n", "- BM-K/KoSimCSE-roberta\n", " - https://huggingface.co/BM-K/KoSimCSE-roberta\n", "Inference Code Ref: \n", "- Huggingface Sagemaker-sdk - Deploy 🤗 Transformers for inference\n", " - https://github.com/huggingface/notebooks/blob/main/sagemaker/11_deploy_model_from_hf_hub/deploy_transformer_model_from_hf_hub.ipynb\n", "- Sentence Embeddings with Hugging Face Transformers, Sentence Transformers and Amazon SageMaker - Custom Inference for creating document embeddings with Hugging Face's Transformers\n", " - https://github.com/huggingface/notebooks/blob/main/sagemaker/17_custom_inference_script/sagemaker-notebook.ipynb\n", " " ] }, { "cell_type": "markdown", "id": "904a2890-5d85-4014-b15f-7432a61573f5", "metadata": {}, "source": [ "# 0. 기본 환경 설정" ] }, { "cell_type": "code", "execution_count": 2, "id": "8e58cd73-1557-4944-8769-2bc1d08f946e", "metadata": { "tags": [] }, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2\n", "\n", "# src 폴더 경로 설정\n", "import sys\n", "sys.path.append('../common_code')" ] }, { "cell_type": "markdown", "id": "3c215704-222f-40a3-b018-d4b2b5fa16e1", "metadata": {}, "source": [ "# 1. HF Hub로 부터 모델 및 토큰 나이저 로딩" ] }, { "cell_type": "code", "execution_count": 3, "id": "8fcd49e4-d2be-4cd0-9a00-95bcfcdbb002", "metadata": { "tags": [] }, "outputs": [], "source": [ "import torch\n", "from transformers import AutoModel, AutoTokenizer\n", "\n", "\n", "model = AutoModel.from_pretrained('BM-K/KoSimCSE-roberta')\n", "tokenizer = AutoTokenizer.from_pretrained('BM-K/KoSimCSE-roberta')\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "35340a2c-ad28-4b1a-b0fd-48c81733810b", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "sample : \n", " 이번 주 일요일에 분당 이마트 점은 문을 여나요\n", "embeding size: 768\n", "embeding content from 0 to 10 out of 768: \n", " tensor([-0.2569, -0.1982, 0.8970, -1.7043, -0.1197, 0.2873, 0.3933, -0.4806,\n", " -0.1716, -0.6642], grad_fn=)\n" ] } ], "source": [ "sample = \"이번 주 일요일에 분당 이마트 점은 문을 여나요\"\n", "\n", "inputs = tokenizer(sample, padding=True, truncation=True, return_tensors=\"pt\")\n", "embeddings, _ = model(**inputs, return_dict=False)\n", "\n", "emb_len = len(embeddings[0][0])\n", "print(\"sample : \\n\", sample)\n", "print(\"embeding size: \", emb_len)\n", "print(f\"embeding content from 0 to 10 out of {emb_len}: \\n\", embeddings[0][0][0:10])" ] }, { "cell_type": "markdown", "id": "d2fedd92-1a77-41a7-a3a3-90b59aba23d5", "metadata": {}, "source": [ "## 추론 테스트 및 문장 유사도 측정\n", "- 아래 첫문장, 두번째 문장의 유사도를 구함\n", "- 아래 첫문장, 세째 문장의 유사도를 구함\n", "- 최종적으로 유사도 수치를 비교 함" ] }, { "cell_type": "code", "execution_count": 5, "id": "06a3c433-9d37-4680-9e91-7eaa39ed005b", "metadata": { "tags": [] }, "outputs": [], "source": [ "def show_embedding_score(tokenizer, model, sentences):\n", " inputs = tokenizer(sentences, padding=True, truncation=True, return_tensors=\"pt\")\n", " embeddings, _ = model(**inputs, return_dict=False)\n", "\n", " score01 = cal_score(embeddings[0][0], embeddings[1][0])\n", " score02 = cal_score(embeddings[0][0], embeddings[2][0])\n", "\n", " print(score01, score02)\n", "\n", "def cal_score(a, b):\n", " '''\n", " 코사인 유사도 구하는 함수\n", " '''\n", " if len(a.shape) == 1: a = a.unsqueeze(0)\n", " if len(b.shape) == 1: b = b.unsqueeze(0)\n", "\n", " a_norm = a / a.norm(dim=1)[:, None]\n", " b_norm = b / b.norm(dim=1)[:, None]\n", " return torch.mm(a_norm, b_norm.transpose(0, 1)) * 100\n", " " ] }, { "cell_type": "code", "execution_count": 6, "id": "03f7244f-9a05-411b-be3f-2486bb518c1b", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([[92.7287]], grad_fn=) tensor([[79.8030]], grad_fn=)\n" ] } ], "source": [ "sentences1 = ['이번 주 일요일에 분당 이마트 점은 문을 여나요',\n", " '일요일에 분당 이마트는 문 열어요?',\n", " '분당 이마트 점은 토요일에 몇 시까지 하나요']\n", "\n", "show_embedding_score(tokenizer, model, sentences1) " ] }, { "cell_type": "markdown", "id": "1525156e-b71e-4cc4-bdee-bf4ac4cacfc0", "metadata": {}, "source": [ "# 2. 세이지 메이커로 모델 배포" ] }, { "cell_type": "code", "execution_count": 7, "id": "56cf132c-def0-46d0-9581-4822c6067f14", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "sagemaker role arn: arn:aws:iam::057716757052:role/gen_ai_gsmoon\n" ] } ], "source": [ "import sagemaker\n", "import boto3\n", "\n", "try:\n", " role = sagemaker.get_execution_role()\n", "except ValueError:\n", " iam = boto3.client('iam')\n", " role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']\n", "\n", "print(f\"sagemaker role arn: {role}\")" ] }, { "cell_type": "markdown", "id": "bafe6665-0e66-4089-9900-bf7787c2d74f", "metadata": {}, "source": [ "## HF Model ID, HF_TASK 정의" ] }, { "cell_type": "code", "execution_count": 8, "id": "6fbd1714-0d82-496e-b267-9afb2b05f923", "metadata": { "tags": [] }, "outputs": [], "source": [ "from sagemaker.huggingface import HuggingFaceModel\n", "\n", "# Hub Model configuration. https://huggingface.co/models\n", "hub = {\n", " 'HF_MODEL_ID':'BM-K/KoSimCSE-roberta', # model_id from hf.co/models\n", " 'HF_TASK':'feature-extraction',\n", " 'SAGEMAKER_MODEL_SERVER_TIMEOUT':'3600', \n", " 'TS_MAX_RESPONSE_SIZE':'2000000000',\n", " 'TS_MAX_REQUEST_SIZE':'2000000000',\n", " 'MMS_MAX_RESPONSE_SIZE':'2000000000',\n", " 'MMS_MAX_REQUEST_SIZE':'2000000000'\n", "} \n", "\n", "# create Hugging Face Model Class\n", "huggingface_model = HuggingFaceModel(\n", " env=hub,\n", " role=role, # iam role with permissions to create an Endpoint\n", " transformers_version=\"4.26\", # transformers version used\n", " pytorch_version=\"1.13\", # pytorch version used\n", " py_version=\"py39\", # python version of the DLC\n", ")" ] }, { "cell_type": "markdown", "id": "2cd8c52d-081c-46db-849f-b1f199078ba7", "metadata": {}, "source": [ "## 모델 배포" ] }, { "cell_type": "code", "execution_count": 9, "id": "1bf7c67f-3c56-4872-b74e-c52d20d76c7a", "metadata": { "tags": [] }, "outputs": [], "source": [ "from datetime import datetime\n", "\n", "time_stamp = datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\")\n", "\n", "endpoint_name = f\"KoSimCSE-roberta-\" + time_stamp" ] }, { "cell_type": "code", "execution_count": 10, "id": "006baf1e-f3d4-4f14-b396-358cf051e1bc", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "----!CPU times: user 157 ms, sys: 5.86 ms, total: 162 ms\n", "Wall time: 2min 32s\n" ] } ], "source": [ "%%time\n", "\n", "# deploy model to SageMaker Inference\n", "predictor = huggingface_model.deploy(\n", " initial_instance_count=1,\n", " endpoint_name = endpoint_name,\n", " instance_type=\"ml.m5.xlarge\"\n", ")" ] }, { "cell_type": "markdown", "id": "372071e1-7e2e-45d3-9404-7c2064917af5", "metadata": {}, "source": [ "# 3. 추론" ] }, { "cell_type": "code", "execution_count": 11, "id": "7679e49b-d177-4836-9657-98a86ae6d727", "metadata": { "tags": [] }, "outputs": [], "source": [ "sentences2 = ['분당 이마트점에 KT 대리점이 있나요?',\n", " '거기 이마트점에 KT 대리점이 있나요?',\n", " '분당 아미트 점은 지하 주차장이 있나요?']\n" ] }, { "cell_type": "code", "execution_count": 12, "id": "d82b36b6-6d12-424c-9b42-4fce8852bfcf", "metadata": { "tags": [] }, "outputs": [], "source": [ "import numpy as np\n", "\n", "payload_1 = {\n", " \"inputs\" : sentences1\n", "}\n", "\n", "payload_2 = {\n", " \"inputs\" : sentences2\n", "}\n", "\n", "def predict_payload(data):\n", " res = predictor.predict(data=data)\n", " res = np.array(res) # .squeeze().squeeze()\n", " # print(\"res: \", res.shape)\n", " # print(\"embedding dimension: \", len(res[0][0][0]))\n", " return res\n", "\n" ] }, { "cell_type": "markdown", "id": "56e44b83-6048-49c7-9e72-e6b206b06b9d", "metadata": {}, "source": [ "## Sample Test (한개의 문장 임베딩 보여 주기)" ] }, { "cell_type": "code", "execution_count": 13, "id": "a17a1fab-2e3f-4023-a334-54eaa260fe32", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "payload_0 : \n", " {'inputs': '이번 주 일요일에 분당 이마트 점은 문을 여나요'}\n", "embeding size: 768\n", "embeding content from 0 to 10 out of 768: [-0.2569178 -0.19823857 0.89699048 -1.70428038 -0.11973442 0.28725004\n", " 0.39328021 -0.48056296 -0.17160198 -0.66424179]\n" ] } ], "source": [ "payload_0 = {\n", " \"inputs\" : \"이번 주 일요일에 분당 이마트 점은 문을 여나요\"\n", "}\n", "\n", "\n", "response = predict_payload(payload_0)\n", "emb_len = len(response[0][0])\n", "print(\"payload_0 : \\n\", payload_0)\n", "print(\"embeding size: \", emb_len)\n", "print(f\"embeding content from 0 to 10 out of {emb_len}: \", response[0][0][0:10])" ] }, { "cell_type": "code", "execution_count": 14, "id": "1de687a7-1bdb-465a-aac2-c706572fd66c", "metadata": { "tags": [] }, "outputs": [], "source": [ "def show_embedding_score2(payload):\n", " '''\n", " # res \n", " # 1st dim: samples, 2nd dim: place_hoder, 3rd_dim : CLS, ohter tokens \n", " # res.shape --> (3,1)\n", " # len(res[1][0]) --> 11 두번째 샘플의 11개 토큰\n", " # len(res[1][0][0]) --> 두번째 샘플의 , 첫번째 토큰 임베딩 (764 사이즈)\n", " '''\n", " res = predict_payload(payload) \n", " embeddings_0 = torch.Tensor(res[0][0][0]) \n", " embeddings_1 = torch.Tensor(res[1][0][0])\n", " embeddings_2 = torch.Tensor(res[2][0][0])\n", "\n", " score01 = cal_score(embeddings_0, embeddings_1)\n", " score02 = cal_score(embeddings_0, embeddings_2) \n", " print(score01, score02)\n", " \n" ] }, { "cell_type": "code", "execution_count": 15, "id": "43a2b9d8-9b4e-42f4-9874-05981cfb9044", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "payload_1: \n", " {'inputs': ['이번 주 일요일에 분당 이마트 점은 문을 여나요', '일요일에 분당 이마트는 문 열어요?', '분당 이마트 점은 토요일에 몇 시까지 하나요']}\n", "tensor([[92.7287]]) tensor([[79.8030]])\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ ":13: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n", " res = np.array(res) # .squeeze().squeeze()\n" ] } ], "source": [ "print(\"payload_1: \\n\", payload_1)\n", "show_embedding_score2(payload_1)" ] }, { "cell_type": "code", "execution_count": 16, "id": "053cfc3b-b896-4006-b01e-a45046bff45a", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "payload_2: \n", " {'inputs': ['분당 이마트점에 KT 대리점이 있나요?', '거기 이마트점에 KT 대리점이 있나요?', '분당 아미트 점은 지하 주차장이 있나요?']}\n", "tensor([[89.2611]]) tensor([[53.1729]])\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ ":13: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n", " res = np.array(res) # .squeeze().squeeze()\n" ] } ], "source": [ "print(\"payload_2: \\n\", payload_2)\n", "show_embedding_score2(payload_2)" ] }, { "cell_type": "markdown", "id": "42d35390-34cf-4868-9742-1b031c43dfea", "metadata": {}, "source": [ "# 4. Boto3 invoke_endpoint() 사용하여 추론" ] }, { "cell_type": "code", "execution_count": 17, "id": "1a8dd8ad-aa9e-4e6d-885c-a682a503344e", "metadata": { "tags": [] }, "outputs": [], "source": [ "endpoint_name = predictor.endpoint_name" ] }, { "cell_type": "code", "execution_count": 18, "id": "fc845e26-dfb0-4af5-85fc-ae5d2fda85d0", "metadata": {}, "outputs": [], "source": [ "import boto3\n", "import json\n", "\n", "def query_endpoint_embedding_with_json_payload(encoded_json, endpoint_name, content_type=\"application/json\"):\n", " client = boto3.client(\"runtime.sagemaker\")\n", " response = client.invoke_endpoint(\n", " EndpointName=endpoint_name, ContentType=content_type, Body=encoded_json\n", " )\n", " return response\n", "\n", "def transform_output(output: bytes) -> str:\n", " response_json = json.loads(output.read().decode(\"utf-8\"))\n", " # return response_json\n", " return response_json[0][0]\n" ] }, { "cell_type": "code", "execution_count": 19, "id": "a5fb4731-9420-44b5-bf27-0a57a62ac9e5", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "첫문장 임베딩 사이즈: 768\n", "두번째 문장 임베딩 사이즈: 768\n" ] } ], "source": [ "sentences2_1 = \"분당 이마트점에 KT 대리점이 있나요?\"\n", "sentences2_2 = \"거기 이마트점에 KT 대리점이 있나요?\"\n", "\n", "payload_2_1 = {\n", " \"inputs\" : sentences2_1\n", "}\n", "\n", "payload_2_2 = {\n", " \"inputs\" : sentences2_2\n", "}\n", "\n", "# 첫번째 문장\n", "query_response = query_endpoint_embedding_with_json_payload(\n", " json.dumps(payload_2_1).encode(\"utf-8\"), endpoint_name=endpoint_name\n", ")\n", "\n", "emb_1 = transform_output(query_response['Body'])\n", "print(\"첫문장 임베딩 사이즈: \", len(emb_1))\n", "\n", "# 두번째 문장\n", "query_response = query_endpoint_embedding_with_json_payload(\n", " json.dumps(payload_2_2).encode(\"utf-8\"), endpoint_name=endpoint_name\n", ")\n", " \n", "emb_2 = transform_output(query_response['Body'])\n", "print(\"두번째 문장 임베딩 사이즈: \", len(emb_2))" ] }, { "cell_type": "code", "execution_count": 20, "id": "45e6874b-fd56-4a67-9ee0-8adc403b079a", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([[89.2611]])\n" ] } ], "source": [ "def show_embedding_score3(emb1, emb2):\n", "\n", " embeddings_0 = torch.Tensor(emb1) \n", " embeddings_1 = torch.Tensor(emb2)\n", "\n", " score01 = cal_score(embeddings_0, embeddings_1)\n", "\n", " print(score01)\n", "\n", "show_embedding_score3(emb_1, emb_2) " ] }, { "cell_type": "markdown", "id": "727b5bee-47d7-4c91-8fc3-05fd13fb077b", "metadata": {}, "source": [ "# 5. 엔드포인트 삭제" ] }, { "cell_type": "code", "execution_count": 21, "id": "fa92abce-3249-477d-b982-fc11f48c7883", "metadata": { "tags": [] }, "outputs": [], "source": [ "# # delete endpoint\n", "# predictor.delete_model()\n", "# predictor.delete_endpoint()" ] }, { "cell_type": "code", "execution_count": 22, "id": "9ec91af7-a2e8-4697-9dd8-d93c57db8b63", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Stored 'embedding_model_endpoint_name' (str)\n" ] } ], "source": [ "embedding_model_endpoint_name = endpoint_name\n", "%store embedding_model_endpoint_name" ] }, { "cell_type": "code", "execution_count": 23, "id": "9411ea52-6795-42e1-a4b1-97f7c5ccab2c", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "'KoSimCSE-roberta-2023-06-04-11-24-26'" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "embedding_model_endpoint_name" ] }, { "cell_type": "code", "execution_count": null, "id": "d4e4c252-3b38-4461-a150-896e96e62764", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 } ], "instance_type": "ml.m5.2xlarge", "kernelspec": { "display_name": "Python 3 (PyTorch 1.13 Python 3.9 CPU Optimized)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/pytorch-1.13-cpu-py39" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 5 }