{
"cells": [
{
"cell_type": "markdown",
"id": "9fb44f6d",
"metadata": {},
"source": [
"## Text Generation using Different Prompting Techniques with Amazon SageMaker JumpStart SDK and Falcon 40B Instruct Language Model\n",
"\n",
"---\n",
"This Amazon SageMaker Studio Notebook demonstrates how to use the SageMaker Python SDK to firstly deploy Falcon-40B-Instruct Large Language Model and then generate text using advanced prompting engineering techniques. \n",
"\n",
"This notebook has the following prerequisites:\n",
"- Select an AWS region where [Amazon SageMaker JumpStart](https://aws.amazon.com/sagemaker/jumpstart) is available. \n",
"- [Setup Amazon SageMaker Domain](https://docs.aws.amazon.com/sagemaker/latest/dg/onboard-quick-start.html).\n",
"- [Available service queta](https://docs.aws.amazon.com/general/latest/gr/sagemaker.html) for \"ml.g5.12xlarge for endpoint usage\".\n",
"- Less than $10 per hour to spend on Amazon SageMaker JumpStart model deployments and usage of other AWS services.\n",
"\n",
"This notebook is based on the following references:\n",
"- [Amazon SageMaker JumpStart SDK](https://sagemaker.readthedocs.io/en/v2.82.0/overview.html#use-prebuilt-models-with-sagemaker-jumpstart), providing pretrained models for a wide range of problem types to help you get started with machine learning.\n",
"- [Falcon-40B](https://huggingface.co/tiiuae/falcon-40b), an open source language model with 40B parameters decoder-only model built by TII.\n",
"- [Prompt Engineering](https://www.promptingguide.ai/), an online project aiming to educate Gen AI practitioners about prompt engineering. \n",
"---"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9b05b931-992e-4526-978d-f03196874a3b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# install pythn libraries\n",
"!pip install --upgrade pip --quiet\n",
"!pip install --upgrade sagemaker --quiet"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d65a3c82-d17c-4855-97e1-c49ae0d61eef",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# important required libraries\n",
"from sagemaker.jumpstart.model import JumpStartModel"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "08508ffa-90f4-4c8c-9c69-b8002bb37f12",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Define SageMaker JumpStart Model using model id, instance type, and endpoint timeout\n",
"my_model = JumpStartModel(model_id=\"huggingface-llm-falcon-40b-bf16\",\n",
" instance_type=\"ml.g5.12xlarge\",\n",
" env={'ENDPOINT_SERVER_TIMEOUT':'300'})"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "85a2a8e5-789f-4041-9927-221257126653",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"%%time\n",
"\n",
"# Host the model on the instance and deploy an inference endpoint\n",
"# Because the model size is >80GB, expecy deploy() to take 15 min!\n",
"predictor = my_model.deploy()"
]
},
{
"cell_type": "markdown",
"id": "471b13e2-9d0b-4158-8841-a6c07a7d3b45",
"metadata": {
"tags": []
},
"source": [
"---\n",
"**Prompting Techniques**\n",
"\n",
"Large language model (LLM) prompting is a technique for using a LLM to perform a task that it has not been explicitly trained on. This is done by providing the LLM with a prompt, which is a short piece of text that instructs the model on what to do. Prompting is a powerful technique that can be used to do a variety of tasks, including: text summarization, question answering, and code generation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "77efa7b9-4e50-400b-b7c7-8a8ea07d82a1",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"stop_keywords = [\"<|endoftext|>\", \"</s>\"]\n",
"max_new_tokens = 100\n",
"do_sample = False\n",
"temperature = 1"
]
},
{
"cell_type": "markdown",
"id": "bd22952a-4992-41ab-b4d2-2cfb15559652",
"metadata": {
"tags": []
},
"source": [
"**Technique 1: Zero-Shot Prompting**\n",
"\n",
"Zero-shot prompting allows a model to perform a task without any additional training data by providing the LLM with a prompt that describes the task, and the model then uses its knowledge of the world to generate the desired output."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c1a55aa5-f2ad-4db8-9718-b76f969cffbe",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Zero-Shot Prompting Example 1\n",
"prompt = \"Classify the following text into neutral, negative or positive.\\n\" \\\n",
" \"Text: I think the vacation was not bad.\\n\" \\\n",
" \"Sentiment:\"\n",
"\n",
"payload = {\n",
" \"inputs\": prompt,\n",
" \"parameters\": {\n",
" \"stop\": stop_keywords,\n",
" \"temperature\": temperature,\n",
" \"max_new_tokens\": max_new_tokens,\n",
" \"do_sample\": do_sample\n",
" }\n",
"}\n",
"response = predictor.predict(payload)\n",
"\n",
"print(\"Zero-Shot Prompting Example 1:\")\n",
"print(prompt)\n",
"print(\"\\033[95m\")\n",
"print(response[0][\"generated_text\"].strip())"
]
},
{
"cell_type": "markdown",
"id": "5e153983-9964-4e8f-b148-5a2e968d61cc",
"metadata": {},
"source": [
"**Technique 2: Few-Shot Prompting**\n",
"\n",
"Few-shot prompting allows a model to process examples before attempting a task, steering the model to better performance."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "008e8836-7a49-407f-bf04-8dcf59de8dbf",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"## Zero-Shot Prompting Example 1\n",
"prompt = \"\"\"A \"whatpu\" is a small, furry animal native to Tanzania. \"\"\" \\\n",
" \"\"\"An example of a sentence that uses the word whatpu is: \"\"\" \\\n",
" \"\"\"We were traveling in Africa and we saw these very cute whatpus.\\n\"\"\" \\\n",
" \"\"\"To do a \"farduddle\" means to jump up and down really fast. \"\"\" \\\n",
" \"\"\"An example of a sentence that uses the word farduddle is:\"\"\"\n",
"\n",
"payload = {\n",
" \"inputs\": prompt,\n",
" \"parameters\": {\n",
" \"stop\": stop_keywords,\n",
" \"temperature\": temperature,\n",
" \"max_new_tokens\": max_new_tokens,\n",
" \"do_sample\": do_sample\n",
" }\n",
"}\n",
"response = predictor.predict(payload)\n",
"\n",
"print(\"Few-Shot Prompting Example 1:\")\n",
"print(prompt)\n",
"print(\"\\033[95m\")\n",
"print(response[0][\"generated_text\"].strip())"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "04b532e1-0fe6-4b67-8d6a-63174610f354",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"## Few-Shot Prompting Example 2\n",
"prompt = \"This is awesome! // Negative\\n\" \\\n",
" \"This is bad! // Positive\\n\" \\\n",
" \"Wow that movie was rad! // Positive\\n\" \\\n",
" \"What a horrible show! //\"\n",
"\n",
"payload = {\n",
" \"inputs\": prompt,\n",
" \"parameters\": {\n",
" \"stop\": stop_keywords,\n",
" \"temperature\": temperature,\n",
" \"max_new_tokens\": max_new_tokens,\n",
" \"do_sample\": do_sample\n",
" }\n",
"}\n",
"response = predictor.predict(payload)\n",
"\n",
"print(\"Few-Shot Prompting Example 2:\")\n",
"print(prompt)\n",
"print(\"\\033[95m\")\n",
"print(response[0][\"generated_text\"].strip())"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e977436a-f5d7-4f42-ac36-66ea00a4f66f",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"## Few-Shot Prompting Example Example 3\n",
"prompt = \"Add 2+3: 5\\n\" \\\n",
" \"Add 3+6: 9\\n\" \\\n",
" \"Add 7+1:\"\n",
"\n",
"payload = {\n",
" \"inputs\": prompt,\n",
" \"parameters\": {\n",
" \"stop\": stop_keywords,\n",
" \"temperature\": temperature,\n",
" \"max_new_tokens\": max_new_tokens,\n",
" \"do_sample\": do_sample\n",
" }\n",
"}\n",
"response = predictor.predict(payload)\n",
"\n",
"print(\"Few-Shot Prompting Example 3:\")\n",
"print(prompt)\n",
"print(\"\\033[95m\")\n",
"print(response[0][\"generated_text\"].strip())"
]
},
{
"cell_type": "markdown",
"id": "ccf74caa-c563-4b1e-9585-a28237dc539f",
"metadata": {},
"source": [
"**Technique 3: Chain-of-Thought Prompting**\n",
"\n",
"Chain-of-Thought prompting encourages the model to explain its reasoning. This method enables complex reasoning capabilities through intermediate reasoning steps."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b253a50c-a024-4608-9502-00591764eb5c",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"## Chain-of-Thought Prompting Example 1\n",
"prompt = \"I bought 10 apples from the market. \" \\\n",
" \"Then, I gave 2 apples to the neighbor and 2 to the repairman. \" \\\n",
" \"Then, I then went and bought 5 more apples but I ate 1. \" \\\n",
" \"How many apples did I remain with? Let us think step by step.\"\n",
"\n",
"payload = {\n",
" \"inputs\": prompt,\n",
" \"parameters\": {\n",
" \"stop\": stop_keywords,\n",
" \"temperature\": temperature,\n",
" \"max_new_tokens\": max_new_tokens,\n",
" \"do_sample\": do_sample\n",
" }\n",
"}\n",
"response = predictor.predict(payload)\n",
"\n",
"print(\"Chain-of-Thought Prompting Example 1:\")\n",
"print(prompt)\n",
"print(\"\\033[95m\")\n",
"print(response[0][\"generated_text\"].strip())"
]
},
{
"cell_type": "markdown",
"id": "850ad903-7381-466a-b0e1-33cd61532786",
"metadata": {},
"source": [
"**Technique 4: Self-Consistency**\n",
"\n",
"This technique samples multiple generations through multiple few-shot chain-of-thought prompts, and then selects the most consistent answer. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e829e375-9ac3-46ba-a4fa-8753196f2a2f",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"## Self-Consistency Prompting Example 1\n",
"prompt = \"When I was 6, my sister was half my age. \" \\\n",
" \"Now I’m 58. How old is my sister? \"\n",
"\n",
"payload = {\n",
" \"inputs\": prompt,\n",
" \"parameters\": {\n",
" \"stop\": stop_keywords,\n",
" \"temperature\": temperature,\n",
" \"max_new_tokens\": max_new_tokens,\n",
" \"do_sample\": do_sample\n",
" }\n",
"}\n",
"response1 = predictor.predict(payload)\n",
"response2 = predictor.predict(payload)\n",
"response3 = predictor.predict(payload)\n",
"\n",
"print(\"Self-Consistency Prompting Example 1:\")\n",
"print(prompt)\n",
"print(\"\\033[95m\")\n",
"print(response1[0][\"generated_text\"].strip())\n",
"print(response2[0][\"generated_text\"].strip())\n",
"print(response3[0][\"generated_text\"].strip())"
]
},
{
"cell_type": "markdown",
"id": "b04d99e2",
"metadata": {},
"source": [
"### SageMaker Clean up "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6c3b60c2",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Delete the SageMaker endpoint\n",
"predictor.delete_endpoint()"
]
}
],
"metadata": {
"availableInstances": [
{
"_defaultOrder": 0,
"_isFastLaunch": true,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 4,
"name": "ml.t3.medium",
"vcpuNum": 2
},
{
"_defaultOrder": 1,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.t3.large",
"vcpuNum": 2
},
{
"_defaultOrder": 2,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.t3.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 3,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.t3.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 4,
"_isFastLaunch": true,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.m5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 5,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.m5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 6,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.m5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 7,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.m5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 8,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.m5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 9,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.m5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 10,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.m5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 11,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.m5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 12,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.m5d.large",
"vcpuNum": 2
},
{
"_defaultOrder": 13,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.m5d.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 14,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.m5d.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 15,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.m5d.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 16,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.m5d.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 17,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.m5d.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 18,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.m5d.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 19,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.m5d.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 20,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": true,
"memoryGiB": 0,
"name": "ml.geospatial.interactive",
"supportedImageNames": [
"sagemaker-geospatial-v1-0"
],
"vcpuNum": 0
},
{
"_defaultOrder": 21,
"_isFastLaunch": true,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 4,
"name": "ml.c5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 22,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.c5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 23,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.c5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 24,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.c5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 25,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 72,
"name": "ml.c5.9xlarge",
"vcpuNum": 36
},
{
"_defaultOrder": 26,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 96,
"name": "ml.c5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 27,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 144,
"name": "ml.c5.18xlarge",
"vcpuNum": 72
},
{
"_defaultOrder": 28,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.c5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 29,
"_isFastLaunch": true,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.g4dn.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 30,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.g4dn.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 31,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.g4dn.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 32,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.g4dn.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 33,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.g4dn.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 34,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.g4dn.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 35,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 61,
"name": "ml.p3.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 36,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 244,
"name": "ml.p3.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 37,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 488,
"name": "ml.p3.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 38,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.p3dn.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 39,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.r5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 40,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.r5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 41,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.r5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 42,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.r5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 43,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.r5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 44,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.r5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 45,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 512,
"name": "ml.r5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 46,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.r5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 47,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.g5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 48,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.g5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 49,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.g5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 50,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.g5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 51,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.g5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 52,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.g5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 53,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.g5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 54,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.g5.48xlarge",
"vcpuNum": 192
},
{
"_defaultOrder": 55,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 1152,
"name": "ml.p4d.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 56,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 1152,
"name": "ml.p4de.24xlarge",
"vcpuNum": 96
}
],
"instance_type": "ml.t3.medium",
"kernelspec": {
"display_name": "Python 3 (Data Science 2.0)",
"language": "python",
"name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-38"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.13"
}
},
"nbformat": 4,
"nbformat_minor": 5
}