{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "d0916a3a-e402-48b7-a775-ce739e4aeaf4", "metadata": {}, "source": [ "# Bedrock boto3 Setup" ] }, { "attachments": {}, "cell_type": "markdown", "id": "bbab02f1-3eac-4274-b06b-d51ce586df2c", "metadata": { "tags": [] }, "source": [ "--- \n", "\n", "In this demo notebook, we demonstrate how to use the `boto3` Python SDK to work with [Bedrock](https://aws.amazon.com/bedrock/) Foundational Models.\n", "\n", "---" ] }, { "attachments": {}, "cell_type": "markdown", "id": "4c1fda97-9150-484a-8cfa-86ec9568fc61", "metadata": {}, "source": [ "## Prerequisites" ] }, { "attachments": {}, "cell_type": "markdown", "id": "27a83a8d-9527-48b4-92ff-fce963fbe3b5", "metadata": {}, "source": [ "---\n", "Before executing any of the notebook in this workshop, execute the following cells to add Bedrock extensions to the `boto3` Python SDK\n", "\n", "---" ] }, { "cell_type": "code", "execution_count": null, "id": "108c611c-7246-45c4-9f1e-76888b5076eb", "metadata": { "tags": [] }, "outputs": [], "source": [ "# Make sure you run `download-dependencies.sh` from the root of the repository to download the dependencies before running this cell\n", "%pip install ../dependencies/botocore-1.29.162-py3-none-any.whl ../dependencies/boto3-1.26.162-py3-none-any.whl ../dependencies/awscli-1.27.162-py3-none-any.whl --force-reinstall\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "3f1c8940", "metadata": {}, "source": [ "You also need to install [langchain](https://github.com/hwchase17/langchain)" ] }, { "cell_type": "code", "execution_count": null, "id": "e692c0d3", "metadata": { "tags": [] }, "outputs": [], "source": [ "%pip install langchain==0.0.190 --quiet" ] }, { "attachments": {}, "cell_type": "markdown", "id": "be703e81", "metadata": {}, "source": [ "## Create the boto3 client\n", "\n", "Interaction with the Bedrock API is done via boto3 SDK. To create a the Bedrock client, we are providing an utility method that supports different options for passing credentials to boto3. \n", "If you are running these notebooks from your own computer, make sure you have [installed the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) before proceeding.\n", "\n", "\n", "#### Use default credential chain\n", "\n", "If you are running this notebook from a Sagemaker Studio notebook and your Sagemaker Studio role has permissions to access Bedrock you can just run the cells below as-is. This is also the case if you are running these notebooks from a computer whose default credentials have access to Bedrock\n", "\n", "#### Use a different role\n", "\n", "In case you or your company has setup a specific role to access Bedrock, you can specify such role by uncommenting the line `#os.environ['BEDROCK_ASSUME_ROLE'] = ''` in the cell below before executing it. Ensure that your current user or role have permissions to assume such role.\n", "\n", "#### Use a specific profile\n", "\n", "In case you are running this notebooks from your own computer and you have setup the AWS CLI with multiple profiles and the profile which has access to Bedrock is not the default one, you can uncomment the line `#os.environ['AWS_PROFILE'] = ''` and specify the profile to use.\n", "\n", "#### Note about `langchain`\n", "\n", "The Bedrock classes provided by `langchain` create a default Bedrock boto3 client. We recommend to explicitly create the Bedrock client using the instructions below, and pass it to the class instantiation methods using `client=boto3_bedrock`" ] }, { "cell_type": "code", "execution_count": null, "id": "b031c34a", "metadata": {}, "outputs": [], "source": [ "#### Un comment the following lines to run from your local environment outside of the AWS account with Bedrock access\n", "\n", "\n", "#import os\n", "#os.environ['BEDROCK_ASSUME_ROLE'] = ''\n", "#os.environ['AWS_PROFILE'] = ''" ] }, { "cell_type": "code", "execution_count": null, "id": "89bfd1e4", "metadata": {}, "outputs": [], "source": [ "import os\n", "import sys\n", "import json\n", "\n", "module_path = \"..\"\n", "sys.path.append(os.path.abspath(module_path))\n", "from utils import bedrock, print_ww\n", "\n", "os.environ['AWS_DEFAULT_REGION'] = 'us-east-1'\n", "if('BEDROCK_ASSUME_ROLE' in os.environ):\n", " boto3_bedrock = bedrock.get_bedrock_client(os.environ.get('BEDROCK_ASSUME_ROLE', None))\n", "else:\n", " boto3_bedrock = bedrock.get_bedrock_client()\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "9e9174c4-326a-463e-92e1-8c7e47111269", "metadata": {}, "source": [ "#### We can validate our connection by testing out the `list_foundation_models()` method, which will tell us all the models available for us to use " ] }, { "cell_type": "code", "execution_count": null, "id": "f67b4466-12ff-4975-9811-7a19c6206604", "metadata": { "tags": [] }, "outputs": [], "source": [ "boto3_bedrock.list_foundation_models()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "be9044d4-9d04-47c3-86ca-5b206585b784", "metadata": {}, "source": [ "#### In this Notebook we will be using the `invoke_model()` method of Amazon Bedrock. This will be the primary method we use for most of our Text Generation and Processing tasks. " ] }, { "attachments": {}, "cell_type": "markdown", "id": "881928fb-4daf-47e5-a2b6-b2292a679a81", "metadata": {}, "source": [ "# `InvokeModel` body and output\n", "\n", "#### We provide the details about the format for the input and output format of `invoke_model()` for the different foundation models\n", "\n", "## Titan Large\n", "\n", "#### Input\n", "```json\n", "{ \n", " \"inputText\": \"\",\n", " \"textGenerationConfig\" : { \n", " \"maxTokenCount\": 512,\n", " \"stopSequences\": [],\n", " \"temperature\":0.1, \n", " \"topP\":0.9\n", " }\n", "}\n", "```\n", "\n", "#### Output\n", "\n", "```json\n", "{\n", " \"inputTextTokenCount\": 613,\n", " \"results\": [{\n", " \"tokenCount\": 219,\n", " \"outputText\": \"\"\n", " }]\n", "}\n", "```\n", "\n", "## Jurassic Grande and Jumbo \n", "\n", "#### Input\n", "\n", "```json\n", "{\n", " \"prompt\": \"\",\n", " \"maxTokens\": 200,\n", " \"temperature\": 0.5,\n", " \"topP\": 0.5,\n", " \"stopSequences\": [],\n", " \"countPenalty\": {\n", " \"scale\": 0\n", " },\n", " \"presencePenalty\": {\n", " \"scale\": 0\n", " },\n", " \"frequencyPenalty\": {\n", " \"scale\": 0\n", " }\n", "}\n", "```\n", "\n", "#### Output\n", "\n", "```json\n", "{\n", " \"id\": 1234,\n", " \"prompt\": {\n", " \"text\": \"\",\n", " \"tokens\": [\n", " {\n", " \"generatedToken\": {\n", " \"token\": \"\\u2581who\\u2581is\",\n", " \"logprob\": -12.980147361755371,\n", " \"raw_logprob\": -12.980147361755371\n", " },\n", " \"topTokens\": null,\n", " \"textRange\": {\n", " \"start\": 0,\n", " \"end\": 6\n", " }\n", " },\n", " ...\n", " ]\n", " },\n", " \"completions\": [\n", " {\n", " \"data\": {\n", " \"text\": \"\",\n", " \"tokens\": [\n", " {\n", " \"generatedToken\": {\n", " \"token\": \"<|newline|>\",\n", " \"logprob\": 0.0,\n", " \"raw_logprob\": -0.01293118204921484\n", " },\n", " \"topTokens\": null,\n", " \"textRange\": {\n", " \"start\": 0,\n", " \"end\": 1\n", " }\n", " },\n", " ...\n", " ]\n", " },\n", " \"finishReason\": {\n", " \"reason\": \"endoftext\"\n", " }\n", " }\n", " ]\n", "}\n", "```\n", "\n", "## Claude\n", "\n", "#### Input\n", "\n", "```json\n", "{\n", " \"prompt\": \"\\n\\nHuman:\\n\\nAnswer:\",\n", " \"max_tokens_to_sample\": 300,\n", " \"temperature\": 0.5,\n", " \"top_k\": 250,\n", " \"top_p\": 1,\n", " \"stop_sequences\": [\n", " \"\\n\\nHuman:\"\n", " ]\n", "}\n", "```\n", "\n", "#### Output\n", "\n", "```json\n", "{\n", " \"completion\": \" \",\n", " \"stop_reason\": \"stop_sequence\"\n", "}\n", "```\n", "\n", "## Stable Diffusion XL\n", "\n", "### Input\n", "\n", "```json\n", "{\n", " \"text_prompts\": [\n", " { \n", " \"text\": \"this is where you place your input text\" \n", " }\n", " ],\n", " \"cfg_scale\":10,\n", " \"seed\":0,\n", " \"steps\":50\n", "}\n", "```\n", "\n", "### Output\n", "\n", "```json\n", "{ \n", " \"result\": \"success\", \n", " \"artifacts\": [\n", " {\n", " \"seed\": 123, \n", " \"base64\": \"\",\n", " \"finishReason\": \"SUCCESS\"\n", " }\n", "}\n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "id": "80f4adca-cfc4-439b-84b7-e528398684e3", "metadata": {}, "source": [ "# Common inference parameter definitions\n", "\n", "## Randomness and Diversity\n", "\n", "Foundation models support the following parameters to control randomness and diversity in the \n", "response.\n", "\n", "**Temperature** – Large language models use probability to construct the words in a sequence. For any \n", "given next word, there is a probability distribution of options for the next word in the sequence. When \n", "you set the temperature closer to zero, the model tends to select the higher-probability words. When \n", "you set the temperature further away from zero, the model may select a lower-probability word.\n", "\n", "In technical terms, the temperature modulates the probability density function for the next tokens, \n", "implementing the temperature sampling technique. This parameter can deepen or flatten the density \n", "function curve. A lower value results in a steeper curve with more deterministic responses, and a higher \n", "value results in a flatter curve with more random responses.\n", "\n", "**Top K** – Temperature defines the probability distribution of potential words, and Top K defines the cut \n", "off where the model no longer selects the words. For example, if K=50, the model selects from 50 of the \n", "most probable words that could be next in a given sequence. This reduces the probability that an unusual \n", "word gets selected next in a sequence.\n", "In technical terms, Top K is the number of the highest-probability vocabulary tokens to keep for Top-\n", "K-filtering - This limits the distribution of probable tokens, so the model chooses one of the highest-\n", "probability tokens.\n", "\n", "**Top P** – Top P defines a cut off based on the sum of probabilities of the potential choices. If you set Top \n", "P below 1.0, the model considers the most probable options and ignores less probable ones. Top P is \n", "similar to Top K, but instead of capping the number of choices, it caps choices based on the sum of their \n", "probabilities.\n", "For the example prompt \"I hear the hoof beats of ,\" you may want the model to provide \"horses,\" \n", "\"zebras\" or \"unicorns\" as the next word. If you set the temperature to its maximum, without capping \n", "Top K or Top P, you increase the probability of getting unusual results such as \"unicorns.\" If you set the \n", "temperature to 0, you increase the probability of \"horses.\" If you set a high temperature and set Top K or \n", "Top P to the maximum, you increase the probability of \"horses\" or \"zebras,\" and decrease the probability \n", "of \"unicorns.\"\n", "\n", "## Length\n", "\n", "The following parameters control the length of the generated response.\n", "\n", "**Response length** – Configures the minimum and maximum number of tokens to use in the generated \n", "response.\n", "\n", "**Length penalty** – Length penalty optimizes the model to be more concise in its output by penalizing \n", "longer responses. Length penalty differs from response length as the response length is a hard cut off for \n", "the minimum or maximum response length.\n", "\n", "In technical terms, the length penalty penalizes the model exponentially for lengthy responses. 0.0 \n", "means no penalty. Set a value less than 0.0 for the model to generate longer sequences, or set a value \n", "greater than 0.0 for the model to produce shorter sequences.\n", "\n", "## Repetitions\n", "\n", "The following parameters help control repetition in the generated response.\n", "\n", "**Repetition penalty (presence penalty)** – Prevents repetitions of the same words (tokens) in responses. \n", "1.0 means no penalty. Greater than 1.0 decreases repetition." ] }, { "cell_type": "code", "execution_count": null, "id": "722bf913-3957-457f-804a-89900dd85c79", "metadata": { "tags": [] }, "outputs": [], "source": [ "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\\nBlog:\"\"\"" ] }, { "attachments": {}, "cell_type": "markdown", "id": "ce22c308-ebbf-4ef5-a823-832b7c236e31", "metadata": {}, "source": [ "## 2. Accessing Bedrock Foundation Models" ] }, { "attachments": {}, "cell_type": "markdown", "id": "893872fe-04fa-4f09-9736-6c6173ec1fc2", "metadata": { "tags": [] }, "source": [ "### Let's try the prompt with the Titan Model on Bedrock" ] }, { "cell_type": "code", "execution_count": null, "id": "7df55eed-a3cf-426c-95ea-ec60dade6477", "metadata": { "tags": [] }, "outputs": [], "source": [ "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\\nBlog:\"\"\" # If you'd like to try your own prompt, edit this parameter!" ] }, { "cell_type": "code", "execution_count": null, "id": "dd2bb671-6b10-4948-9e5e-95d6ced3b86f", "metadata": { "tags": [] }, "outputs": [], "source": [ "body = json.dumps({\"inputText\": prompt_data})\n", "modelId = \"amazon.titan-tg1-large\" \n", "accept = \"application/json\"\n", "contentType = \"application/json\"\n", "\n", "response = boto3_bedrock.invoke_model(\n", " body=body, modelId=modelId, accept=accept, contentType=contentType\n", ")\n", "response_body = json.loads(response.get(\"body\").read())\n", "\n", "print(response_body.get(\"results\")[0].get(\"outputText\"))" ] }, { "attachments": {}, "cell_type": "markdown", "id": "3d7c0fe6-576a-4380-89aa-726bab5d65ff", "metadata": {}, "source": [ "### Let's try the prompt with the Anthropic Claude Instant Model on Bedrock" ] }, { "cell_type": "code", "execution_count": null, "id": "0ba33ac0-fa16-4c4f-b882-e838d0cb5830", "metadata": { "tags": [] }, "outputs": [], "source": [ "body = json.dumps({\"prompt\": prompt_data, \"max_tokens_to_sample\": 500})\n", "modelId = \"anthropic.claude-instant-v1\" # change this to use a different version from the model provider\n", "accept = \"application/json\"\n", "contentType = \"application/json\"\n", "\n", "response = boto3_bedrock.invoke_model(\n", " body=body, modelId=modelId, accept=accept, contentType=contentType\n", ")\n", "response_body = json.loads(response.get(\"body\").read())\n", "\n", "print(response_body.get(\"completion\"))" ] }, { "attachments": {}, "cell_type": "markdown", "id": "ed0e3144-c6df-400d-aab1-1540614dbbde", "metadata": {}, "source": [ "### Let's try the prompt with the Jurassic Grande Model on Bedrock" ] }, { "cell_type": "code", "execution_count": null, "id": "c02d1585-945e-45d1-99d2-171e956138f8", "metadata": { "tags": [] }, "outputs": [], "source": [ "body = json.dumps({\"prompt\": prompt_data, \"maxTokens\": 200})\n", "modelId = \"ai21.j2-grande-instruct\" # change this to use a different version from the model provider\n", "accept = \"application/json\"\n", "contentType = \"application/json\"\n", "\n", "response = boto3_bedrock.invoke_model(\n", " body=body, modelId=modelId, accept=accept, contentType=contentType\n", ")\n", "response_body = json.loads(response.get(\"body\").read())\n", "\n", "print(response_body.get(\"completions\")[0].get(\"data\").get(\"text\"))" ] }, { "attachments": {}, "cell_type": "markdown", "id": "4b3619b5", "metadata": {}, "source": [ "### Let's try the streaming output from Bedrock" ] }, { "cell_type": "code", "execution_count": null, "id": "c69627e3", "metadata": {}, "outputs": [], "source": [ "from IPython.display import display, display_markdown, Markdown, clear_output\n", "\n", "body = json.dumps({\"prompt\": prompt_data, \"max_tokens_to_sample\": 200})\n", "modelId = \"anthropic.claude-instant-v1\" # change this to use a different version from the model provider\n", "accept = \"application/json\"\n", "contentType = \"application/json\"\n", "\n", "response = boto3_bedrock.invoke_model_with_response_stream(body=body, modelId=modelId, accept=accept, contentType=contentType)\n", "stream = response.get('body')\n", "output = []\n", "\n", "if stream:\n", " for event in stream:\n", " chunk = event.get('chunk')\n", " if chunk:\n", " chunk_obj = json.loads(chunk.get('bytes').decode())\n", " text = chunk_obj['completion']\n", " clear_output(wait=True)\n", " output.append(text)\n", " display_markdown(Markdown(''.join(output)))\n", " " ] }, { "attachments": {}, "cell_type": "markdown", "id": "bc498bea", "metadata": {}, "source": [ "### Let's try the prompt with the Stable Diffusion XL on Bedrock" ] }, { "cell_type": "code", "execution_count": null, "id": "173e51a2", "metadata": {}, "outputs": [], "source": [ "prompt_data = \"a fine image of an astronaut riding a horse on Mars\"\n", "body = json.dumps({\n", " \"text_prompts\": [\n", " { \n", " \"text\": prompt_data \n", " }\n", " ],\n", " \"cfg_scale\":10,\n", " \"seed\":20,\n", " \"steps\":50\n", "})\n", "modelId = \"stability.stable-diffusion-xl\" \n", "accept = \"application/json\"\n", "contentType = \"application/json\"\n", "\n", "response = boto3_bedrock.invoke_model(\n", " body=body, modelId=modelId, accept=accept, contentType=contentType\n", ")\n", "response_body = json.loads(response.get(\"body\").read())\n", "\n", "print(response_body['result'])\n", "print(f'{response_body.get(\"artifacts\")[0].get(\"base64\")[0:80]}...')" ] }, { "attachments": {}, "cell_type": "markdown", "id": "2a00bb66", "metadata": {}, "source": [ "The output is a base64 encoded string of the image. You can use ans image processing library such as Pillow to decode the image as in the example below:\n", "\n", "```python\n", "base_64_img_str = response_body.get(\"artifacts\")[0].get(\"base64\")\n", "image = Image.open(io.BytesIO(base64.decodebytes(bytes(base_64_img_str, \"utf-8\"))))\n", "```" ] }, { "attachments": {}, "cell_type": "markdown", "id": "1ef3451d-b66a-4b11-a1ed-734bf9e7bbec", "metadata": {}, "source": [ "# Embeddings\n", "\n", "Use text embeddings to convert text into meaningful vector representations. You input a body of text \n", "and the output is a (1 x n) vector. You can use embedding vectors for a wide variety of applications. \n", "Bedrock currently offers one model for text embedding that supports text similarity (finding the \n", "semantic similarity between bodies of text) and text retrieval (such as search).\n", "For the text embeddings model, the input text size is 512 tokens and the output vector length is 4096.\n", "To use a text embeddings model, use the InvokeModel API operation or the Python SDK.\n", "Use InvokeModel to retrieve the vector representation of the input text from the specified model.\n", "\n", "At the time of writing you can only use `amazon.titan-e1t-medium` as embedding model via the API.\n", "\n", "#### Input\n", "\n", "```json\n", "{\n", " \"inputText\": \"\"\n", "}\n", "```\n", "\n", "#### Output\n", "\n", "```json\n", "{\n", " \"embedding\": []\n", "}\n", "```\n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "9645dbd8", "metadata": {}, "source": [ "Let's see how to generate embeddings of some text:" ] }, { "cell_type": "code", "execution_count": null, "id": "1085cc56", "metadata": {}, "outputs": [], "source": [ "prompt_data = \"Amazon Bedrock supports foundation models from industry-leading providers such as \\\n", "AI21 Labs, Anthropic, Stability AI, and Amazon. Choose the model that is best suited to achieving your unique goals.\"" ] }, { "cell_type": "code", "execution_count": null, "id": "5c54b424", "metadata": {}, "outputs": [], "source": [ "body = json.dumps({\"inputText\": prompt_data})\n", "modelId = \"amazon.titan-e1t-medium\" # change this to use a different version from the model provider\n", "accept = \"application/json\"\n", "contentType = \"application/json\"\n", "\n", "response = boto3_bedrock.invoke_model(\n", " body=body, modelId=modelId, accept=accept, contentType=contentType\n", ")\n", "response_body = json.loads(response.get(\"body\").read())\n", "\n", "embedding = response_body.get(\"embedding\")\n", "print(f\"The embedding vector has {len(embedding)} values\\n{embedding[0:3]+['...']+embedding[-3:]}\")" ] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 }, { "_defaultOrder": 55, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 56, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4de.24xlarge", "vcpuNum": 96 } ], "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 5 }