{ "cells": [ { "cell_type": "markdown", "id": "a0fcbeec-8bef-486f-aaa4-32f5c8756156", "metadata": { "tags": [] }, "source": [ "# Bias detection for LLMs\n", "In this notebook we'll explore a variety of techniques to identify biased behavior in large language models, with a focus on Titan Text, available via Amazon Bedrock.\n", "\n", "We'll use a dataset from Amazon, [BOLD](https://github.com/amazon-science/bold), and a framework from Hugging Face, [Evaluate](https://github.com/huggingface/evaluate)." ] }, { "cell_type": "code", "execution_count": null, "id": "5aebccab-3a0d-4cc8-a68d-2f983f43a21b", "metadata": { "tags": [] }, "outputs": [], "source": [ "!pip uninstall -y sagemaker --quiet\n", "!pip install sagemaker --quiet" ] }, { "cell_type": "markdown", "id": "cdfa9c44-2212-4f2f-9a4d-a7fa2fd3f23c", "metadata": {}, "source": [ "### Step 1. Deploy the model through Jumpstart\n", "\n", "\n", "In this demo notebook, we demonstrate how to use the SageMaker Python SDK to deploy Falcon models for text generation. It is a permissively licensed (Apache-2.0) open source model trained on the RefinedWeb dataset. We show several example use cases including code generation, question answering, translation etc." ] }, { "cell_type": "code", "execution_count": null, "id": "acc3e871-2abb-4006-9f1d-436ec8b148bd", "metadata": { "tags": [] }, "outputs": [], "source": [ "import boto3\n", "import json\n", "\n", "model_id, model_version, = (\n", " \"huggingface-textgeneration-falcon-7b-instruct-bf16\",\n", " \"*\",\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "4a939e62-1f86-4371-81c5-8c35caf3e9b6", "metadata": { "tags": [] }, "outputs": [], "source": [ "\n", "\n", "%%time\n", "from sagemaker.jumpstart.model import JumpStartModel\n", "from sagemaker.serializers import JSONSerializer\n", "\n", "\n", "my_model = JumpStartModel(model_id=model_id)\n", "predictor = my_model.deploy()\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "51ed1c78-bf31-496d-a6a7-8ed421f83ebb", "metadata": { "tags": [] }, "outputs": [], "source": [ "%%time\n", "\n", "predictor.serializer = JSONSerializer()\n", "predictor.content_type = \"application/json\"\n", "\n", "\n", "payload = {\n", " \"text_inputs\": \"Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\\nDaniel: Hello, Girafatron!\\nGirafatron:\",\n", " \"max_new_tokens\": 50,\n", " \"return_full_text\": False,\n", " \"do_sample\": True,\n", " \"top_k\": 10,\n", "}\n", "\n", "response = predictor.predict(payload)\n", "print(response[\"generated_texts\"][0])" ] }, { "cell_type": "code", "execution_count": null, "id": "4df4c93f-407d-415e-9601-e7e274f208df", "metadata": { "tags": [] }, "outputs": [], "source": [ "def test_llm(prompt_data):\n", "\n", " payload = {\"text_inputs\": prompt_data, \n", " \"max_new_tokens\": 50,\n", " \"return_full_text\": False,\n", " \"do_sample\": True,\n", " \"top_k\": 10,}\n", " \n", "\n", " response_body = predictor.predict(payload)\n", " \n", " \n", " rt = response_body[\"generated_texts\"][0]\n", " \n", " return rt\n" ] }, { "cell_type": "code", "execution_count": null, "id": "90470919-cf15-4e70-a515-c33a2f6509a2", "metadata": { "tags": [] }, "outputs": [], "source": [ "prompt_data =\" My name is Bharathi and I work very hard. Quesion: Should we hire this person? Answer: \" #If you'd like to try your own prompt, edit this parameter!#If you'd like to try your own prompt, edit this parameter!\n", "\n", "test_llm(prompt_data)" ] }, { "cell_type": "markdown", "id": "b7629c6c-72bb-4a1a-ac7c-ee4a55dc937d", "metadata": {}, "source": [ "### Step 2. Download the Amazon BOLD dataset\n", "Next, we'll download Amazon's [BOLD: Bias in Open-Ended Language Generation Dataset](https://github.com/amazon-science/bold/tree/main). This has over 20,000 prompts designed to evaluate fairness in the response of the model." ] }, { "cell_type": "code", "execution_count": null, "id": "dd290dd6-18d9-48e2-b4d8-ac24a0ec3492", "metadata": { "tags": [] }, "outputs": [], "source": [ "!git clone https://github.com/amazon-science/bold.git " ] }, { "cell_type": "code", "execution_count": null, "id": "ca80d1d8-aa18-42cb-8fa9-3f1381acea30", "metadata": { "tags": [] }, "outputs": [], "source": [ "import json" ] }, { "cell_type": "code", "execution_count": null, "id": "22372e9a-1f32-4141-8a32-152a14522f64", "metadata": { "tags": [] }, "outputs": [], "source": [ "f_name = 'bold/prompts/gender_prompt.json'\n", "f = open(f_name)\n", "gender_prompts = json.load(f)" ] }, { "cell_type": "code", "execution_count": null, "id": "b86d0835-ced0-468a-b177-d594074a197f", "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "sample_prompts = list(gender_prompts['American_actresses'].items())[:10]" ] }, { "cell_type": "markdown", "id": "b9d0e12e-dffb-47e3-83cb-4f0bbfec6033", "metadata": {}, "source": [ "### Step 3. Invoke the model with the prompts, and capture the responses" ] }, { "cell_type": "code", "execution_count": null, "id": "152e77b7-f2f1-4c5a-b6a4-6d062148a485", "metadata": { "tags": [] }, "outputs": [], "source": [ "import time\n", "\n", "responses = {}\n", "\n", "for subject, p in sample_prompts:\n", " \n", " prompt = p[0] \n", " \n", " output = test_llm(prompt)\n", " \n", " responses[subject] = {prompt:output}\n", " \n", " # try not to hit the throttle\n", " time.sleep(10)\n", " " ] }, { "cell_type": "code", "execution_count": null, "id": "2d1a503c-c695-460f-9f06-e0ae40225ecb", "metadata": { "tags": [] }, "outputs": [], "source": [ "json_object = json.dumps(responses, indent=4)\n", " \n", "# Writing to sample.json\n", "with open(\"bias_results_gender.json\", \"w\") as outfile:\n", " outfile.write(json_object)" ] }, { "cell_type": "markdown", "id": "0dd62c50-ba0d-444b-9d12-499661fa0b93", "metadata": {}, "source": [ "### Step 4. Use Hugging Face `evaluate` to quantify the bias of the responses\n", "We'll explore the work from their blog post [here](https://huggingface.co/blog/evaluating-llm-bias)." ] }, { "cell_type": "code", "execution_count": null, "id": "a5083bf0-1713-4f84-b381-79b22805103c", "metadata": { "tags": [] }, "outputs": [], "source": [ "!pip install torch\n", "!pip install transformers\n", "!pip install evaluate" ] }, { "cell_type": "code", "execution_count": null, "id": "3d0168d2-2e7f-43da-be03-9f96ca92cfff", "metadata": { "tags": [] }, "outputs": [], "source": [ "import evaluate" ] }, { "cell_type": "markdown", "id": "a269c14e-bbd5-4d3d-8ff5-e583e29771e9", "metadata": {}, "source": [ "## Toxicity: assess how likely they are to produce problematic content, such as hate speech" ] }, { "cell_type": "code", "execution_count": null, "id": "ad4baa83-dab3-42b1-a3d5-875f899af0f7", "metadata": { "tags": [] }, "outputs": [], "source": [ "toxicity = evaluate.load(\"toxicity\")" ] }, { "cell_type": "code", "execution_count": null, "id": "98b879f7-9359-4001-8ccf-7d6450f25b74", "metadata": { "tags": [] }, "outputs": [], "source": [ "import json\n", " \n", "# Opening JSON file\n", "f = open(\"bias_results.json\")\n", " \n", "# returns JSON object as a dictionary\n", "data = json.load(f)" ] }, { "cell_type": "code", "execution_count": null, "id": "107ee373-9b24-4365-9c51-236d558ac5ab", "metadata": { "tags": [] }, "outputs": [], "source": [ "model_responses = []\n", "\n", "for category in data.keys():\n", " \n", " dict_prediction = data[category]\n", " string_prediction = next(iter(dict_prediction.values()))\n", "\n", " model_responses.append(string_prediction)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "7622a21a-ab02-4698-9d1c-a37fad5714a6", "metadata": { "tags": [] }, "outputs": [], "source": [ "toxicity.compute(predictions=model_responses, aggregation=\"ratio\")" ] }, { "cell_type": "markdown", "id": "205a427e-dc15-4461-b422-4fe1acbaa45b", "metadata": { "tags": [] }, "source": [ "## Language Polarity: evaluating whether it has different language polarity towards different demographic groups." ] }, { "cell_type": "code", "execution_count": null, "id": "f86d1e98-ad5f-4b06-94f7-0e2738c96761", "metadata": { "tags": [] }, "outputs": [], "source": [ "f_name = 'bold/prompts/race_prompt.json'\n", "f = open(f_name)\n", "race_prompts = json.load(f)\n", "\n", "sample_prompts_a = list(race_prompts['European_Americans'].items())[:10]\n", "sample_prompts_b = list(race_prompts['European_Americans'].items())[:10]" ] }, { "cell_type": "code", "execution_count": null, "id": "87d2d673-dd45-4e5e-bace-b5cfb749e7fd", "metadata": { "tags": [] }, "outputs": [], "source": [ "def save_model_responses(sample_prompts, file_name):\n", " responses = {}\n", "\n", " for subject, p in sample_prompts:\n", "\n", " prompt = p[0] \n", "\n", " output = test_llm(prompt)\n", "\n", " responses[subject] = {prompt:output}\n", "\n", " # try not to hit the throttle\n", " time.sleep(10)\n", "\n", " json_object = json.dumps(responses, indent=4)\n", "\n", " # Writing to sample.json\n", " with open(f\"bias_results_{file_name}.json\", \"w\") as outfile:\n", " outfile.write(json_object)\n", "\n", "save_model_responses(sample_prompts_a, 'race_a')\n", "save_model_responses(sample_prompts_b, 'race_b')" ] }, { "cell_type": "code", "execution_count": null, "id": "8d865738-5418-466a-a670-0ac99831ee06", "metadata": { "tags": [] }, "outputs": [], "source": [ "def get_model_responses_as_list(file_name):\n", " \n", " # Opening JSON file\n", " f = open(f\"bias_results_{file_name}.json\")\n", "\n", " # returns JSON object as a dictionary\n", " data = json.load(f)\n", "\n", " model_responses = []\n", "\n", " for category in data.keys():\n", "\n", " dict_prediction = data[category]\n", " string_prediction = next(iter(dict_prediction.values()))\n", "\n", " model_responses.append(string_prediction)\n", " return model_responses\n", "\n", "group_a_responses = get_model_responses_as_list('race_a')\n", "group_b_responses = get_model_responses_as_list('race_b')" ] }, { "cell_type": "code", "execution_count": null, "id": "ef2860b4-ecdd-4e03-8c66-a644cddb8d78", "metadata": { "tags": [] }, "outputs": [], "source": [ "# regard = evaluate.load(\"regard\", module_type=\"measurement\")\n", "regard = evaluate.load(\"regard\", \"compare\")" ] }, { "cell_type": "code", "execution_count": null, "id": "07dcfb22-323f-4d84-8a70-057deacce76c", "metadata": { "tags": [] }, "outputs": [], "source": [ "regard_results = regard.compute(data = group_a_responses, references = group_b_responses)\n", "print({k: round(v, 2) for k, v in regard_results['regard_difference'].items()})" ] }, { "cell_type": "markdown", "id": "cb3fce0d-bfc0-4c2c-bcc3-c1089925887e", "metadata": {}, "source": [ "### Based on the Regard scores above, the completions for race a have a more positive regard, whereas completions for race b have a more neutral regard.\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "b6b64d30-a4f3-4baa-83fd-cc0c7b8d405c", "metadata": { "tags": [] }, "source": [ "## Hurtful sentence completions" ] }, { "cell_type": "code", "execution_count": null, "id": "d355809b-5bf0-4f83-9480-b47db1888f9d", "metadata": { "tags": [] }, "outputs": [], "source": [ "!pip install --quiet unidecode" ] }, { "cell_type": "code", "execution_count": null, "id": "4bec752e-dbb7-4bf3-b4b2-155cf9bbf7c6", "metadata": { "tags": [] }, "outputs": [], "source": [ "honest = evaluate.load(\"honest\", \"en\")" ] }, { "cell_type": "code", "execution_count": null, "id": "e59ae263-3491-4327-9539-10ac441ccec8", "metadata": { "tags": [] }, "outputs": [], "source": [ "groups = ['race a', 'race b']\n", "honest_result = honest.compute(predictions=[group_a_responses, group_b_responses], groups=groups)" ] }, { "cell_type": "code", "execution_count": null, "id": "ead94e85-6443-4801-84cc-2c948dc0a8cf", "metadata": { "tags": [] }, "outputs": [], "source": [ "honest_result" ] }, { "cell_type": "markdown", "id": "3c2f439e-1b86-4afb-a3ff-571801e4509a", "metadata": { "tags": [] }, "source": [ "## Higher HONEST scores mean more hurtful completions. Based on the model completions above, we have evidence that the model does not generate more harmful completions for racial group a compared to group b." ] }, { "cell_type": "code", "execution_count": null, "id": "bebd74e9-9ec9-4629-b27b-4de9d57f1707", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 }, { "_defaultOrder": 55, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 56, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4de.24xlarge", "vcpuNum": 96 } ], "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Data Science)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 5 }