{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Run Inference on the FLAN-T5 Model BEFORE Fine Tuning\n", "\n", "In this lab we will now switch gears to the dialogue summarization task using generative AI instead of the data processing section. Prompt engineering is an important concept to using foundation models for text generation. We will explore how the input text can directly impact the output of the model in this notebook. Check out [this blog](https://www.amazon.science/blog/emnlp-prompt-engineering-is-the-new-feature-engineering) from Amazon Science for a quick introduction to prompt engineering.\n", "\n", "For our specific use case, we can generate summaries with the FLAN-T5 model using zero-shot inference to see how well the base LLM performs without any fine-tuning. Check out [this blog from AWS](https://aws.amazon.com/blogs/machine-learning/zero-shot-prompting-for-the-flan-t5-foundation-model-in-amazon-sagemaker-jumpstart/) for a quick description of what zero shot learning is and why it is an important concept to the FLAN model and many more!" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "\n", "## Set up Kernel and Required Dependencies" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "First, check that the correct kernel is chosen.\n", "\n", "\n", "\n", "You can click on that to see and check the details of the image, kernel, and instance type.\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import psutil\n", "\n", "notebook_memory = psutil.virtual_memory()\n", "print(notebook_memory)\n", "\n", "if notebook_memory.total < 32 * 1000 * 1000 * 1000:\n", " print('*******************************************') \n", " print('YOU ARE NOT USING THE CORRECT INSTANCE TYPE')\n", " print('PLEASE CHANGE INSTANCE TYPE TO m5.2xlarge ')\n", " print('*******************************************')\n", "else:\n", " correct_instance_type=True" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "%store -r setup_dependencies_passed" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "try:\n", " setup_dependencies_passed\n", "except NameError:\n", " print(\"++++++++++++++++++++++++++++++++++++++++++++++\")\n", " print(\"[ERROR] YOU HAVE TO RUN NOTEBOOK #01. \")\n", " print(\"You did not install the required libraries. \")\n", " print(\"++++++++++++++++++++++++++++++++++++++++++++++\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "model_checkpoint='google/flan-t5-base'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "%store model_checkpoint" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "%store -r model_checkpoint" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "huggingface_dataset_name = \"knkarthick/dialogsum\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Load the Summarization Dataset" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "from datasets import load_dataset\n", "dataset = load_dataset(huggingface_dataset_name)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "example_indices = [40, 80, 160,]\n", "print('Example Input Dialogue:')\n", "print(dataset['test'][example_indices[0]]['dialogue'])\n", "print()\n", "print('Example Output Summary:')\n", "print(dataset['test'][example_indices[0]]['summary'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Load the LLM and its Tokenizer" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "iAYlS40Z3l-v", "tags": [] }, "outputs": [], "source": [ "from transformers import AutoTokenizer\n", " \n", "tokenizer = AutoTokenizer.from_pretrained(model_checkpoint, use_fast=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sPqQA3TT3l_I", "tags": [] }, "outputs": [], "source": [ "from transformers import AutoModelForSeq2SeqLM\n", "\n", "model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Explore What Happens Without Prompt Engineering\n", "\n", "Without any prompt engineering (changes to the dialogue text), you can see below how the model is unsure of what task it is supposed to accomplish, so it tries to make up the next sentence in the dialogue. Not a bad guess from the model, but it isn't the task we want the model to perform in this situation." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "dialogue = dataset['test'][example_indices[0]]['dialogue']\n", "summary = dataset['test'][example_indices[0]]['summary']\n", "\n", "inputs = tokenizer(dialogue, return_tensors='pt')\n", "output = tokenizer.decode(\n", " model.generate(\n", " inputs[\"input_ids\"], \n", " max_new_tokens=50,\n", " )[0], \n", " skip_special_tokens=True\n", ")\n", "print(f'INPUT PROMPT:\\n{dialogue}\\n')\n", "print(f'MODEL GENERATION:\\n{output}')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "dialogue = dataset['test'][example_indices[1]]['dialogue']\n", "summary = dataset['test'][example_indices[1]]['summary']\n", "\n", "inputs = tokenizer(dialogue, return_tensors='pt')\n", "output = tokenizer.decode(\n", " model.generate(\n", " inputs[\"input_ids\"], \n", " max_new_tokens=50,\n", " )[0], \n", " skip_special_tokens=True\n", ")\n", "print(f'INPUT PROMPT:\\n{dialogue}\\n')\n", "print(f'MODEL GENERATION:\\n{output}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Add an Text Prompt to the Model Input\n", "\n", "Adding the \"summary:\" command to the end seems to really help the model understand what it is supposed to do now. Notice how the model still does not pick up on the nuance of the conversations though. This is what we will hope to solve via fine-tuning." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "start_prompt = 'Summarize the following conversation.\\n'\n", "end_prompt = '\\n\\nSummary: '\n", "dialogue = dataset['test'][example_indices[0]]['dialogue']\n", "summary = dataset['test'][example_indices[0]]['summary']\n", "prompt = f'{start_prompt}{dialogue}{end_prompt}'\n", "\n", "inputs = tokenizer(prompt, return_tensors='pt')\n", "output = tokenizer.decode(\n", " model.generate(\n", " inputs[\"input_ids\"], \n", " max_new_tokens=50,\n", " )[0], \n", " skip_special_tokens=True\n", ")\n", "print(f'INPUT PROMPT:\\n{prompt}\\n')\n", "print(f'MODEL GENERATION:\\n{output}\\n')\n", "print(f'BASELINE SUMMARY:\\n{summary}')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "start_prompt = 'Summarize the following conversation.\\n'\n", "end_prompt = '\\n\\nSummary: '\n", "dialogue = dataset['test'][example_indices[1]]['dialogue']\n", "summary = dataset['test'][example_indices[1]]['summary']\n", "prompt = f'{start_prompt}{dialogue}{end_prompt}'\n", "\n", "inputs = tokenizer(prompt, return_tensors='pt')\n", "output = tokenizer.decode(\n", " model.generate(\n", " inputs[\"input_ids\"], \n", " max_new_tokens=50,\n", " )[0], \n", " skip_special_tokens=True\n", ")\n", "print(f'INPUT PROMPT:\\n{prompt}\\n')\n", "print(f'MODEL GENERATION:\\n{output}\\n')\n", "print(f'BASELINE SUMMARY:\\n{summary}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Try Out a Different Prompt Template From FLAN\n", "\n", "The next natural thing to check is if the text at the start and end of the dialogue are optimized to the task at hand. FLAN has many prompt templates that are published for certain tasks [here](https://github.com/google-research/FLAN/tree/main/flan/v2). As you can see below, selecting one of the pre-built FLAN prompts does help in the second example, but the first example still struggles to pick up on the nuance of the conversation." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "start_prompt = 'Dialogue:\\n'\n", "end_prompt = '\\nWhat was going on?'\n", "dialogue = dataset['test'][example_indices[0]]['dialogue']\n", "summary = dataset['test'][example_indices[0]]['summary']\n", "prompt = f'{start_prompt}{dialogue}{end_prompt}'\n", "\n", "inputs = tokenizer(prompt, return_tensors='pt')\n", "output = tokenizer.decode(\n", " model.generate(\n", " inputs[\"input_ids\"], \n", " max_new_tokens=50,\n", " )[0], \n", " skip_special_tokens=True\n", ")\n", "print(f'INPUT PROMPT:\\n{prompt}\\n')\n", "print(f'MODEL GENERATION:\\n{output}\\n')\n", "print(f'BASELINE SUMMARY:\\n{summary}')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "start_prompt = 'Dialogue:\\n'\n", "end_prompt = '\\nWhat was going on?'\n", "dialogue = dataset['test'][example_indices[1]]['dialogue']\n", "summary = dataset['test'][example_indices[1]]['summary']\n", "prompt = f'{start_prompt}{dialogue}{end_prompt}'\n", "\n", "inputs = tokenizer(prompt, return_tensors='pt')\n", "output = tokenizer.decode(\n", " model.generate(\n", " inputs[\"input_ids\"], \n", " max_new_tokens=50,\n", " )[0], \n", " skip_special_tokens=True\n", ")\n", "print(f'INPUT PROMPT:\\n{prompt}\\n')\n", "print(f'MODEL GENERATION:\\n{output}\\n')\n", "print(f'BASELINE SUMMARY:\\n{summary}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Use few-shot Inference\n", "\n", "Few shot inference is the practice of providing an LLM examples of what outputs should look like for a given task. You can read more about it in [this blog from HuggingFace](https://huggingface.co/blog/few-shot-learning-gpt-neo-and-inference-api) which goes over why is a useful tool and how it can be used!\n", "\n", "In our example, you can see how providing 2 examples to the model provides the model more information and qualitatively improves the summary in the example below." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "start_prompt = 'Dialogue:\\n'\n", "end_prompt = '\\nWhat was going on? '\n", "stop_sequence = '\\n\\n\\n'" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "def make_prompt(num_shots):\n", " prompt = ''\n", " for i in range(num_shots + 1):\n", " if i == num_shots:\n", " dialogue = dataset['test'][example_indices[0]]['dialogue']\n", " summary = dataset['test'][example_indices[0]]['summary']\n", " prompt = prompt + f'{start_prompt}{dialogue}{end_prompt}'\n", " else:\n", " dialogue = dataset['test'][example_indices[i+1]]['dialogue']\n", " summary = dataset['test'][example_indices[i+1]]['summary']\n", " prompt = prompt + f'{start_prompt}{dialogue}{end_prompt}{summary}\\n{stop_sequence}\\n'\n", " return prompt" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "few_shot_prompt = make_prompt(2)\n", "print(few_shot_prompt)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "inputs = tokenizer(few_shot_prompt, return_tensors='pt')\n", "output = tokenizer.decode(\n", " model.generate(\n", " inputs[\"input_ids\"],\n", " max_new_tokens=50,\n", " )[0], \n", " skip_special_tokens=True\n", ")\n", "print(f'FEW SHOT RESPONSE: {output}')\n", "summary = dataset['test'][example_indices[0]]['summary']\n", "print(f'EXPECTED RESPONSE: {summary}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Conclusion\n", "\n", "As you can see, prompt engineering can take us a long way for this use case, but there are some limitations. Next, we will start to explore how you can use fine-tuning to help your LLM to understand a particular use case in better depth!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Release Resources" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "%%html\n", "\n", "

Shutting down your kernel for this notebook to release resources.

\n", "\n", " \n", "" ] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 }, { "_defaultOrder": 55, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 56, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4de.24xlarge", "vcpuNum": 96 } ], "colab": { "name": "Fine-tune a language model", "provenance": [] }, "instance_type": "ml.m5.2xlarge", "kernelspec": { "display_name": "Python 3 (Data Science)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" }, "vscode": { "interpreter": { "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49" } } }, "nbformat": 4, "nbformat_minor": 4 }