{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8af3794b",
   "metadata": {},
   "source": [
    "# Introduction to SageMaker Built-In Algorithms - Text Generation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "446b1b24",
   "metadata": {},
   "source": [
    "---\n",
    "In this demo notebook, we demonstrate how to use the SageMaker Python SDK to deploy Open-LLAMA model for text generation. It is a permissively licensed (Apache-2.0) open source reproduction of [Meta AI’s LLaMA 7B](https://huggingface.co/decapoda-research/llama-7b-hf) trained on the [RedPajama dataset](https://www.together.xyz/blog/redpajama) which is a reproduction of the LLaMA training dataset containing over 1.2 trillion tokens.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b55e677-3429-4668-b100-bd63d2a4c401",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip uninstall -y sagemaker --quiet\n",
    "!pip install sagemaker --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "014424a8-7f8f-46a7-8963-2c3d454878b8",
   "metadata": {
    "jumpStartAlterations": [
     "modelIdVersion"
    ]
   },
   "outputs": [],
   "source": [
    "model_id, model_version, = (\n",
    "    \"huggingface-textgeneration-open-llama\",\n",
    "    \"*\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e52afae-868d-4736-881f-7180f393003a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.jumpstart.model import JumpStartModel\n",
    "from sagemaker.serializers import JSONSerializer\n",
    "\n",
    "my_model = JumpStartModel(model_id=model_id)\n",
    "predictor = my_model.deploy()\n",
    "\n",
    "predictor.serializer = JSONSerializer()\n",
    "predictor.content_type = \"application/json\"\n",
    "\n",
    "payload = {\n",
    "    \"text_inputs\": \"Building a website can be done in 10 simple steps:\",\n",
    "    \"max_length\": 110,\n",
    "    \"no_repeat_ngram_size\": 3,\n",
    "}\n",
    "response = predictor.predict(payload)\n",
    "print(response[\"generated_texts\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d14ad3c1",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# Sentiment-analysis\n",
    "payload = {\n",
    "    \"text_inputs\": \"\"\"\"I hate it when my phone battery dies.\"\n",
    "                Sentiment: Negative\n",
    "                ###\n",
    "                Tweet: \"My day has been :+1:\"\n",
    "                Sentiment: Positive\n",
    "                ###\n",
    "                Tweet: \"This is the link to the article\"\n",
    "                Sentiment: Neutral\n",
    "                ###\n",
    "                Tweet: \"This new music video was incredibile\"\n",
    "                Sentiment:\"\"\"\n",
    "}\n",
    "response = predictor.predict(payload)\n",
    "print(response[\"generated_texts\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57c1c1aa",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# Question answering\n",
    "payload = {\n",
    "    \"text_inputs\": \"Could you remind me when was the C programming language invented?\",\n",
    "    \"max_length\": 34,\n",
    "}\n",
    "response = predictor.predict(payload)\n",
    "print(response[\"generated_texts\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16fa885e",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# Recipe generation\n",
    "payload = {\"text_inputs\": \"What is the recipe for a delicious lemon cheesecake?\", \"max_length\": 70}\n",
    "response = predictor.predict(payload)\n",
    "print(response[\"generated_texts\"][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ef7207e-01ba-4ac2-b4a9-c8f6f0e1c498",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Supported Parameters\n",
    "\n",
    "***\n",
    "This model supports many parameters while performing inference. They include:\n",
    "\n",
    "* **max_length:** Model generates text until the output length (which includes the input context length) reaches `max_length`. If specified, it must be a positive integer.\n",
    "* **num_return_sequences:** Number of output sequences returned. If specified, it must be a positive integer.\n",
    "* **num_beams:** Number of beams used in the greedy search. If specified, it must be integer greater than or equal to `num_return_sequences`.\n",
    "* **no_repeat_ngram_size:** Model ensures that a sequence of words of `no_repeat_ngram_size` is not repeated in the output sequence. If specified, it must be a positive integer greater than 1.\n",
    "* **temperature:** Controls the randomness in the output. Higher temperature results in output sequence with low-probability words and lower temperature results in output sequence with high-probability words. If `temperature` -> 0, it results in greedy decoding. If specified, it must be a positive float.\n",
    "* **early_stopping:** If True, text generation is finished when all beam hypotheses reach the end of sentence token. If specified, it must be boolean.\n",
    "* **do_sample:** If True, sample the next word as per the likelihood. If specified, it must be boolean.\n",
    "* **top_k:** In each step of text generation, sample from only the `top_k` most likely words. If specified, it must be a positive integer.\n",
    "* **top_p:** In each step of text generation, sample from the smallest possible set of words with cumulative probability `top_p`. If specified, it must be a float between 0 and 1.\n",
    "* **seed:** Fix the randomized state for reproducibility. If specified, it must be an integer.\n",
    "* **return_full_text:** If True, input text will be part of the output generated text. If specified, it must be boolean. The default value for it is False.\n",
    "\n",
    "We may specify any subset of the parameters mentioned above while invoking an endpoint.\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e062d29",
   "metadata": {},
   "source": [
    "### Clean up the endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24cc5560",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Delete the SageMaker endpoint\n",
    "predictor.delete_model()\n",
    "predictor.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}