{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "da51ef5c-c3b9-4524-8dd1-0cc1d1aabb33",
   "metadata": {},
   "source": [
    "# Chain of Thought Prompt Engineering\n",
    "\n",
    "References: \n",
    "- https://medium.com/nlplanet/two-minutes-nlp-making-large-language-models-reason-with-chain-of-thought-prompting-401fd3c964d0\n",
    "- https://arxiv.org/pdf/2201.11903.pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "021eae70-425a-4b4e-a1c5-f9d6a0fb19b9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "import boto3\n",
    "sess = sagemaker.Session()\n",
    "import json\n",
    "\n",
    "sm_client = boto3.client(\"sagemaker\")\n",
    "smr_client = boto3.client(\"sagemaker-runtime\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3671a5e-7178-4fcf-8a6d-6c192e6e3901",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "endpoint_name = 'huggingface-pytorch-inference-2023-03-09-02-00-08-813'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b76269c2-84cf-4e9c-8fd6-7a4a98bd653f",
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    \"early_stopping\": True,\n",
    "    \"length_penalty\": 2.0,\n",
    "    \"max_new_tokens\": 50,\n",
    "    \"temperature\": .1,\n",
    "    \"min_length\": 10,\n",
    "    \"no_repeat_ngram_size\": 3,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c7ffb25-c448-40a7-bb40-0ea6429c484e",
   "metadata": {},
   "source": [
    "## Mathematical Reasoning \n",
    "### Zero-shot Prompting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ff14937-e376-4d8a-807d-5d1f8bb2b433",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"QUESTION: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can have 3 tennis balls. How many tennis balls does he have now?\n",
    "ANSWER: The answer is 11.\n",
    "\n",
    "QUESTION: John takes care of 10 dogs. Each dog takes .5 hours a day to walk and take care of their business. How many hours a week does he spend taking care of dogs?\n",
    "ANSWER:\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "## expected output:\n",
    "# # [{'generated_text': 'The answer is 7 hours a day because 10 x.5 = 7 hours. He spends 7 hours taking care of dogs a week because 7 x 7 = 56 hours.'}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da3bf110-b2fa-43d9-af22-9cea0b292f9e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a78c66d5-7e84-4770-b286-4cec6bc9bdcd",
   "metadata": {},
   "source": [
    "### With Chain of Thought Prompting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d9101e6-95ae-4cd9-b0bf-247dabd13939",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"QUESTION: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?\n",
    "ANSWER: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.\n",
    "\n",
    "QUESTION: John takes care of 10 dogs. Each dog takes .5 hours a day to walk and take care of their business. How many hours a week does he spend taking care of dogs?\n",
    "ANSWER:\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "# # [{'generated_text': '\"He spends 10 *.5 = 5 hours a day taking care of dogs. So he spends 5 * 7 = 35 hours / week taking care dogs. The answer is 35.\"'}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "486ec9b0-3166-435d-857b-e3e020fb72a4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c2a7a9a-560a-4e8c-af66-3dd60a3f26b4",
   "metadata": {},
   "source": [
    "## Advanced Mathematical Reasoning - with chain of thought prompting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73e3e4c6-d47c-412c-a60f-8109dd5509c0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"QUESTION: Ducks need to eat 3.5 pounds of insects each week to survive. If there is a flock of ten ducks, how many pounds of insects do they need per day?\n",
    "ANSWER: Ducks need 3.5 pounds of insects each week. If there is a flock of 10 ducks, then they need 3.5 x 10 = 35 pounds of insects each week. If they need 35 pounds of insects each week, then they need 35 / 7 = 5 pounds of insects each day. The answer is 5. \n",
    "\n",
    "QUESTION: It takes Matthew 3 minutes to dig a small hole for shrubs and 10 minutes to dig a large hole for trees. How many hours will it take him to dig 30 small holes and 15 large holes?\n",
    "ANSWER: It takes Matthew 3 minutes to dig a small hole and 10 minutes to dig a large hole. So, it takes Matthew 3 x 30 = 90 minutes to dig 30 small holes. It takes Matthew 10 x 15 = 150 minutes to dig 15 large holes. So, it takes Matthew 90 + 150 = 240 minutes to dig 30 small holes and 15 large holes. 240 minutes is 4 hours. The answer is 4 hours. \n",
    "\n",
    "QUESTION: I have 10 liters of orange drink that are two-thirds water and I wish to add it to 15 liters of pineapple drink that is three-fifths water. But as I pour it, I spill one liter of the orange drink. How much water is in the remaining 24 liters?\n",
    "ANSWER:\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "# # [{'generated_text': '\"The orange drink is 10 x 2 / 3 = 8 liters of water. The pineapple drink is 15 x 3 / 5 = 12 liter of water in it. The total water in the orange and pineapple drinks is 8\"'}]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "055a2507-4a7d-4a95-8a63-23f089f3830c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "688bf575-84ba-41d5-b8fd-5087ba8460b8",
   "metadata": {},
   "source": [
    "As you can see in the above example for complex mathematical reasoning the models might not give you the right predicted output. \n",
    "The correnct answer is: \n",
    "\n",
    "\"The orange drink is 10liters, 1 liter was dropped, remaining drink has 9 * 2/3 = 6 liters of water. The pineapple drink is 15 x 3 / 5 = 9 liter of water in it. The total water in the orange and pineapple drinks is 15\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4606545c-6a9d-46b6-865c-27132795575c",
   "metadata": {},
   "source": [
    "## Symbolic Reasoning\n",
    "For symbolic reasoning, consider the tasks of last letter concatenation, reverse list, and coin flip shown in the next image."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "480b3ef3-f76d-4d51-a531-f565a390a8b3",
   "metadata": {},
   "source": [
    "### Zero shot prompting"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1be113e0-6240-43ae-b6af-ea25c5839944",
   "metadata": {},
   "source": [
    "### Last Letter Concatenation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1d40e14-f4e2-42a3-97cd-04f574a84679",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"QUESTION: Take the last letters of the words in \"Elon Musk\" and con-catenate them.\n",
    "ANSWER: \n",
    "\n",
    "\"\"\"\n",
    "\n",
    "# # [{\"generated_text\":\"musk elon n\"}]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c275d2d-7a50-480e-9a3e-c56581c8d5ff",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8646ff76-7ebe-4bed-bd5e-cdaa7e8d86c4",
   "metadata": {},
   "source": [
    "### With Chain of thought prompting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb9b491b-a601-44c8-b29a-99d4815c1417",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"QUESTION: Take the last letters of the words in \"Elon Musk\" and con-catenate them.\n",
    "ANSWER: The last letter of \"Elon\" is \"n\". The last letter of \"Musk\" is \"k'. Concatenating them is \"nk\". So the answer is nk.\n",
    "\n",
    "QUESTION: Take the last letters of the words in \"Mani Khanuja\" and con-catenate them.\n",
    "ANSWER: The last letter of \"Mani\" is \"i\". The last letter of \"Khanuja\" is \"a\". Concatenating them is \"ia\". So the answer is ia. \n",
    "\n",
    "QUESTION: Take the last letters of the words in \"John Doe\" and con-catenate them.\n",
    "ANSWER:\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "# #[{\"generated_text\":\"Doe. So the answer is Doe\"}]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4179187b-1b08-4a8c-8b8e-838186368d51",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9040836-3f01-4863-a366-7b9592565907",
   "metadata": {},
   "source": [
    "### Reverse List\n",
    "\n",
    "### Zero shot prompting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed5cd80b-42bc-4f33-aeae-f546597eb6f6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"QUESTION: Reverse the sequence \"glasses, pen, alarm, license\".\n",
    "ANSWER: \n",
    "\n",
    "\"\"\"\n",
    "\n",
    "# # [{\"generated_text\":\"license, alarm, pen, glasses, reversed\"}]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e4cbde2d-31ca-422c-94b3-897b9e0b67d2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88f2ab1b-7c41-4978-9655-7ca9dbc487c8",
   "metadata": {},
   "source": [
    "### With Chain of Thought prompting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6ffac0a-ef37-4e40-a655-bf1e65a42676",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"QUESTION: Reverse the sequence \"glasses, pen, alarm, license\".\n",
    "ANSWER: First is glasses. Second is pen. Third is alarm. Fourth is license. Now to reverse, change the order to: Fourth is license.\n",
    "Third is alarm. Second is pen. First is glasses. So the answer is\n",
    "\"license, alarm, pen, glasses\".\n",
    "\n",
    "QUESTION: Reverse the sequence \"telephone, clock, board, spectacles\".\n",
    "ANSWER: First is telephone. Second is clock. Third is board. Fourth is spectacles. Now to reverse, change the order to: Fourth is spectacles.\n",
    "Third is board. Second is clock. First is telephone. So the answer is\n",
    "\"spectacles, board, clock, telephone\".\n",
    "\n",
    "QUESTION: Reverse the sequence \"cup, plate, food, fruits\".\n",
    "ANSWER:\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "# # [{\"generated_text\":\"fruits, food, plate, cup\\\\\" is correct.\"}]'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5d7f6c93-674d-48b7-a937-290fe6ce4b7d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdbe8381-d5c6-4265-926c-33af9bf58fd2",
   "metadata": {},
   "source": [
    "### Coin Flip\n",
    "### Zero shot prompting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d5c9664-659c-48a3-8bfa-9b3c2af29d98",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"QUESTION:  A coin is heads up. John does not flip the coin. Shalonda does not flip the coin. Is the coin still heads up?\n",
    "ANSWER: \n",
    "\n",
    "\"\"\"\n",
    "\n",
    "## [{\"generated_text\":\"yes.... and it will remain heads up.\"}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c94a0f94-0096-419b-86a5-29b7520642e3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1acdad74-8277-4c2d-b721-94d9e9af7f64",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = \"\"\"Q: A coin is heads up. Maybelle flips the coin. Shalonda does not flip the coin. Is the coin still heads up?\n",
    "A: The coin was flipped by Maybelle. So the coin was flipped 1 time, which is an odd number. The coin started heads up, so after an odd number of flips, it will be tails up. So the answer\n",
    "is no.\n",
    "\n",
    "QUESTION:  A coin is heads up. John does not flip the coin. Shalonda does not flip the coin. Is the coin still heads up?\n",
    "ANSWER:\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "## '[{\"generated_text\":\"The coin is still heads up because John and Shalonda did not flip the coin. So the answer is yes.\"}]'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "084eab74-cd14-4c90-971a-0aa7a5dfdd59",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "response_model = smr_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name,\n",
    "    Body=json.dumps(\n",
    "        {\n",
    "            \"inputs\": payload,\n",
    "            \"parameters\": parameters,\n",
    "        }\n",
    "    ),\n",
    "    ContentType=\"application/json\",\n",
    ")\n",
    "\n",
    "response_model[\"Body\"].read().decode(\"utf8\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6fdf3963-a797-41c9-8591-38eb8281771d",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (PyTorch 1.10 Python 3.8 CPU Optimized)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/pytorch-1.10-cpu-py38"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}