{ "cells": [ { "cell_type": "markdown", "id": "1a6ee048-b25c-42b8-9fb2-e9ee09736731", "metadata": {}, "source": [ "### 基于段落文档(非FAQ)知识的测试\n", "For closed-book QA, each demonstration is formatted as follows to construct few-shot prompts. \n", "Swapping the question with the evidence (longer distance between questions and answers) is found to consistently yield lower results across all datasets.\n", "\n", "#### Evidence: ...\n", "#### Question: ...\n", "#### Answer: ..." ] }, { "cell_type": "code", "execution_count": 14, "id": "f3a08e65-26c0-4cf0-b473-5af693057c80", "metadata": { "tags": [] }, "outputs": [], "source": [ "import boto3\n", "import re\n", "import json\n", "from typing import List\n", "\n", "sm_client = boto3.client(\"sagemaker-runtime\")\n", "\n", "# llm_endpoint = 'bloomz-7b1-mt-2023-04-19-09-41-24-189-endpoint'\n", "# llm_model_name = 'bloomz-7b'\n", "llm_endpoint = 'chatglm-2023-04-27-06-17-07-867-endpoint'\n", "llm_model_name = 'chatglm-7b'" ] }, { "cell_type": "code", "execution_count": 16, "id": "2148e354-e33c-49d6-840b-4a3cbdd507e2", "metadata": { "tags": [] }, "outputs": [], "source": [ "def enforce_stop_tokens(text: str, stop: List[str]) -> str:\n", " \"\"\"Cut off the text as soon as any stop words occur.\"\"\"\n", " if stop is None:\n", " return text\n", " \n", " return re.split(\"|\".join(stop), text)[0]\n", "\n", "def Generate(smr_client, llm_endpoint, prompt, llm_name, stop=None, history=[]):\n", " answer = None\n", " if llm_name == \"chatglm-7b\":\n", " # logger.info(\"call chatglm...\")\n", " parameters = {\n", " \"max_length\": 2048,\n", " \"temperature\": 0.01,\n", " \"num_beams\": 1, # >1可能会报错，\"probability tensor contains either `inf`, `nan` or element < 0\"； 即使remove_invalid_values=True也不能解决\n", " \"do_sample\": False,\n", " \"top_p\": 0.7,\n", " \"logits_processor\" : None,\n", " # \"remove_invalid_values\" : True\n", " }\n", " response_model = smr_client.invoke_endpoint(\n", " EndpointName=llm_endpoint,\n", " Body=json.dumps(\n", " {\n", " \"inputs\": prompt,\n", " \"parameters\": parameters,\n", " \"history\" : history\n", " }\n", " ),\n", " ContentType=\"application/json\",\n", " )\n", "\n", " json_ret = json.loads(response_model['Body'].read().decode('utf8'))\n", "\n", " answer = json_ret['outputs']\n", " elif llm_name == \"bloomz-7b\":\n", " # logger.info(\"call bloomz...\")\n", " parameters = {\n", " # \"early_stopping\": True,\n", " \"length_penalty\": 1.0,\n", " \"max_new_tokens\": 200,\n", " \"temperature\": 0,\n", " \"min_length\": 20,\n", " \"no_repeat_ngram_size\": 200,\n", " # \"eos_token_id\": ['\\n']\n", " }\n", "\n", " response_model = smr_client.invoke_endpoint(\n", " EndpointName=llm_endpoint,\n", " Body=json.dumps(\n", " {\n", " \"inputs\": prompt,\n", " \"parameters\": parameters\n", " }\n", " ),\n", " ContentType=\"application/json\",\n", " )\n", " \n", " json_ret = json.loads(response_model['Body'].read().decode('utf8'))\n", " answer = json_ret['outputs'][len(prompt):]\n", "\n", " return enforce_stop_tokens(answer, stop)" ] }, { "cell_type": "markdown", "id": "73ddd976-d589-4eba-88af-1f695c23768f", "metadata": {}, "source": [ "### 可以用以下代码调节三个变量, 去实验不同prompt模版和问题以及知识组合的效果\n", "- AWS_Knowledge_QA_Prompt\n", "- post_text\n", "- fewshots_str " ] }, { "cell_type": "code", "execution_count": 44, "id": "b49a1c87-58cb-4bbe-ac74-49cbaac31318", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "I\\'m sorry, but I\\'m not able to provide a specific answer to your question as I don\\'t have enough information about your problem. Could you please provide more details about the problem, such as the model number of your dishwasher, the error message or any other information that you\\'re seeing in the dishwasher?\n" ] } ], "source": [ "A_Role=\"customer\"\n", "B_Role=\"MideaBot\"\n", "STOP=[f\"\\n{A_Role}\", f\"\\n{B_Role}\"]\n", "\n", "post_text = \"why does dishwasher not run\"\n", "fewshots_str = \"\"\"Question: Dishwasher does not run or stops during a cycle\\nAnswer: Below are some possible causes and the related solution:\\n\\nCause: There is no power to the dishwasher.\\nSolution: Please replace fuse or reset circuit breaker.\\n\\nCause: The door is not closed tightly and latched.\\nSolution: Close and latch the door.\\n\\nCause: Most cycles pause for 5 minutes near the start of a cycle to let the dishes soak.\\nSolution: This is normal, and reduces the energy required to clean the dishes.\\n\\nCause: Certain cycles repeatedly pause for 15 to 65 seconds during the main wash.\\nSolution: This is normal.\\n\\nCause: The Delay feature is selected.\\nSolution: To reset, see “Options” in the Controls section. \\n\\n\\n\\nQuestion: Dishwasher will not fill\\nAnswer: Below are some possible causes and the related solution:\\n\\nCause: The water shutoff valve is not turned on.\\nSolution: Turn on the water valve.\\n\\nCause: The door is not closed tightly and latched.\\nSolution: Close and latch the door.\\n\\nCause: Water pressure is too low. Verify a minimum of\\nSolution: 20 psi water pressure is maintained during the filling operation.\\n\\n\\n\\n\\nQuestion: Dishwasher seems to run too long\\nAnswer: Below are some possible causes and the related solution:\\n\\nCause: The water supplied to the dishwasher is not hot enough.\\nSolution: Run the water in the adjacent sink until it gets hot before turning on the dishwasher.\\n\\nCause: The dishwasher runs longer while heating water, and a delay automatically occurs in some wash and rinse cycles until the water reaches the proper temperature.\\nSolution: same to previous.\\n\\nCause: The Sanitize option is selected.\\nSolution: The cycle time increases to heat the water to the correct temperature for sanitizing.\\n\\nCause: The cycle time will vary depending on soil level. When heavy soil is detected, the Normal cycle will automatically increase the cycle time.\\nSolution: Dishwasher is connected to the Cold water supply. Verify the dishwasher is connected correctly\\n\\n\"\"\"\n", "\n", "\n", "AWS_Knowledge_QA_Prompt = \"\"\"{B} is customer service chatbot of Midea, {B} can answer customers' questions according to the following document segments ```{fewshots}```, and {B} answer ```not find any message``` if not find related message in ```{fewshots}```\\n\\n{A}: {question}\\n{B}: \"\"\"\n", "final_prompt = AWS_Knowledge_QA_Prompt.format(fewshots=fewshots_str, question=post_text, A=A_Role, B=B_Role)\n", "\n", "answer = Generate(sm_client, llm_endpoint, prompt=final_prompt, llm_name=llm_model_name, stop=STOP)\n", "print(answer)" ] }, { "cell_type": "code", "execution_count": 19, "id": "db3ee4c5-6654-4205-ac35-899bb799ffdb", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "AWS Clean Rooms 是 AWS 提供的用于数据集中进行协作和分析的工具，可以帮助用户和合作伙伴在数据集中执行 SQL 查询和分析，而无需将数据导出或移动到其他平台。AWS Clean Rooms 提供了一个洁净室的概念，使用户可以在原始数据位置读取数据并应用内置的分析规则，从而遵守严格的数据处理政策。它还提供了可配置的数据访问控制和审计支持控件，帮助用户确保数据的安全性和合规性。使用 AWS Clean Rooms，用户可以轻松创建自己的数据洁净室，并与数千家已经在使用的公司合作，快速分析数据集并提取新的见解。\n" ] } ], "source": [ "A_Role=\"用户\"\n", "B_Role=\"AWSBot\"\n", "STOP=[f\"\\n{A_Role}\", f\"\\n{B_Role}\"]\n", "\n", "post_text = \"什么是AWS Clean Rooms\"\n", "fewshots_str = \"\"\"AWS Clean Rooms帮助您和您的合作伙伴分析和协作处理您的集体数据集，以获得新的见解，而无需相互透露基础数据。您可以使用AWSClean Rooms它在几分钟内创建自己的洁净室，只需几个步骤即可开始分析您的集体数据集。您可以选择要与之协作的合作伙伴，选择他们的数据集，并为参与者配置限制。\n", "借助AWS Clean Rooms，您可以与数千家已经在使用的公司合作AWS。协作不需要将数据移出AWS或加载到其他平台。运行查询时，从其原始位置AWSClean Rooms读取数据并应用内置分析规则来帮助您保持对数据的控制。\n", "AWSClean Rooms提供您可以配置的内置数据访问控制和审计支持控件。这些控件包括：\n", "1.分析规则，限制SQL查询和提供输出限制的机制\n", "2.加密计算，即使Clean Rooms在处理查询时也能保持数据加密，以遵守严格的数据处理政策\n", "3.查询日志，以查看查询并帮助支持审计\n", "\"\"\"\n", "\n", "AWS_Knowledge_QA_Prompt = \"\"\"{B} 是AWS的智能客服, {B} 能参考以下文档回答客户的问题 ```{fewshots}```\\n\\n{A}: {question}\\n{B}: \"\"\"\n", "final_prompt = AWS_Knowledge_QA_Prompt.format(fewshots=fewshots_str, question=post_text, A=A_Role, B=B_Role)\n", "\n", "answer = Generate(sm_client, llm_endpoint, prompt=final_prompt, llm_name=llm_model_name, stop=STOP)\n", "print(answer)" ] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 } ], "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Data Science)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" } }, "nbformat": 4, "nbformat_minor": 5 }