{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Summarization on Custom Dataset with SageMaker Jumpstart and [LangChain](https://python.langchain.com/en/latest/index.html) Library\n", "\n", "Reference: https://github.com/gkamradt/langchain-tutorials/tree/main/data_generation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", " There are two main types of methods for summarizing text: abstractive and extractive.\n", "\n", "Abstractive summarization generates a new shorter summary in its own words based on understanding the meaning and concepts of the original text. It analyzes the text using advanced natural language techniques to grasp the key ideas and then expresses those ideas in a summarized form using different words and phrases. This is similar to how humans summarize by reading something and then explaining the main points in their own words.\n", "\n", "Extractive summarization works by selecting the most important sentences, phrases or words from the original text to construct a summary. It calculates the weight or importance of each part of the text using algorithms and then chooses the parts with the highest weights to put into the summary. This pulls summarizes by extracting key elements from the text itself rather than interpreting the meaning.\n", "\n", "So in short, abstractive summarization rewrites the key ideas in new words while extractive summarization selects the most salient parts of the existing text. Both aim to distill the essence and most significant information from the original document into a condensed summary.\n", "\n", "We're going to run through 3 methods for summarization that start with basic prompting to summarizing large documents using `map_reduce` method. These aren't the only options, feel free to modify it based on your use case. \n", "\n", "**3 Levels Of Summarization:**\n", "1. **Summarize a couple sentences** - Basic Prompt\n", "2. **Summarize a couple paragraphs** - Prompt Templates\n", "3. **Summarize a large document with multiple pages** - Map Reduce\n", "4. **Summarize a book**\n", "\n", "In this notebook we will demonstrate how to use **AI21 Summary API** for text summarization using a library of documents as a reference.\n", "\n", "**This notebook serves a template such that you can easily replace the example dataset by your own to build a custom text summarization application.**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Deploy large language model (LLM) and embedding model in SageMaker JumpStart\n", "\n", "Make sure to deploy the ai21 summary model from jumpstart before you begin following the notebook and provide the endpoint here. You can do this by subscribing to the AI21 Summarize model, then clicking on `Open Notebook` option. This will open the notebook in Amazon SageMaker Studio. Run through the notebook to deploy the model, capture the endpoint name and return to this notebook. " ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false }, "pycharm": { "name": "#%%\n" }, "scrolled": true, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\u001b[33mWARNING: jsonschema 3.2.0 does not provide the extra 'format-nongpl'\u001b[0m\u001b[33m\n", "\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n", "\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.1.2\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n" ] } ], "source": [ "!pip install --upgrade sagemaker --quiet\n", "!pip install ipywidgets==7.0.0 --quiet\n", "!pip install langchain==0.0.148 --quiet\n", "!pip install faiss-cpu --quiet\n", "!pip install pytesseract --quiet\n", "!pip install unstructured --quiet\n", "!pip install transformers --quiet\n", "!pip install pypdf --quiet" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/opt/conda/lib/python3.8/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n", " warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n" ] } ], "source": [ "import time\n", "import sagemaker, boto3, json\n", "from sagemaker.session import Session\n", "from sagemaker.model import Model\n", "from sagemaker import image_uris, model_uris, script_uris, hyperparameters\n", "from sagemaker.predictor import Predictor\n", "from sagemaker.utils import name_from_base\n", "from typing import Any, Dict, List, Optional\n", "from langchain.embeddings import SagemakerEndpointEmbeddings\n", "from langchain.llms.sagemaker_endpoint import ContentHandlerBase\n", "from langchain import PromptTemplate\n", "\n", "sagemaker_session = Session()\n", "aws_role = sagemaker_session.get_caller_identity_arn()\n", "aws_region = boto3.Session().region_name\n", "sess = sagemaker.Session()\n", "model_version = \"*\"\n", "endpoint_name = 'summarize' # replace this with your endpoint name." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summarize couple of sentences " ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "tags": [] }, "outputs": [], "source": [ "prompt = \"\"\"\n", "Philosophy (from Greek: φιλοσοφία, philosophia, 'love of wisdom') \\\n", "is the systematized study of general and fundamental questions, \\\n", "such as those about existence, reason, knowledge, values, mind, and language. \\\n", "Some sources claim the term was coined by Pythagoras (c. 570 – c. 495 BCE), \\\n", "although this theory is disputed by some. Philosophical methods include questioning, \\\n", "critical discussion, rational argument, and systematic presentation.\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we wrap up our SageMaker endpoints for LLM into `langchain.llms.sagemaker_endpoint.SagemakerEndpoint`. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "tags": [] }, "outputs": [], "source": [ "from langchain.llms.sagemaker_endpoint import LLMContentHandler, SagemakerEndpoint\n", "\n", "class ContentHandler(LLMContentHandler):\n", " content_type = \"application/json\"\n", " accepts = \"application/json\"\n", "\n", " def transform_input(self, prompt: str, model_kwargs={}) -> bytes:\n", " input_str = json.dumps({\n", " \"source\":prompt,\n", " \"sourceType\":\"TEXT\"})\n", " return input_str.encode(\"utf-8\")\n", "\n", " def transform_output(self, output: bytes) -> str:\n", " response_json = json.loads(output.read().decode(\"utf-8\"))\n", " return response_json[\"summary\"]\n", "\n", "\n", "content_handler = ContentHandler()\n", "\n", "sm_llm = SagemakerEndpoint(\n", " endpoint_name=endpoint_name, ## add endpoint name for ai21 summary model\n", " region_name=aws_region,\n", " # model_kwargs=parameters,\n", " content_handler=content_handler,\n", ")" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Our prompt has 108 tokens\n" ] } ], "source": [ "num_tokens = sm_llm.get_num_tokens(prompt)\n", "print (f\"Our prompt has {num_tokens} tokens\")" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Philosophy is the systematized study of general and fundamental questions, such as those about existence, reason, knowledge, values, mind, and language.\n" ] } ], "source": [ "output = sm_llm(prompt)\n", "print (output)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "tags": [] }, "outputs": [], "source": [ "prompt = \"\"\"\n", "Write a ~ 1 sentence summary of the following text:\n", "\n", "TEXT:\n", "Philosophy (from Greek: φιλοσοφία, philosophia, 'love of wisdom') \\\n", "is the systematized study of general and fundamental questions, \\\n", "such as those about existence, reason, knowledge, values, mind, and language. \\\n", "Some sources claim the term was coined by Pythagoras (c. 570 – c. 495 BCE), \\\n", "although this theory is disputed by some. Philosophical methods include questioning, \\\n", "critical discussion, rational argument, and systematic presentation.\n", "\"\"\"" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Philosophy is the systematized study of general and fundamental questions, such as those about existence, reason, knowledge, values, mind, and language.\n" ] } ], "source": [ "output = sm_llm(prompt)\n", "print (output)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summarize a couple paragraphs - Prompt Templates\n", "\n", "Prompt templates are a great way to dynamically place text within your prompts. They are like [python f-strings](https://realpython.com/python-f-strings/) but specialized for working with language models.\n", "\n", "We're going to look at 2 short Paul Graham essays" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "tags": [] }, "outputs": [], "source": [ "paul_graham_essays = ['data/PaulGrahamEssaySmall/getideas.txt', 'data/PaulGrahamEssaySmall/noob.txt']\n", "\n", "essays = []\n", "\n", "for file_name in paul_graham_essays:\n", " with open(file_name, 'r') as file:\n", " essays.append(file.read())" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Essay #1: January 2023(Someone fed my essays into GPT to make something that could answer\n", "questions based on them, then asked it where good ideas come from. The\n", "answer was ok, but not what I would have said. This is what I would have said.)The way to get new ideas is to notice anomalies: what seems strange,\n", "\n", "\n", "Essay #2: January 2020When I was young, I thought old people had everything figured out.\n", "Now that I'm old, I know this isn't true.I constantly feel like a noob. It seems like I'm always talking to\n", "some startup working in a new field I know nothing about, or reading\n", "a book about a topic I don't understand well\n", "\n" ] } ], "source": [ "for i, essay in enumerate(essays):\n", " print (f\"Essay #{i+1}: {essay[:300]}\\n\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next let's create a prompt template which will hold our instructions and a placeholder for the essay. In this example we only want a 1 sentence summary to come back" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "tags": [] }, "outputs": [], "source": [ "template = \"\"\"\n", "Write a ~ 50 words summary of the following text:\n", "{essay}\n", "\"\"\"\n", "\n", "prompt = PromptTemplate(\n", " input_variables=[\"essay\"],\n", " template=template\n", ")" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This prompt + essay has 204 tokens\n", "Summary: You can get new ideas by noticing anomalies, and the best place to look for them is at the frontiers of knowledge. Knowledge grows fractally, and these gaps seem obvious, but no one has tried them yet.\n", "\n", "\n", "This prompt + essay has 499 tokens\n", "Summary: The feeling of being a noob is inversely correlated with actual ignorance. If you stay in your home country, you'll feel less of a noob than if you move to Farawavia, where everything works differently, and yet you'll know more.\n", "Our dislike of feeling like a noob is our brain telling us to come on, come on, figure this out. But now that food is plentiful, our dislike of feeling like a noob leads us astray.\n", "\n", "\n" ] } ], "source": [ "for essay in essays:\n", " summary_prompt = prompt.format(essay=essay)\n", " \n", " num_tokens = sm_llm.get_num_tokens(summary_prompt)\n", " print (f\"This prompt + essay has {num_tokens} tokens\")\n", " \n", " summary = sm_llm(summary_prompt)\n", " \n", " print (f\"Summary: {summary.strip()}\")\n", " print (\"\\n\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summarize a couple pages multiple pages - MapReduce\n", "\n", "If you have multiple pages you'd like to summarize, you'll likely run into a token limit. Token limits won't always be a problem, but it is good to know how to handle them if you run into the issue.\n", "\n", "The chain type \"Map Reduce\" is a method that helps with this. You first generate a summary of smaller chunks (that fit within the token limit) and then you get a summary of the summaries.\\\n", "\n", "Check out [this video](https://www.youtube.com/watch?v=f9_BWhCI4Zo) for more information on how chain types work." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "tags": [] }, "outputs": [], "source": [ "from langchain.chains.summarize import load_summarize_chain\n", "from langchain.text_splitter import RecursiveCharacterTextSplitter" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "tags": [] }, "outputs": [], "source": [ "paul_graham_essay = 'data/PaulGrahamEssays/startupideas.txt'\n", "\n", "with open(paul_graham_essay, 'r') as file:\n", " essay = file.read()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Token indices sequence length is longer than the specified maximum sequence length for this model (9568 > 1024). Running this sequence through the model will result in indexing errors\n" ] }, { "data": { "text/plain": [ "9568" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sm_llm.get_num_tokens(essay)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's too many, let's split our text up into chunks so they fit into the prompt limit. I'm going a chunk size of 10,000 characters. \n", "\n", "> You can think of tokens as pieces of words used for natural language processing. For English text, **1 token is approximately 4 characters** or 0.75 words. As a point of reference, the collected works of Shakespeare are about 900,000 words or 1.2M tokens.\n", "\n", "This means the number of tokens we should expect is 10,000 / 4 = ~2,500 token chunks. But this will vary, each body of text/code will be different" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "tags": [] }, "outputs": [], "source": [ "text_splitter = RecursiveCharacterTextSplitter(separators=[\"\\n\\n\", \"\\n\"], chunk_size=10000, chunk_overlap=500)\n", "\n", "docs = text_splitter.create_documents([essay])" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Token indices sequence length is longer than the specified maximum sequence length for this model (2086 > 1024). Running this sequence through the model will result in indexing errors\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Now we have 5 documents and the first one has 2086 tokens\n" ] } ], "source": [ "num_docs = len(docs)\n", "\n", "num_tokens_first_doc = sm_llm.get_num_tokens(docs[0].page_content)\n", "\n", "print (f\"Now we have {num_docs} documents and the first one has {num_tokens_first_doc} tokens\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Great, assuming that number of tokens is consistent in the other docs we should be good to go. Let's use LangChain's [load_summarize_chain](https://python.langchain.com/en/latest/use_cases/summarization.html) method, we will use `refine` chain type for summarization. We first need to initialize our chain" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "tags": [] }, "outputs": [], "source": [ "summary_chain = load_summarize_chain(llm=sm_llm, chain_type='map_reduce',\n", " verbose=True # Set verbose=True if you want to see the prompts being used\n", " )" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new MapReduceDocumentsChain chain...\u001b[0m\n", "Prompt after formatting:\n", "\u001b[32;1m\u001b[1;3mWrite a concise summary of the following:\n", "\n", "\n", "\"Want to start a startup? Get funded by\n", "Y Combinator.\n", "\n", "\n", "\n", "\n", "November 2012The way to get startup ideas is not to try to think of startup\n", "ideas. It's to look for problems, preferably problems you have\n", "yourself.The very best startup ideas tend to have three things in common:\n", "they're something the founders themselves want, that they themselves\n", "can build, and that few others realize are worth doing. Microsoft,\n", "Apple, Yahoo, Google, and Facebook all began this way.\n", "ProblemsWhy is it so important to work on a problem you have? Among other\n", "things, it ensures the problem really exists. It sounds obvious\n", "to say you should only work on problems that exist. And yet by far\n", "the most common mistake startups make is to solve problems no one\n", "has.I made it myself. In 1995 I started a company to put art galleries\n", "online. But galleries didn't want to be online. It's not how the\n", "art business works. So why did I spend 6 months working on this\n", "stupid idea? Because I didn't pay attention to users. I invented\n", "a model of the world that didn't correspond to reality, and worked\n", "from that. I didn't notice my model was wrong until I tried\n", "to convince users to pay for what we'd built. Even then I took\n", "embarrassingly long to catch on. I was attached to my model of the\n", "world, and I'd spent a lot of time on the software. They had to\n", "want it!Why do so many founders build things no one wants? Because they\n", "begin by trying to think of startup ideas. That m.o. is doubly\n", "dangerous: it doesn't merely yield few good ideas; it yields bad\n", "ideas that sound plausible enough to fool you into working on them.At YC we call these \"made-up\" or \"sitcom\" startup ideas. Imagine\n", "one of the characters on a TV show was starting a startup. The\n", "writers would have to invent something for it to do. But coming\n", "up with good startup ideas is hard. It's not something you can do\n", "for the asking. So (unless they got amazingly lucky) the writers\n", "would come up with an idea that sounded plausible, but was actually\n", "bad.For example, a social network for pet owners. It doesn't sound\n", "obviously mistaken. Millions of people have pets. Often they care\n", "a lot about their pets and spend a lot of money on them. Surely\n", "many of these people would like a site where they could talk to\n", "other pet owners. Not all of them perhaps, but if just 2 or 3\n", "percent were regular visitors, you could have millions of users.\n", "You could serve them targeted offers, and maybe charge for premium\n", "features. \n", "[1]The danger of an idea like this is that when you run it by your\n", "friends with pets, they don't say \"I would never use this.\" They\n", "say \"Yeah, maybe I could see using something like that.\" Even when\n", "the startup launches, it will sound plausible to a lot of people.\n", "They don't want to use it themselves, at least not right now, but\n", "they could imagine other people wanting it. Sum that reaction\n", "across the entire population, and you have zero users. \n", "[2]\n", "WellWhen a startup launches, there have to be at least some users who\n", "really need what they're making — not just people who could see\n", "themselves using it one day, but who want it urgently. Usually\n", "this initial group of users is small, for the simple reason that\n", "if there were something that large numbers of people urgently needed\n", "and that could be built with the amount of effort a startup usually\n", "puts into a version one, it would probably already exist. Which\n", "means you have to compromise on one dimension: you can either build\n", "something a large number of people want a small amount, or something\n", "a small number of people want a large amount. Choose the latter.\n", "Not all ideas of that type are good startup ideas, but nearly all\n", "good startup ideas are of that type.Imagine a graph whose x axis represents all the people who might\n", "want what you're making and whose y axis represents how much they\n", "want it. If you invert the scale on the y axis, you can envision\n", "companies as holes. Google is an immense crater: hundreds of\n", "millions of people use it, and they need it a lot. A startup just\n", "starting out can't expect to excavate that much volume. So you\n", "have two choices about the shape of hole you start with. You can\n", "either dig a hole that's broad but shallow, or one that's narrow\n", "and deep, like a well.Made-up startup ideas are usually of the first type. Lots of people\n", "are mildly interested in a social network for pet owners.Nearly all good startup ideas are of the second type. Microsoft\n", "was a well when they made Altair Basic. There were only a couple\n", "thousand Altair owners, but without this software they were programming\n", "in machine language. Thirty years later Facebook had the same\n", "shape. Their first site was exclusively for Harvard students, of\n", "which there are only a few thousand, but those few thousand users\n", "wanted it a lot.When you have an idea for a startup, ask yourself: who wants this\n", "right now? Who wants this so much that they'll use it even when\n", "it's a crappy version one made by a two-person startup they've never\n", "heard of? If you can't answer that, the idea is probably bad. \n", "[3]You don't need the narrowness of the well per se. It's depth you\n", "need; you get narrowness as a byproduct of optimizing for depth\n", "(and speed). But you almost always do get it. In practice the\n", "link between depth and narrowness is so strong that it's a good\n", "sign when you know that an idea will appeal strongly to a specific\n", "group or type of user.But while demand shaped like a well is almost a necessary condition\n", "for a good startup idea, it's not a sufficient one. If Mark\n", "Zuckerberg had built something that could only ever have appealed\n", "to Harvard students, it would not have been a good startup idea.\n", "Facebook was a good idea because it started with a small market\n", "there was a fast path out of. Colleges are similar enough that if\n", "you build a facebook that works at Harvard, it will work at any\n", "college. So you spread rapidly through all the colleges. Once you\n", "have all the college students, you get everyone else simply by\n", "letting them in.Similarly for Microsoft: Basic for the Altair; Basic for other\n", "machines; other languages besides Basic; operating systems;\n", "applications; IPO.\n", "SelfHow do you tell whether there's a path out of an idea? How do you\n", "tell whether something is the germ of a giant company, or just a\n", "niche product? Often you can't. The founders of Airbnb didn't\n", "realize at first how big a market they were tapping. Initially\n", "they had a much narrower idea. They were going to let hosts rent\n", "out space on their floors during conventions. They didn't foresee\n", "the expansion of this idea; it forced itself upon them gradually.\n", "All they knew at first is that they were onto something. That's\n", "probably as much as Bill Gates or Mark Zuckerberg knew at first.Occasionally it's obvious from the beginning when there's a path\n", "out of the initial niche. And sometimes I can see a path that's\n", "not immediately obvious; that's one of our specialties at YC. But\n", "there are limits to how well this can be done, no matter how much\n", "experience you have. The most important thing to understand about\n", "paths out of the initial idea is the meta-fact that these are hard\n", "to see.So if you can't predict whether there's a path out of an idea, how\n", "do you choose between ideas? The truth is disappointing but\n", "interesting: if you're the right sort of person, you have the right\n", "sort of hunches. If you're at the leading edge of a field that's\n", "changing fast, when you have a hunch that something is worth doing,\n", "you're more likely to be right.In Zen and the Art of Motorcycle Maintenance, Robert Pirsig says:\n", "\n", " You want to know how to paint a perfect painting? It's easy. Make\n", " yourself perfect and then just paint naturally.\n", "\n", "I've wondered about that passage since I read it in high school.\n", "I'm not sure how useful his advice is for painting specifically,\n", "but it fits this situation well. Empirically, the way to have good\n", "startup ideas is to become the sort of person who has them.Being at the leading edge of a field doesn't mean you have to be\n", "one of the people pushing it forward. You can also be at the leading\n", "edge as a user. It was not so much because he was a programmer\n", "that Facebook seemed a good idea to Mark Zuckerberg as because he\n", "used computers so much. If you'd asked most 40 year olds in 2004\n", "whether they'd like to publish their lives semi-publicly on the\n", "Internet, they'd have been horrified at the idea. But Mark already\n", "lived online; to him it seemed natural.Paul Buchheit says that people at the leading edge of a rapidly\n", "changing field \"live in the future.\" Combine that with Pirsig and\n", "you get:\n", "\n", " Live in the future, then build what's missing.\"\n", "\n", "\n", "CONCISE SUMMARY:\u001b[0m\n", "Prompt after formatting:\n", "\u001b[32;1m\u001b[1;3mWrite a concise summary of the following:\n", "\n", "\n", "\"Live in the future, then build what's missing.\n", "\n", "That describes the way many if not most of the biggest startups got\n", "started. Neither Apple nor Yahoo nor Google nor Facebook were even\n", "supposed to be companies at first. They grew out of things their\n", "founders built because there seemed a gap in the world.If you look at the way successful founders have had their ideas,\n", "it's generally the result of some external stimulus hitting a\n", "prepared mind. Bill Gates and Paul Allen hear about the Altair and\n", "think \"I bet we could write a Basic interpreter for it.\" Drew Houston\n", "realizes he's forgotten his USB stick and thinks \"I really need to\n", "make my files live online.\" Lots of people heard about the Altair.\n", "Lots forgot USB sticks. The reason those stimuli caused those\n", "founders to start companies was that their experiences had prepared\n", "them to notice the opportunities they represented.The verb you want to be using with respect to startup ideas is not\n", "\"think up\" but \"notice.\" At YC we call ideas that grow naturally\n", "out of the founders' own experiences \"organic\" startup ideas. The\n", "most successful startups almost all begin this way.That may not have been what you wanted to hear. You may have\n", "expected recipes for coming up with startup ideas, and instead I'm\n", "telling you that the key is to have a mind that's prepared in the\n", "right way. But disappointing though it may be, this is the truth.\n", "And it is a recipe of a sort, just one that in the worst case takes\n", "a year rather than a weekend.If you're not at the leading edge of some rapidly changing field,\n", "you can get to one. For example, anyone reasonably smart can\n", "probably get to an edge of programming (e.g. building mobile apps)\n", "in a year. Since a successful startup will consume at least 3-5\n", "years of your life, a year's preparation would be a reasonable\n", "investment. Especially if you're also looking for a cofounder.\n", "[4]You don't have to learn programming to be at the leading edge of a\n", "domain that's changing fast. Other domains change fast. But while\n", "learning to hack is not necessary, it is for the forseeable future\n", "sufficient. As Marc Andreessen put it, software is eating the world,\n", "and this trend has decades left to run.Knowing how to hack also means that when you have ideas, you'll be\n", "able to implement them. That's not absolutely necessary (Jeff Bezos\n", "couldn't) but it's an advantage. It's a big advantage, when you're\n", "considering an idea like putting a college facebook online, if\n", "instead of merely thinking \"That's an interesting idea,\" you can\n", "think instead \"That's an interesting idea. I'll try building an\n", "initial version tonight.\" It's even better when you're both a\n", "programmer and the target user, because then the cycle of generating\n", "new versions and testing them on users can happen inside one head.\n", "NoticingOnce you're living in the future in some respect, the way to notice\n", "startup ideas is to look for things that seem to be missing. If\n", "you're really at the leading edge of a rapidly changing field, there\n", "will be things that are obviously missing. What won't be obvious\n", "is that they're startup ideas. So if you want to find startup\n", "ideas, don't merely turn on the filter \"What's missing?\" Also turn\n", "off every other filter, particularly \"Could this be a big company?\"\n", "There's plenty of time to apply that test later. But if you're\n", "thinking about that initially, it may not only filter out lots\n", "of good ideas, but also cause you to focus on bad ones.Most things that are missing will take some time to see. You almost\n", "have to trick yourself into seeing the ideas around you.But you know the ideas are out there. This is not one of those\n", "problems where there might not be an answer. It's impossibly\n", "unlikely that this is the exact moment when technological progress\n", "stops. You can be sure people are going to build things in the\n", "next few years that will make you think \"What did I do before x?\"And when these problems get solved, they will probably seem flamingly\n", "obvious in retrospect. What you need to do is turn off the filters\n", "that usually prevent you from seeing them. The most powerful is\n", "simply taking the current state of the world for granted. Even the\n", "most radically open-minded of us mostly do that. You couldn't get\n", "from your bed to the front door if you stopped to question everything.But if you're looking for startup ideas you can sacrifice some of\n", "the efficiency of taking the status quo for granted and start to\n", "question things. Why is your inbox overflowing? Because you get\n", "a lot of email, or because it's hard to get email out of your inbox?\n", "Why do you get so much email? What problems are people trying to\n", "solve by sending you email? Are there better ways to solve them?\n", "And why is it hard to get emails out of your inbox? Why do you\n", "keep emails around after you've read them? Is an inbox the optimal\n", "tool for that?Pay particular attention to things that chafe you. The advantage\n", "of taking the status quo for granted is not just that it makes life\n", "(locally) more efficient, but also that it makes life more tolerable.\n", "If you knew about all the things we'll get in the next 50 years but\n", "don't have yet, you'd find present day life pretty constraining,\n", "just as someone from the present would if they were sent back 50\n", "years in a time machine. When something annoys you, it could be\n", "because you're living in the future.When you find the right sort of problem, you should probably be\n", "able to describe it as obvious, at least to you. When we started\n", "Viaweb, all the online stores were built by hand, by web designers\n", "making individual HTML pages. It was obvious to us as programmers\n", "that these sites would have to be generated by software.\n", "[5]Which means, strangely enough, that coming up with startup ideas\n", "is a question of seeing the obvious. That suggests how weird this\n", "process is: you're trying to see things that are obvious, and yet\n", "that you hadn't seen.Since what you need to do here is loosen up your own mind, it may\n", "be best not to make too much of a direct frontal attack on the\n", "problem — i.e. to sit down and try to think of ideas. The best\n", "plan may be just to keep a background process running, looking for\n", "things that seem to be missing. Work on hard problems, driven\n", "mainly by curiosity, but have a second self watching over your\n", "shoulder, taking note of gaps and anomalies. \n", "[6]Give yourself some time. You have a lot of control over the rate\n", "at which you turn yours into a prepared mind, but you have less\n", "control over the stimuli that spark ideas when they hit it. If\n", "Bill Gates and Paul Allen had constrained themselves to come up\n", "with a startup idea in one month, what if they'd chosen a month\n", "before the Altair appeared? They probably would have worked on a\n", "less promising idea. Drew Houston did work on a less promising\n", "idea before Dropbox: an SAT prep startup. But Dropbox was a much\n", "better idea, both in the absolute sense and also as a match for his\n", "skills.\n", "[7]A good way to trick yourself into noticing ideas is to work on\n", "projects that seem like they'd be cool. If you do that, you'll\n", "naturally tend to build things that are missing. It wouldn't seem\n", "as interesting to build something that already existed.Just as trying to think up startup ideas tends to produce bad ones,\n", "working on things that could be dismissed as \"toys\" often produces\n", "good ones. When something is described as a toy, that means it has\n", "everything an idea needs except being important. It's cool; users\n", "love it; it just doesn't matter. But if you're living in the future\n", "and you build something cool that users love, it may matter more\n", "than outsiders think. Microcomputers seemed like toys when Apple\n", "and Microsoft started working on them. I'm old enough to remember\n", "that era; the usual term for people with their own microcomputers\n", "was \"hobbyists.\" BackRub seemed like an inconsequential science\n", "project. The Facebook was just a way for undergrads to stalk one\n", "another.At YC we're excited when we meet startups working on things that\n", "we could imagine know-it-alls on forums dismissing as toys. To us\n", "that's positive evidence an idea is good.If you can afford to take a long view (and arguably you can't afford\n", "not to), you can turn \"Live in the future and build what's missing\"\n", "into something even better:\n", "\n", " Live in the future and build what seems interesting.\"\n", "\n", "\n", "CONCISE SUMMARY:\u001b[0m\n", "Prompt after formatting:\n", "\u001b[32;1m\u001b[1;3mWrite a concise summary of the following:\n", "\n", "\n", "\"SchoolThat's what I'd advise college students to do, rather than trying\n", "to learn about \"entrepreneurship.\" \"Entrepreneurship\" is something\n", "you learn best by doing it. The examples of the most successful\n", "founders make that clear. What you should be spending your time\n", "on in college is ratcheting yourself into the future. College is\n", "an incomparable opportunity to do that. What a waste to sacrifice\n", "an opportunity to solve the hard part of starting a startup — becoming \n", "the sort of person who can have organic startup ideas — by \n", "spending time learning about the easy part. Especially since\n", "you won't even really learn about it, any more than you'd learn\n", "about sex in a class. All you'll learn is the words for things.The clash of domains is a particularly fruitful source of ideas.\n", "If you know a lot about programming and you start learning about\n", "some other field, you'll probably see problems that software could\n", "solve. In fact, you're doubly likely to find good problems in\n", "another domain: (a) the inhabitants of that domain are not as likely\n", "as software people to have already solved their problems with\n", "software, and (b) since you come into the new domain totally ignorant,\n", "you don't even know what the status quo is to take it for granted.So if you're a CS major and you want to start a startup, instead\n", "of taking a class on entrepreneurship you're better off taking a\n", "class on, say, genetics. Or better still, go work for a biotech\n", "company. CS majors normally get summer jobs at computer hardware\n", "or software companies. But if you want to find startup ideas, you\n", "might do better to get a summer job in some unrelated field. \n", "[8]Or don't take any extra classes, and just build things. It's no\n", "coincidence that Microsoft and Facebook both got started in January.\n", "At Harvard that is (or was) Reading Period, when students have no\n", "classes to attend because they're supposed to be studying for finals.\n", "[9]But don't feel like you have to build things that will become startups. That's\n", "premature optimization. Just build things. Preferably with other\n", "students. It's not just the classes that make a university such a\n", "good place to crank oneself into the future. You're also surrounded\n", "by other people trying to do the same thing. If you work together\n", "with them on projects, you'll end up producing not just organic\n", "ideas, but organic ideas with organic founding teams — and that,\n", "empirically, is the best combination.Beware of research. If an undergrad writes something all his friends\n", "start using, it's quite likely to represent a good startup idea.\n", "Whereas a PhD dissertation is extremely unlikely to. For some\n", "reason, the more a project has to count as research, the less likely\n", "it is to be something that could be turned into a startup.\n", "[10]\n", "I think the reason is that the subset of ideas that count as research\n", "is so narrow that it's unlikely that a project that satisfied that\n", "constraint would also satisfy the orthogonal constraint of solving\n", "users' problems. Whereas when students (or professors) build\n", "something as a side-project, they automatically gravitate toward\n", "solving users' problems — perhaps even with an additional energy\n", "that comes from being freed from the constraints of research.\n", "CompetitionBecause a good idea should seem obvious, when you have one you'll\n", "tend to feel that you're late. Don't let that deter you. Worrying\n", "that you're late is one of the signs of a good idea. Ten minutes\n", "of searching the web will usually settle the question. Even if you\n", "find someone else working on the same thing, you're probably not\n", "too late. It's exceptionally rare for startups to be killed by\n", "competitors — so rare that you can almost discount the possibility.\n", "So unless you discover a competitor with the sort of lock-in that\n", "would prevent users from choosing you, don't discard the idea.If you're uncertain, ask users. The question of whether you're too\n", "late is subsumed by the question of whether anyone urgently needs\n", "what you plan to make. If you have something that no competitor\n", "does and that some subset of users urgently need, you have a\n", "beachhead. \n", "[11]The question then is whether that beachhead is big enough. Or more\n", "importantly, who's in it: if the beachhead consists of people doing\n", "something lots more people will be doing in the future, then it's\n", "probably big enough no matter how small it is. For example, if\n", "you're building something differentiated from competitors by the\n", "fact that it works on phones, but it only works on the newest phones,\n", "that's probably a big enough beachhead.Err on the side of doing things where you'll face competitors.\n", "Inexperienced founders usually give competitors more credit than\n", "they deserve. Whether you succeed depends far more on you than on\n", "your competitors. So better a good idea with competitors than a\n", "bad one without.You don't need to worry about entering a \"crowded market\" so long\n", "as you have a thesis about what everyone else in it is overlooking.\n", "In fact that's a very promising starting point. Google was that\n", "type of idea. Your thesis has to be more precise than \"we're going\n", "to make an x that doesn't suck\" though. You have to be able to\n", "phrase it in terms of something the incumbents are overlooking.\n", "Best of all is when you can say that they didn't have the courage\n", "of their convictions, and that your plan is what they'd have done\n", "if they'd followed through on their own insights. Google was that\n", "type of idea too. The search engines that preceded them shied away\n", "from the most radical implications of what they were doing — particularly \n", "that the better a job they did, the faster users would\n", "leave.A crowded market is actually a good sign, because it means both\n", "that there's demand and that none of the existing solutions are\n", "good enough. A startup can't hope to enter a market that's obviously\n", "big and yet in which they have no competitors. So any startup that\n", "succeeds is either going to be entering a market with existing\n", "competitors, but armed with some secret weapon that will get them\n", "all the users (like Google), or entering a market that looks small\n", "but which will turn out to be big (like Microsoft). \n", "[12]\n", "FiltersThere are two more filters you'll need to turn off if you want to\n", "notice startup ideas: the unsexy filter and the schlep filter.Most programmers wish they could start a startup by just writing\n", "some brilliant code, pushing it to a server, and having users pay\n", "them lots of money. They'd prefer not to deal with tedious problems\n", "or get involved in messy ways with the real world. Which is a\n", "reasonable preference, because such things slow you down. But this\n", "preference is so widespread that the space of convenient startup\n", "ideas has been stripped pretty clean. If you let your mind wander\n", "a few blocks down the street to the messy, tedious ideas, you'll\n", "find valuable ones just sitting there waiting to be implemented.The schlep filter is so dangerous that I wrote a separate essay\n", "about the condition it induces, which I called \n", "schlep blindness.\n", "I gave Stripe as an example of a startup that benefited from turning\n", "off this filter, and a pretty striking example it is. Thousands\n", "of programmers were in a position to see this idea; thousands of\n", "programmers knew how painful it was to process payments before\n", "Stripe. But when they looked for startup ideas they didn't see\n", "this one, because unconsciously they shrank from having to deal\n", "with payments. And dealing with payments is a schlep for Stripe,\n", "but not an intolerable one. In fact they might have had net less\n", "pain; because the fear of dealing with payments kept most people\n", "away from this idea, Stripe has had comparatively smooth sailing\n", "in other areas that are sometimes painful, like user acquisition.\n", "They didn't have to try very hard to make themselves heard by users,\n", "because users were desperately waiting for what they were building.The unsexy filter is similar to the schlep filter, except it keeps\n", "you from working on problems you despise rather than ones you fear.\n", "We overcame this one to work on Viaweb. There were interesting\n", "things about the architecture of our software, but we weren't\n", "interested in ecommerce per se. We could see the problem was one\n", "that needed to be solved though.Turning off the schlep filter is more important than turning off\n", "the unsexy filter, because the schlep filter is more likely to be\n", "an illusion. And even to the degree it isn't, it's a worse form\n", "of self-indulgence. Starting a successful startup is going to be\n", "fairly laborious no matter what. Even if the product doesn't entail\n", "a lot of schleps, you'll still have plenty dealing with investors,\n", "hiring and firing people, and so on. So if there's some idea you\n", "think would be cool but you're kept away from by fear of the schleps\n", "involved, don't worry: any sufficiently good idea will have as many.The unsexy filter, while still a source of error, is not as entirely\n", "useless as the schlep filter. If you're at the leading edge of a\n", "field that's changing rapidly, your ideas about what's sexy will\n", "be somewhat correlated with what's valuable in practice. Particularly\n", "as you get older and more experienced. Plus if you find an idea\n", "sexy, you'll work on it more enthusiastically. \n", "[13]\n", "RecipesWhile the best way to discover startup ideas is to become the sort\n", "of person who has them and then build whatever interests you,\n", "sometimes you don't have that luxury. Sometimes you need an idea\n", "now. For example, if you're working on a startup and your initial\n", "idea turns out to be bad.For the rest of this essay I'll talk about tricks for coming up\n", "with startup ideas on demand. Although empirically you're better\n", "off using the organic strategy, you could succeed this way. You\n", "just have to be more disciplined. When you use the organic method,\n", "you don't even notice an idea unless it's evidence that something\n", "is truly missing. But when you make a conscious effort to think\n", "of startup ideas, you have to replace this natural constraint with\"\n", "\n", "\n", "CONCISE SUMMARY:\u001b[0m\n", "Prompt after formatting:\n", "\u001b[32;1m\u001b[1;3mWrite a concise summary of the following:\n", "\n", "\n", "\"idea turns out to be bad.For the rest of this essay I'll talk about tricks for coming up\n", "with startup ideas on demand. Although empirically you're better\n", "off using the organic strategy, you could succeed this way. You\n", "just have to be more disciplined. When you use the organic method,\n", "you don't even notice an idea unless it's evidence that something\n", "is truly missing. But when you make a conscious effort to think\n", "of startup ideas, you have to replace this natural constraint with\n", "self-discipline. You'll see a lot more ideas, most of them bad,\n", "so you need to be able to filter them.One of the biggest dangers of not using the organic method is the\n", "example of the organic method. Organic ideas feel like inspirations.\n", "There are a lot of stories about successful startups that began\n", "when the founders had what seemed a crazy idea but \"just knew\" it\n", "was promising. When you feel that about an idea you've had while\n", "trying to come up with startup ideas, you're probably mistaken.When searching for ideas, look in areas where you have some expertise.\n", "If you're a database expert, don't build a chat app for teenagers\n", "(unless you're also a teenager). Maybe it's a good idea, but you\n", "can't trust your judgment about that, so ignore it. There have to\n", "be other ideas that involve databases, and whose quality you can\n", "judge. Do you find it hard to come up with good ideas involving\n", "databases? That's because your expertise raises your standards.\n", "Your ideas about chat apps are just as bad, but you're giving\n", "yourself a Dunning-Kruger pass in that domain.The place to start looking for ideas is things you need. There\n", "must be things you need.\n", "[14]One good trick is to ask yourself whether in your previous job you\n", "ever found yourself saying \"Why doesn't someone make x? If someone\n", "made x we'd buy it in a second.\" If you can think of any x people\n", "said that about, you probably have an idea. You know there's demand,\n", "and people don't say that about things that are impossible to build.More generally, try asking yourself whether there's something unusual\n", "about you that makes your needs different from most other people's.\n", "You're probably not the only one. It's especially good if you're\n", "different in a way people will increasingly be.If you're changing ideas, one unusual thing about you is the idea\n", "you'd previously been working on. Did you discover any needs while\n", "working on it? Several well-known startups began this way. Hotmail\n", "began as something its founders wrote to talk about their previous\n", "startup idea while they were working at their day jobs. \n", "[15]A particularly promising way to be unusual is to be young. Some\n", "of the most valuable new ideas take root first among people in their\n", "teens and early twenties. And while young founders are at a\n", "disadvantage in some respects, they're the only ones who really\n", "understand their peers. It would have been very hard for someone\n", "who wasn't a college student to start Facebook. So if you're a\n", "young founder (under 23 say), are there things you and your friends\n", "would like to do that current technology won't let you?The next best thing to an unmet need of your own is an unmet need\n", "of someone else. Try talking to everyone you can about the gaps\n", "they find in the world. What's missing? What would they like to\n", "do that they can't? What's tedious or annoying, particularly in\n", "their work? Let the conversation get general; don't be trying too\n", "hard to find startup ideas. You're just looking for something to\n", "spark a thought. Maybe you'll notice a problem they didn't consciously\n", "realize they had, because you know how to solve it.When you find an unmet need that isn't your own, it may be somewhat\n", "blurry at first. The person who needs something may not know exactly\n", "what they need. In that case I often recommend that founders act\n", "like consultants — that they do what they'd do if they'd been\n", "retained to solve the problems of this one user. People's problems\n", "are similar enough that nearly all the code you write this way will\n", "be reusable, and whatever isn't will be a small price to start out\n", "certain that you've reached the bottom of the well.\n", "[16]One way to ensure you do a good job solving other people's problems\n", "is to make them your own. When Rajat Suri of E la Carte decided\n", "to write software for restaurants, he got a job as a waiter to learn\n", "how restaurants worked. That may seem like taking things to extremes,\n", "but startups are extreme. We love it when founders do such things.In fact, one strategy I recommend to people who need a new idea is\n", "not merely to turn off their schlep and unsexy filters, but to seek\n", "out ideas that are unsexy or involve schleps. Don't try to start\n", "Twitter. Those ideas are so rare that you can't find them by looking\n", "for them. Make something unsexy that people will pay you for.A good trick for bypassing the schlep and to some extent the unsexy\n", "filter is to ask what you wish someone else would build, so that\n", "you could use it. What would you pay for right now?Since startups often garbage-collect broken companies and industries,\n", "it can be a good trick to look for those that are dying, or deserve\n", "to, and try to imagine what kind of company would profit from their\n", "demise. For example, journalism is in free fall at the moment.\n", "But there may still be money to be made from something like journalism.\n", "What sort of company might cause people in the future to say \"this\n", "replaced journalism\" on some axis?But imagine asking that in the future, not now. When one company\n", "or industry replaces another, it usually comes in from the side.\n", "So don't look for a replacement for x; look for something that\n", "people will later say turned out to be a replacement for x. And\n", "be imaginative about the axis along which the replacement occurs.\n", "Traditional journalism, for example, is a way for readers to get\n", "information and to kill time, a way for writers to make money and\n", "to get attention, and a vehicle for several different types of\n", "advertising. It could be replaced on any of these axes (it has\n", "already started to be on most).When startups consume incumbents, they usually start by serving\n", "some small but important market that the big players ignore. It's\n", "particularly good if there's an admixture of disdain in the big\n", "players' attitude, because that often misleads them. For example,\n", "after Steve Wozniak built the computer that became the Apple I, he\n", "felt obliged to give his then-employer Hewlett-Packard the option\n", "to produce it. Fortunately for him, they turned it down, and one\n", "of the reasons they did was that it used a TV for a monitor, which\n", "seemed intolerably déclassé to a high-end hardware company like HP\n", "was at the time. \n", "[17]Are there groups of \n", "scruffy \n", "but sophisticated users like the early\n", "microcomputer \"hobbyists\" that are currently being ignored by the\n", "big players? A startup with its sights set on bigger things can\n", "often capture a small market easily by expending an effort that\n", "wouldn't be justified by that market alone.Similarly, since the most successful startups generally ride some\n", "wave bigger than themselves, it could be a good trick to look for\n", "waves and ask how one could benefit from them. The prices of gene\n", "sequencing and 3D printing are both experiencing Moore's Law-like\n", "declines. What new things will we be able to do in the new world\n", "we'll have in a few years? What are we unconsciously ruling out\n", "as impossible that will soon be possible?\n", "OrganicBut talking about looking explicitly for waves makes it clear that\n", "such recipes are plan B for getting startup ideas. Looking for\n", "waves is essentially a way to simulate the organic method. If\n", "you're at the leading edge of some rapidly changing field, you don't\n", "have to look for waves; you are the wave.Finding startup ideas is a subtle business, and that's why most\n", "people who try fail so miserably. It doesn't work well simply to\n", "try to think of startup ideas. If you do that, you get bad ones\n", "that sound dangerously plausible. The best approach is more indirect:\n", "if you have the right sort of background, good startup ideas will\n", "seem obvious to you. But even then, not immediately. It takes\n", "time to come across situations where you notice something missing.\n", "And often these gaps won't seem to be ideas for companies, just\n", "things that would be interesting to build. Which is why it's good\n", "to have the time and the inclination to build things just because\n", "they're interesting.Live in the future and build what seems interesting. Strange as\n", "it sounds, that's the real recipe.\n", "Notes[1]\n", "This form of bad idea has been around as long as the web. It\n", "was common in the 1990s, except then people who had it used to say\n", "they were going to create a portal for x instead of a social network\n", "for x. Structurally the idea is stone soup: you post a sign saying\n", "\"this is the place for people interested in x,\" and all those people\n", "show up and you make money from them. What lures founders into\n", "this sort of idea are statistics about the millions of people who\n", "might be interested in each type of x. What they forget is that\n", "any given person might have 20 affinities by this standard, and no\n", "one is going to visit 20 different communities regularly.[2]\n", "I'm not saying, incidentally, that I know for sure a social\n", "network for pet owners is a bad idea. I know it's a bad idea the\n", "way I know randomly generated DNA would not produce a viable organism.\n", "The set of plausible sounding startup ideas is many times larger\n", "than the set of good ones, and many of the good ones don't even\n", "sound that plausible. So if all you know about a startup idea is\n", "that it sounds plausible, you have to assume it's bad.[3]\n", "More precisely, the users' need has to give them sufficient\n", "activation energy to start using whatever you make, which can vary\n", "a lot. For example, the activation energy for enterprise software\n", "sold through traditional channels is very high, so you'd have to\n", "be a lot better to get users to switch. Whereas the activation\n", "energy required to switch to a new search engine is low. Which in\"\n", "\n", "\n", "CONCISE SUMMARY:\u001b[0m\n", "Prompt after formatting:\n", "\u001b[32;1m\u001b[1;3mWrite a concise summary of the following:\n", "\n", "\n", "\"that it sounds plausible, you have to assume it's bad.[3]\n", "More precisely, the users' need has to give them sufficient\n", "activation energy to start using whatever you make, which can vary\n", "a lot. For example, the activation energy for enterprise software\n", "sold through traditional channels is very high, so you'd have to\n", "be a lot better to get users to switch. Whereas the activation\n", "energy required to switch to a new search engine is low. Which in\n", "turn is why search engines are so much better than enterprise\n", "software.[4]\n", "This gets harder as you get older. While the space of ideas\n", "doesn't have dangerous local maxima, the space of careers does.\n", "There are fairly high walls between most of the paths people take\n", "through life, and the older you get, the higher the walls become.[5]\n", "It was also obvious to us that the web was going to be a big\n", "deal. Few non-programmers grasped that in 1995, but the programmers\n", "had seen what GUIs had done for desktop computers.[6]\n", "Maybe it would work to have this second self keep a journal,\n", "and each night to make a brief entry listing the gaps and anomalies\n", "you'd noticed that day. Not startup ideas, just the raw gaps and\n", "anomalies.[7]\n", "Sam Altman points out that taking time to come up with an\n", "idea is not merely a better strategy in an absolute sense, but also\n", "like an undervalued stock in that so few founders do it.There's comparatively little competition for the best ideas, because\n", "few founders are willing to put in the time required to notice them.\n", "Whereas there is a great deal of competition for mediocre ideas,\n", "because when people make up startup ideas, they tend to make up the\n", "same ones.[8]\n", "For the computer hardware and software companies, summer jobs\n", "are the first phase of the recruiting funnel. But if you're good\n", "you can skip the first phase. If you're good you'll have no trouble\n", "getting hired by these companies when you graduate, regardless of\n", "how you spent your summers.[9]\n", "The empirical evidence suggests that if colleges want to help\n", "their students start startups, the best thing they can do is leave\n", "them alone in the right way.[10]\n", "I'm speaking here of IT startups; in biotech things are different.[11]\n", "This is an instance of a more general rule: focus on users,\n", "not competitors. The most important information about competitors\n", "is what you learn via users anyway.[12]\n", "In practice most successful startups have elements of both.\n", "And you can describe each strategy in terms of the other by adjusting\n", "the boundaries of what you call the market. But it's useful to\n", "consider these two ideas separately.[13]\n", "I almost hesitate to raise that point though. Startups are\n", "businesses; the point of a business is to make money; and with that\n", "additional constraint, you can't expect you'll be able to spend all\n", "your time working on what interests you most.[14]\n", "The need has to be a strong one. You can retroactively\n", "describe any made-up idea as something you need. But do you really\n", "need that recipe site or local event aggregator as much as Drew\n", "Houston needed Dropbox, or Brian Chesky and Joe Gebbia needed Airbnb?Quite often at YC I find myself asking founders \"Would you use this\n", "thing yourself, if you hadn't written it?\" and you'd be surprised\n", "how often the answer is no.[15]\n", "Paul Buchheit points out that trying to sell something bad\n", "can be a source of better ideas:\"The best technique I've found for dealing with YC companies that\n", "have bad ideas is to tell them to go sell the product ASAP (before\n", "wasting time building it). Not only do they learn that nobody\n", "wants what they are building, they very often come back with a\n", "real idea that they discovered in the process of trying to sell\n", "the bad idea.\"[16]\n", "Here's a recipe that might produce the next Facebook, if\n", "you're college students. If you have a connection to one of the\n", "more powerful sororities at your school, approach the queen bees\n", "thereof and offer to be their personal IT consultants, building\n", "anything they could imagine needing in their social lives that\n", "didn't already exist. Anything that got built this way would be\n", "very promising, because such users are not just the most demanding\n", "but also the perfect point to spread from.I have no idea whether this would work.[17]\n", "And the reason it used a TV for a monitor is that Steve Wozniak\n", "started out by solving his own problems. He, like most of his\n", "peers, couldn't afford a monitor.Thanks to Sam Altman, Mike Arrington, Paul Buchheit, John Collison,\n", "Patrick Collison, Garry Tan, and Harj Taggar for reading drafts of\n", "this, and Marc Andreessen, Joe Gebbia, Reid Hoffman, Shel Kaphan,\n", "Mike Moritz and Kevin Systrom for answering my questions about\n", "startup history.\"\n", "\n", "\n", "CONCISE SUMMARY:\u001b[0m\n", "\n", "\n", "\u001b[1m> Entering new StuffDocumentsChain chain...\u001b[0m\n", "\n", "\n", "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n", "Prompt after formatting:\n", "\u001b[32;1m\u001b[1;3mWrite a concise summary of the following:\n", "\n", "\n", "\"In 1995 I started a company to put art galleries online, but the galleries didn't want to be online. Because I didn't pay attention to users, my model of the world was wrong.\n", "At YC we call these ideas \"made-up\" or \"sitcom\" startup ideas, because they sound plausible, but are actually bad. If just 2 or 3 percent of users were regular visitors, you could have millions of users, and maybe charge for premium features.\n", "You can envision companies as holes. Made-up startup ideas are usually of the broad but shallow type, while good startup ideas are narrow and deep.\n", "When you have an idea for a startup, ask yourself who wants this right now?. You almost always get narrowness as a byproduct of optimizing for depth.\n", "If you can't predict whether there's a path out of an idea, how do you choose between ideas?\n", "If you want to have good startup ideas, become the sort of person who has them.\n", "You can also be at the leading edge as a user.\n", "\n", "Many startups got started because the founders saw a gap in the market and built what was missing.\n", "Startup ideas should be noticed, not thought up. Anyone reasonably smart can get to an edge of programming in a year.\n", "You don't have to learn programming to be at the leading edge of a domain that's changing fast. Knowing how to hack also means that you'll be able to implement your ideas.\n", "Most things that are missing will take some time to see, so turn off the filters that prevent you from seeing them.\n", "When you find the right sort of problem, it should be obvious, at least to you, why your inbox is overflowing.\n", "Coming up with startup ideas is a question of seeing the obvious, giving yourself some time, and working on projects that seem like they'd be cool.\n", "At YC, we're excited to meet startups working on innovative ideas.\n", "\n", "Write a concise summary of the following: \"School\" is what college students should be spending their time on, rather than learning about \"entrepreneurship\".\n", "CompetitionBecause a good idea should seem obvious, you'll tend to feel that you're late when you introduce it. Err on the side of doing things where you'll face competitors.\n", "There are two more filters you'll need to turn off if you want to notice startup ideas: the unsexy filter and the schlep filter.\n", "\n", "If you're searching for startup ideas, look in areas where you have some expertise, and ask yourself whether you ever found yourself saying something unusual.\n", "Traditional journalism is a way for readers to get information and to kill time, but startups can capture a small market easily.\n", "Finding startup ideas is a subtle business, and the best approach is more indirect.\n", "\n", "As you get older, it gets harder to come up with an idea, because the space of ideas doesn't have dangerous local maxima, but the space of careers does. Maybe it would work to have a second self keep a journal, and each night to make a brief entry listing the gaps and anomalies you noticed.\n", "If you're good, you can skip the first phase of the recruiting funnel for computer hardware and software companies, and you'll have no trouble getting hired by these companies when you graduate, regardless of how you spent your summers.\n", "Startups are businesses, and the point of a business is to make money. You can't expect to spend all your time working on what interests you most, and the need has to be a strong one.\n", "If you're college students and have a connection to a powerful sorority, offer to be their personal IT consultants and build anything they could imagine needing in their social lives that didn't already exist.\n", "And the reason it used a TV for a monitor is that Steve Wozniak\n", "started out by solving his own problems.\n", "\n", "He, like most of his\n", "peers, couldn't afford a monitor.\n", "Marc Andreessen, Joe Gebbia, Reid Hoffman, Shel Kaphan, Mike Moritz and Kevin Systrom answered my questions about startup history.\"\n", "\n", "\n", "CONCISE SUMMARY:\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n" ] } ], "source": [ "output = summary_chain.run(docs)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "- In 1995 I started a company to put art galleries online, but the galleries didn't want to be online.\n", "- If you want to have good startup ideas, become a person who has them.\n", "- Coming up with startup ideas is a question of seeing the obvious, and asking yourself whether you ever found yourself saying something unusual.\n", "- As you get older, it gets harder to come up with an idea. Maybe a second self could keep a journal.\n", "- Marc Andreessen, Joe Gebbia, Reid Hoffman.\n" ] } ], "source": [ "summaries = output.split('\\n')\n", "for summary in summaries: \n", " print('- '+summary)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This summary is a great start, but lets modify to get only the key points in the summary.\n", "\n", "In order to do this we will use custom promopts (like we did above) to instruct the model on what we need. Please note that the prompts format that is used in the notebook is based on flan t5, taken from this [source.](https://huggingface.co/jordiclive/flan-t5-11b-summarizer-filtered?text=The+tower+is+324+metres+%281%2C063+ft%29+tall%2C+about+the+same+height+as+an+81-storey+building%2C+and+the+tallest+structure+in+Paris.+Its+base+is+square%2C+measuring+125+metres+%28410+ft%29+on+each+side.+During+its+construction%2C+the+Eiffel+Tower+surpassed+the+Washington+Monument+to+become+the+tallest+man-made+structure+in+the+world%2C+a+title+it+held+for+41+years+until+the+Chrysler+Building+in+New+York+City+was+finished+in+1930.+It+was+the+first+structure+to+reach+a+height+of+300+metres.+Due+to+the+addition+of+a+broadcasting+aerial+at+the+top+of+the+tower+in+1957%2C+it+is+now+taller+than+the+Chrysler+Building+by+5.2+metres+%2817+ft%29.+Excluding+transmitters%2C+the+Eiffel+Tower+is+the+second+tallest+free-standing+structure+in+France+after+the+Millau+Viaduct)\n", "\n", "The map_prompt is going to stay the same (just showing it for clarity), but I'll edit the combine_prompt." ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "tags": [] }, "outputs": [], "source": [ "map_prompt = \"\"\"\n", "Write a ~ 500 word summary of the following text:\n", "\"{text}\"\n", "\"\"\"\n", "map_prompt_template = PromptTemplate(template=map_prompt, input_variables=[\"text\"])" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "tags": [] }, "outputs": [], "source": [ "combine_prompt = \"\"\"\n", "Cover only the key points of the text.\n", "{text}\n", "\"\"\"\n", "combine_prompt_template = PromptTemplate(template=combine_prompt, input_variables=[\"text\"])" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "tags": [] }, "outputs": [], "source": [ "summary_chain_key_points = load_summarize_chain(llm=sm_llm,\n", " chain_type='map_reduce',\n", " map_prompt=map_prompt_template,\n", " combine_prompt=combine_prompt_template,\n", " # verbose=True\n", " )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Instead of summarizing all the 30 split documents (chunks), I am using only 15 of them to save time as it can take few minutes and does not run out of memory on the notebook instance." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "tags": [] }, "outputs": [], "source": [ "output_key_points = summary_chain_key_points.run(docs)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "- In 1995 I started a company to put art galleries online. I used recipes to come up with startup ideas on demand.\n", "- When searching for ideas, look in areas where you have some expertise and talk to everyone you can about the gaps they find in the world.\n", "- Startups are businesses, so offer to be their personal IT consultants.\n" ] } ], "source": [ "summaries = output_key_points.split('\\n')\n", "for summary in summaries: \n", " print('- '+summary)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summarize a book" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "number of pages: 282\n" ] } ], "source": [ "from langchain.document_loaders import PyPDFLoader\n", "# Loaders\n", "from langchain.schema import Document\n", "\n", "# Load the book\n", "loader = PyPDFLoader(\"data/book/IntoThinAirBook.pdf\")\n", "pages = loader.load()\n", "\n", "#print number of pages\n", "print('number of pages: ', len(pages))\n", "# Cut out the open and closing parts\n", "pages = pages[28:len(pages)]\n", "\n", "# Combine the pages, and replace the tabs with spaces\n", "text = \"\"\n", "\n", "for page in pages:\n", " text += page.page_content\n", " \n", "text = text.replace('\\t', ' ')" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Token indices sequence length is longer than the specified maximum sequence length for this model (141056 > 1024). Running this sequence through the model will result in indexing errors\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "This book has 141056 tokens in it\n" ] } ], "source": [ "num_tokens = sm_llm.get_num_tokens(text)\n", "\n", "print (f\"This book has {num_tokens} tokens in it\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that AI21 Summarize model can take upto 40k chunk size, therefore, dividing the book into 30k chunks. " ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "tags": [] }, "outputs": [], "source": [ "text_splitter = RecursiveCharacterTextSplitter(separators=[\"\\n\\n\", \"\\n\", \"\\t\"], chunk_size=20000, chunk_overlap=3000)\n", "\n", "docs = text_splitter.create_documents([text])" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Token indices sequence length is longer than the specified maximum sequence length for this model (4872 > 1024). Running this sequence through the model will result in indexing errors\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Now we have 33 documents and the first one has 4872 tokens\n" ] } ], "source": [ "num_docs = len(docs)\n", "\n", "num_tokens_first_doc = sm_llm.get_num_tokens(docs[0].page_content)\n", "\n", "print (f\"Now we have {num_docs} documents and the first one has {num_tokens_first_doc} tokens\")" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "tags": [] }, "outputs": [], "source": [ "map_prompt = \"\"\"\n", "\"{text}\"\n", "\"\"\"\n", "map_prompt_template = PromptTemplate(template=map_prompt, input_variables=[\"text\"])" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "tags": [] }, "outputs": [], "source": [ "map_chain = load_summarize_chain(llm=sm_llm,\n", " chain_type=\"stuff\",\n", " prompt=map_prompt_template)" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Make an empty list to hold your summaries\n", "summary_list = []\n", "\n", "# Loop through a range of the lenght of your selected docs\n", "for i, doc in enumerate(docs):\n", " \n", " # Go get a summary of the chunk\n", " chunk_summary = map_chain.run([doc])\n", " \n", " # Append that summary to your list\n", " summary_list.append(chunk_summary)\n", " \n", " # print (f\"Summary #{i+1} - Preview: {chunk_summary[:250]} \\n\")" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "tags": [] }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Token indices sequence length is longer than the specified maximum sequence length for this model (6506 > 1024). Running this sequence through the model will result in indexing errors\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Your total summary has 6506 tokens\n" ] } ], "source": [ "summaries = \"\\n\".join(summary_list)\n", "\n", "# Convert it back to a document\n", "summaries = Document(page_content=summaries)\n", "\n", "print (f\"Your total summary has {sm_llm.get_num_tokens(summaries.page_content)} tokens\")" ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "tags": [] }, "outputs": [], "source": [ "combine_prompt = \"\"\"\n", "\"{text}\"\n", "\"\"\"\n", "combine_prompt_template = PromptTemplate(template=combine_prompt, input_variables=[\"text\"])" ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "tags": [] }, "outputs": [], "source": [ "reduce_chain = load_summarize_chain(llm=sm_llm,\n", " chain_type=\"stuff\",\n", " prompt=combine_prompt_template,\n", "# verbose=True # Set this to true if you want to see the inner workings\n", " )" ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "tags": [] }, "outputs": [], "source": [ "output = reduce_chain.run([summaries])" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "- Hillary and Tenzing Norgay were the first men to stand atop Mount Everest, but. Hornbein's expedition began from Banepa and spent.\n", "- Doug Fischer and Hall climbed all fourteen 8,000-meter peaks.\n", "- Helen Woodall invited six female candidates to climb Kilimanjaro and banished Sunday Times reporter Ken Vernon and photographer Richard Shorey.\n", "- Sandy Hill Pittman. The. Ang. made a solo ascent sans gas two years later, and I rose lethargic and groggy.\n", "- A radial keratotomy is a surgical procedure to correct myopia. Stuart Hutchison organized a team of four Sherpas to locate the bodies of Weathers and Namba.\n", "- Anatoli Boukreev climbed alone to the summit of Lhotse on October 14, 1996. He denounced the book Into Thin Air.\n" ] } ], "source": [ "key_points = output.split('\\n')\n", "for key_point in key_points: \n", " print('- '+key_point)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clean Up\n", "*NOTE:* Please make sure to delete the endpoint, if you are not using it, as it will incur charges. " ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "tags": [] }, "outputs": [], "source": [ "# # Specify the name of your endpoint\n", "# endpoint_name_llm=\"summarize\"\n", "\n", "# # # Create a low-level SageMaker service client.\n", "# sagemaker_client = boto3.client('sagemaker', region_name=aws_region)\n", " \n", "# # # Delete endpoint configuration\n", "# sagemaker_client.delete_endpoint_config(EndpointConfigName=endpoint_name_llm)\n", "\n", "# # Delete endpoint\n", "# sagemaker_client.delete_endpoint(EndpointName=endpoint_name_llm)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "availableInstances": [ { "_defaultOrder": 0, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.t3.medium", "vcpuNum": 2 }, { "_defaultOrder": 1, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.t3.large", "vcpuNum": 2 }, { "_defaultOrder": 2, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.t3.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 3, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.t3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 4, "_isFastLaunch": true, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5.large", "vcpuNum": 2 }, { "_defaultOrder": 5, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 6, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 7, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 8, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 9, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 10, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 11, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 12, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.m5d.large", "vcpuNum": 2 }, { "_defaultOrder": 13, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.m5d.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 14, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.m5d.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 15, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.m5d.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 16, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.m5d.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 17, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.m5d.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 18, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.m5d.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 19, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.m5d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 20, "_isFastLaunch": false, "category": "General purpose", "gpuNum": 0, "hideHardwareSpecs": true, "memoryGiB": 0, "name": "ml.geospatial.interactive", "supportedImageNames": [ "sagemaker-geospatial-v1-0" ], "vcpuNum": 0 }, { "_defaultOrder": 21, "_isFastLaunch": true, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 4, "name": "ml.c5.large", "vcpuNum": 2 }, { "_defaultOrder": 22, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 8, "name": "ml.c5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 23, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.c5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 24, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.c5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 25, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 72, "name": "ml.c5.9xlarge", "vcpuNum": 36 }, { "_defaultOrder": 26, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 96, "name": "ml.c5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 27, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 144, "name": "ml.c5.18xlarge", "vcpuNum": 72 }, { "_defaultOrder": 28, "_isFastLaunch": false, "category": "Compute optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.c5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 29, "_isFastLaunch": true, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g4dn.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 30, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g4dn.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 31, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g4dn.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 32, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g4dn.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 33, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g4dn.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 34, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g4dn.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 35, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 61, "name": "ml.p3.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 36, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 244, "name": "ml.p3.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 37, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 488, "name": "ml.p3.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 38, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.p3dn.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 39, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.r5.large", "vcpuNum": 2 }, { "_defaultOrder": 40, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.r5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 41, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.r5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 42, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.r5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 43, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.r5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 44, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.r5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 45, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 512, "name": "ml.r5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 46, "_isFastLaunch": false, "category": "Memory Optimized", "gpuNum": 0, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.r5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 47, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 16, "name": "ml.g5.xlarge", "vcpuNum": 4 }, { "_defaultOrder": 48, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 32, "name": "ml.g5.2xlarge", "vcpuNum": 8 }, { "_defaultOrder": 49, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 64, "name": "ml.g5.4xlarge", "vcpuNum": 16 }, { "_defaultOrder": 50, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 128, "name": "ml.g5.8xlarge", "vcpuNum": 32 }, { "_defaultOrder": 51, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 1, "hideHardwareSpecs": false, "memoryGiB": 256, "name": "ml.g5.16xlarge", "vcpuNum": 64 }, { "_defaultOrder": 52, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 192, "name": "ml.g5.12xlarge", "vcpuNum": 48 }, { "_defaultOrder": 53, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 4, "hideHardwareSpecs": false, "memoryGiB": 384, "name": "ml.g5.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 54, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 768, "name": "ml.g5.48xlarge", "vcpuNum": 192 }, { "_defaultOrder": 55, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4d.24xlarge", "vcpuNum": 96 }, { "_defaultOrder": 56, "_isFastLaunch": false, "category": "Accelerated computing", "gpuNum": 8, "hideHardwareSpecs": false, "memoryGiB": 1152, "name": "ml.p4de.24xlarge", "vcpuNum": 96 } ], "instance_type": "ml.t3.medium", "kernelspec": { "display_name": "Python 3 (Data Science 2.0)", "language": "python", "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-38" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.13" } }, "nbformat": 4, "nbformat_minor": 4 }