{ "cells": [ { "cell_type": "markdown", "id": "e8cba0b6", "metadata": {}, "source": [ "\"Open\n", "\n", "## Use Llama 3 to chat about structured data\n", "This demo shows how to use LangChain's SQLDatabaseChain with Llama 3 to query structured data, the 2023-24 NBA roster info, stored in a SQLite DB, to ask Llama 3 questions about your favorite teams or players." ] }, { "cell_type": "markdown", "id": "f839d07d", "metadata": {}, "source": [ "We start by installing the necessary packages:\n", "- [Replicate](https://replicate.com/) to host the Llama 3 model\n", "- [langchain](https://python.langchain.com/docs/get_started/introduction) provides necessary RAG tools for this demo\n", "\n", "**Note** We will be using [Replicate](https://replicate.com/meta/meta-llama-3-8b-instruct) to run the examples here. You will need to first sign in with Replicate with your github account, then create a free API token [here](https://replicate.com/account/api-tokens) that you can use for a while. You can also use other Llama 3 cloud providers such as [Groq](https://console.groq.com/), [Together](https://api.together.xyz/playground/language/meta-llama/Llama-3-8b-hf), or [Anyscale](https://app.endpoints.anyscale.com/playground) - see Section 2 of the Getting to Know Llama [notebook](https://github.com/meta-llama/llama-recipes/blob/main/recipes/quickstart/Getting_to_know_Llama.ipynb) for more information.\n", "\n", "If you'd like to run Llama 3 locally for the benefits of privacy, no cost or no rate limit (some Llama 3 hosting providers set limits for free plan of queries or tokens per second or minute), see [Running Llama Locally](https://github.com/meta-llama/llama-recipes/blob/main/recipes/quickstart/Running_Llama2_Anywhere/Running_Llama_on_Mac_Windows_Linux.ipynb)." ] }, { "cell_type": "code", "execution_count": 1, "id": "33fb3190-59fb-4edd-82dd-f20f6eab3e47", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: langchain in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (0.1.16)\n", "Requirement already satisfied: replicate in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (0.25.2)\n", "Requirement already satisfied: PyYAML>=5.3 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (6.0.1)\n", "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (2.0.29)\n", "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (3.9.5)\n", "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (0.6.4)\n", "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (1.33)\n", "Requirement already satisfied: langchain-community<0.1,>=0.0.32 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (0.0.34)\n", "Requirement already satisfied: langchain-core<0.2.0,>=0.1.42 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (0.1.45)\n", "Requirement already satisfied: langchain-text-splitters<0.1,>=0.0.1 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (0.0.1)\n", "Requirement already satisfied: langsmith<0.2.0,>=0.1.17 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (0.1.50)\n", "Requirement already satisfied: numpy<2,>=1 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (1.26.4)\n", "Requirement already satisfied: pydantic<3,>=1 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (2.7.1)\n", "Requirement already satisfied: requests<3,>=2 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (2.31.0)\n", "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langchain) (8.2.3)\n", "Requirement already satisfied: httpx<1,>=0.21.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from replicate) (0.27.0)\n", "Requirement already satisfied: packaging in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from replicate) (23.2)\n", "Requirement already satisfied: typing-extensions>=4.5.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from replicate) (4.11.0)\n", "Requirement already satisfied: aiosignal>=1.1.2 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n", "Requirement already satisfied: attrs>=17.3.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n", "Requirement already satisfied: frozenlist>=1.1.1 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n", "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (3.21.1)\n", "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (0.9.0)\n", "Requirement already satisfied: anyio in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from httpx<1,>=0.21.0->replicate) (4.3.0)\n", "Requirement already satisfied: certifi in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from httpx<1,>=0.21.0->replicate) (2024.2.2)\n", "Requirement already satisfied: httpcore==1.* in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from httpx<1,>=0.21.0->replicate) (1.0.5)\n", "Requirement already satisfied: idna in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from httpx<1,>=0.21.0->replicate) (3.7)\n", "Requirement already satisfied: sniffio in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from httpx<1,>=0.21.0->replicate) (1.3.1)\n", "Requirement already satisfied: h11<0.15,>=0.13 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from httpcore==1.*->httpx<1,>=0.21.0->replicate) (0.14.0)\n", "Requirement already satisfied: jsonpointer>=1.9 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain) (2.4)\n", "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.17->langchain) (3.10.1)\n", "Requirement already satisfied: annotated-types>=0.4.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from pydantic<3,>=1->langchain) (0.6.0)\n", "Requirement already satisfied: pydantic-core==2.18.2 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from pydantic<3,>=1->langchain) (2.18.2)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from requests<3,>=2->langchain) (3.3.2)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from requests<3,>=2->langchain) (2.2.1)\n", "Requirement already satisfied: mypy-extensions>=0.3.0 in /Users/jeffxtang/anaconda3/envs/llama3/lib/python3.11/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain) (1.0.0)\n" ] } ], "source": [ "!pip install langchain replicate" ] }, { "cell_type": "code", "execution_count": 2, "id": "fa4562d3", "metadata": {}, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ " ········\n" ] } ], "source": [ "from getpass import getpass\n", "import os\n", "\n", "REPLICATE_API_TOKEN = getpass()\n", "os.environ[\"REPLICATE_API_TOKEN\"] = REPLICATE_API_TOKEN" ] }, { "cell_type": "markdown", "id": "1e586b75", "metadata": {}, "source": [ "Next we call the Llama 3 8b chat model from Replicate. You can also use Llama 3 70b model by replacing the `model` name with \"meta/meta-llama-3-70b-instruct\"." ] }, { "cell_type": "code", "execution_count": 3, "id": "9dcd744c", "metadata": {}, "outputs": [], "source": [ "from langchain_community.llms import Replicate\n", "llm = Replicate(\n", " model=\"meta/meta-llama-3-8b-instruct\",\n", " model_kwargs={\"temperature\": 0.0, \"top_p\": 1, \"max_new_tokens\":500}\n", ")" ] }, { "cell_type": "markdown", "id": "6d421ae7", "metadata": {}, "source": [ "Next you will need create the `nba_roster.db` file. \n", "\n", "To do this run the following commands while in this folder:\n", "- `python txt2csv.py` This will convert the `nba.txt` file to `nba_roster.csv`. The `nba.txt` file was created by scraping the NBA roster info from the web.\n", "- Then run `python csv2db.py` to convert `nba_roster.csv` to `nba_roster.db`.\n", "\n", "Once you have your `nba_roster.db` ready, we set up the database to be queried by Llama 2 through Langchain's [SQL chains](https://python.langchain.com/docs/use_cases/qa_structured/sql)." ] }, { "cell_type": "code", "execution_count": 12, "id": "3bb99f39-cd7a-4db6-91dd-02f3bf80347c", "metadata": {}, "outputs": [], "source": [ "from langchain_community.utilities import SQLDatabase\n", "\n", "db = SQLDatabase.from_uri(\"sqlite:///nba_roster.db\", sample_rows_in_table_info=0)\n", "\n", "def get_schema():\n", " return db.get_table_info()\n", "\n", "def run_query(query):\n", " return db.run(query)" ] }, { "cell_type": "code", "execution_count": 13, "id": "4d5049bf-36c4-47ac-b0f8-6cf5c4362729", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'\\nCREATE TABLE nba_roster (\\n\\t\"Team\" TEXT, \\n\\t\"NAME\" TEXT, \\n\\t\"Jersey\" TEXT, \\n\\t\"POS\" TEXT, \\n\\t\"AGE\" INTEGER, \\n\\t\"HT\" TEXT, \\n\\t\"WT\" TEXT, \\n\\t\"COLLEGE\" TEXT, \\n\\t\"SALARY\" TEXT\\n)'" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_schema()" ] }, { "cell_type": "code", "execution_count": 22, "id": "8d793ce7-324b-4861-926c-54973d7c9b43", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Based on the table schema below, write a SQL query that would answer the user's question; just return the SQL query and nothing else.\n", "\n", "Scheme:\n", "\n", "CREATE TABLE nba_roster (\n", "\t\"Team\" TEXT, \n", "\t\"NAME\" TEXT, \n", "\t\"Jersey\" TEXT, \n", "\t\"POS\" TEXT, \n", "\t\"AGE\" INTEGER, \n", "\t\"HT\" TEXT, \n", "\t\"WT\" TEXT, \n", "\t\"COLLEGE\" TEXT, \n", "\t\"SALARY\" TEXT\n", ")\n", "\n", "Question: What team is Klay Thompson on?\n", "\n", "SQL Query:\n" ] } ], "source": [ "question = \"What team is Klay Thompson on?\"\n", "prompt = f\"\"\"Based on the table schema below, write a SQL query that would answer the user's question; just return the SQL query and nothing else.\n", "\n", "Scheme:\n", "{get_schema()}\n", "\n", "Question: {question}\n", "\n", "SQL Query:\"\"\"\n", "\n", "print(prompt)" ] }, { "cell_type": "code", "execution_count": 24, "id": "70776558", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "SELECT \"Team\" FROM nba_roster WHERE \"NAME\" = 'Klay Thompson';\n" ] } ], "source": [ "print(llm.invoke(prompt))" ] }, { "cell_type": "markdown", "id": "afcf423a", "metadata": {}, "source": [ "If you don't have the \"just return the SQL query and nothing else\" in the prompt above, or even with it but asking Llama 2 which doesn't follow instructions as well as Llama 3, you'll likely get more text other than the SQL query back in the answer." ] }, { "cell_type": "code", "execution_count": 26, "id": "39ed4bc3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "I'm happy to help! However, I need more information. Who is \"his\"? Could you please provide more context or clarify who you are referring to?\n" ] } ], "source": [ "# how about a follow up question\n", "print(llm.invoke(\"What's his salary?\"))" ] }, { "cell_type": "markdown", "id": "98b2c523", "metadata": {}, "source": [ "\n", "Since we did not pass any context along with the follow-up to the model it did not know who \"his\" is and just picked LeBron James.\n", "\n", "Let's try to fix the issue that the context (the previous question and answer) was not sent to the model along with the new question.\n", "`SQLDatabaseChain.from_llm` has a parameter \"memory\" which can be set to a `ConversationBufferMemory` instance, which looks promising.\n" ] }, { "cell_type": "code", "execution_count": 8, "id": "0c305278-29d2-4e88-9b3d-ad67c94ce0f2", "metadata": {}, "outputs": [], "source": [ "\n", "from langchain.memory import ConversationBufferMemory\n", "\n", "memory = ConversationBufferMemory()\n", "db_chain_memory = SQLDatabaseChain.from_llm(llm, db, memory=memory, \n", " verbose=True, return_sql=True, \n", " prompt=PromptTemplate(input_variables=[\"input\", \"table_info\"], \n", " template=PROMPT_SUFFIX))" ] }, { "cell_type": "code", "execution_count": 9, "id": "d12b50e7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain] Entering Chain run with input:\n", "\u001b[0m{\n", " \"query\": \"Which team is Klay Thompson in\",\n", " \"history\": \"\"\n", "}\n", "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain > 2:chain:LLMChain] Entering Chain run with input:\n", "\u001b[0m{\n", " \"input\": \"Which team is Klay Thompson in\\nSQLQuery:\",\n", " \"top_k\": \"5\",\n", " \"dialect\": \"sqlite\",\n", " \"table_info\": \"\\nCREATE TABLE nba_roster (\\n\\t\\\"Team\\\" TEXT, \\n\\t\\\"NAME\\\" TEXT, \\n\\t\\\"Jersey\\\" TEXT, \\n\\t\\\"POS\\\" TEXT, \\n\\t\\\"AGE\\\" INTEGER, \\n\\t\\\"HT\\\" TEXT, \\n\\t\\\"WT\\\" TEXT, \\n\\t\\\"COLLEGE\\\" TEXT, \\n\\t\\\"SALARY\\\" TEXT\\n)\",\n", " \"stop\": [\n", " \"\\nSQLResult:\"\n", " ],\n", " \"history\": \"\"\n", "}\n", "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain > 2:chain:LLMChain > 3:llm:Replicate] Entering LLM run with input:\n", "\u001b[0m{\n", " \"prompts\": [\n", " \"Only use the following tables:\\n\\nCREATE TABLE nba_roster (\\n\\t\\\"Team\\\" TEXT, \\n\\t\\\"NAME\\\" TEXT, \\n\\t\\\"Jersey\\\" TEXT, \\n\\t\\\"POS\\\" TEXT, \\n\\t\\\"AGE\\\" INTEGER, \\n\\t\\\"HT\\\" TEXT, \\n\\t\\\"WT\\\" TEXT, \\n\\t\\\"COLLEGE\\\" TEXT, \\n\\t\\\"SALARY\\\" TEXT\\n)\\n\\nQuestion: Which team is Klay Thompson in\\nSQLQuery:\"\n", " ]\n", "}\n", "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain > 2:chain:LLMChain > 3:llm:Replicate] [6.52s] Exiting LLM run with output:\n", "\u001b[0m{\n", " \"generations\": [\n", " [\n", " {\n", " \"text\": \" Sure thing! Based on the information provided in the `nba_roster` table, Klay Thompson is in the Golden State Warriors. Here's the SQL query to retrieve that information:\\n```sql\\nSELECT * FROM nba_roster WHERE Team = 'Golden State Warriors';\\n```\\nThis will return all rows where the `Team` column matches \\\"Golden State Warriors\\\", which should only have one row with Klay Thompson's information.\",\n", " \"generation_info\": null\n", " }\n", " ]\n", " ],\n", " \"llm_output\": null,\n", " \"run\": null\n", "}\n", "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain > 2:chain:LLMChain] [6.52s] Exiting Chain run with output:\n", "\u001b[0m{\n", " \"text\": \" Sure thing! Based on the information provided in the `nba_roster` table, Klay Thompson is in the Golden State Warriors. Here's the SQL query to retrieve that information:\\n```sql\\nSELECT * FROM nba_roster WHERE Team = 'Golden State Warriors';\\n```\\nThis will return all rows where the `Team` column matches \\\"Golden State Warriors\\\", which should only have one row with Klay Thompson's information.\"\n", "}\n", "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain] [6.52s] Exiting Chain run with output:\n", "\u001b[0m{\n", " \"result\": \"Sure thing! Based on the information provided in the `nba_roster` table, Klay Thompson is in the Golden State Warriors. Here's the SQL query to retrieve that information:\\n```sql\\nSELECT * FROM nba_roster WHERE Team = 'Golden State Warriors';\\n```\\nThis will return all rows where the `Team` column matches \\\"Golden State Warriors\\\", which should only have one row with Klay Thompson's information.\"\n", "}\n", "Sure thing! Based on the information provided in the `nba_roster` table, Klay Thompson is in the Golden State Warriors. Here's the SQL query to retrieve that information:\n", "```sql\n", "SELECT * FROM nba_roster WHERE Team = 'Golden State Warriors';\n", "```\n", "This will return all rows where the `Team` column matches \"Golden State Warriors\", which should only have one row with Klay Thompson's information.\n" ] } ], "source": [ "# use the db_chain_memory to run the original question again\n", "question = \"Which team is Klay Thompson in\"\n", "answer = db_chain_memory.run(question)\n", "print(answer)" ] }, { "cell_type": "code", "execution_count": 10, "id": "fee65d5f", "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain] Entering Chain run with input:\n", "\u001b[0m{\n", " \"query\": \"What's his salary\",\n", " \"history\": \"Human: Which team is Klay Thompson in\\nAI: Sure thing! Based on the information provided in the `nba_roster` table, Klay Thompson is in the Golden State Warriors. Here's the SQL query to retrieve that information:\\n```sql\\nSELECT * FROM nba_roster WHERE Team = 'Golden State Warriors';\\n```\\nThis will return all rows where the `Team` column matches \\\"Golden State Warriors\\\", which should only have one row with Klay Thompson's information.\\nHuman: Which team is Klay Thompson in\\nAI: \\\"Sure thing! Based on the information provided in the `nba_roster` table, Klay Thompson is in the Golden State Warriors. Here's the SQL query to retrieve that information:\\\\n```sql\\\\nSELECT * FROM nba_roster WHERE Team = 'Golden State Warriors';\\\\n```\\\\nThis will return all rows where the `Team` column matches \\\\\\\"Golden State Warriors\\\\\\\", which should only have one row with Klay Thompson's information.\\\"\"\n", "}\n", "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain > 2:chain:LLMChain] Entering Chain run with input:\n", "\u001b[0m{\n", " \"input\": \"What's his salary\\nSQLQuery:\",\n", " \"top_k\": \"5\",\n", " \"dialect\": \"sqlite\",\n", " \"table_info\": \"\\nCREATE TABLE nba_roster (\\n\\t\\\"Team\\\" TEXT, \\n\\t\\\"NAME\\\" TEXT, \\n\\t\\\"Jersey\\\" TEXT, \\n\\t\\\"POS\\\" TEXT, \\n\\t\\\"AGE\\\" INTEGER, \\n\\t\\\"HT\\\" TEXT, \\n\\t\\\"WT\\\" TEXT, \\n\\t\\\"COLLEGE\\\" TEXT, \\n\\t\\\"SALARY\\\" TEXT\\n)\",\n", " \"stop\": [\n", " \"\\nSQLResult:\"\n", " ],\n", " \"history\": \"Human: Which team is Klay Thompson in\\nAI: Sure thing! Based on the information provided in the `nba_roster` table, Klay Thompson is in the Golden State Warriors. Here's the SQL query to retrieve that information:\\n```sql\\nSELECT * FROM nba_roster WHERE Team = 'Golden State Warriors';\\n```\\nThis will return all rows where the `Team` column matches \\\"Golden State Warriors\\\", which should only have one row with Klay Thompson's information.\\nHuman: Which team is Klay Thompson in\\nAI: \\\"Sure thing! Based on the information provided in the `nba_roster` table, Klay Thompson is in the Golden State Warriors. Here's the SQL query to retrieve that information:\\\\n```sql\\\\nSELECT * FROM nba_roster WHERE Team = 'Golden State Warriors';\\\\n```\\\\nThis will return all rows where the `Team` column matches \\\\\\\"Golden State Warriors\\\\\\\", which should only have one row with Klay Thompson's information.\\\"\"\n", "}\n", "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain > 2:chain:LLMChain > 3:llm:Replicate] Entering LLM run with input:\n", "\u001b[0m{\n", " \"prompts\": [\n", " \"Only use the following tables:\\n\\nCREATE TABLE nba_roster (\\n\\t\\\"Team\\\" TEXT, \\n\\t\\\"NAME\\\" TEXT, \\n\\t\\\"Jersey\\\" TEXT, \\n\\t\\\"POS\\\" TEXT, \\n\\t\\\"AGE\\\" INTEGER, \\n\\t\\\"HT\\\" TEXT, \\n\\t\\\"WT\\\" TEXT, \\n\\t\\\"COLLEGE\\\" TEXT, \\n\\t\\\"SALARY\\\" TEXT\\n)\\n\\nQuestion: What's his salary\\nSQLQuery:\"\n", " ]\n", "}\n", "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain > 2:chain:LLMChain > 3:llm:Replicate] [5.16s] Exiting LLM run with output:\n", "\u001b[0m{\n", " \"generations\": [\n", " [\n", " {\n", " \"text\": \" Sure thing! To find out what LeBron James' salary is, we can run a SQL query like this:\\n```\\nSELECT SALARY FROM nba_roster WHERE NAME = 'LeBron James';\\n```\\nThis will return the salary for LeBron James, which should be around $30 million per year.\",\n", " \"generation_info\": null\n", " }\n", " ]\n", " ],\n", " \"llm_output\": null,\n", " \"run\": null\n", "}\n", "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain > 2:chain:LLMChain] [5.16s] Exiting Chain run with output:\n", "\u001b[0m{\n", " \"text\": \" Sure thing! To find out what LeBron James' salary is, we can run a SQL query like this:\\n```\\nSELECT SALARY FROM nba_roster WHERE NAME = 'LeBron James';\\n```\\nThis will return the salary for LeBron James, which should be around $30 million per year.\"\n", "}\n", "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:SQLDatabaseChain] [5.16s] Exiting Chain run with output:\n", "\u001b[0m{\n", " \"result\": \"Sure thing! To find out what LeBron James' salary is, we can run a SQL query like this:\\n```\\nSELECT SALARY FROM nba_roster WHERE NAME = 'LeBron James';\\n```\\nThis will return the salary for LeBron James, which should be around $30 million per year.\"\n", "}\n", "Sure thing! To find out what LeBron James' salary is, we can run a SQL query like this:\n", "```\n", "SELECT SALARY FROM nba_roster WHERE NAME = 'LeBron James';\n", "```\n", "This will return the salary for LeBron James, which should be around $30 million per year.\n" ] } ], "source": [ "import json\n", "\n", "memory.save_context({\"input\": question},\n", " {\"output\": json.dumps(answer)})\n", "followup = \"What's his salary\"\n", "followup_answer = db_chain_memory.run(followup)\n", "print(followup_answer)\n", "\n", "# \"Entering Chain run with input\" does show \"history\" including our question: \"Human: Which team is Klay Thompson in\"\n", "# but it doesn't get passed to \"Entering LLM run with input\", so the llm output still is Lebron James." ] }, { "cell_type": "markdown", "id": "d23d47e9", "metadata": {}, "source": [ "Looks like we hit a [known issue](https://github.com/langchain-ai/langchain/issues/6918#issuecomment-1632932653) with adding memory to SQLDatabaseChain, even after the [PR](https://github.com/langchain-ai/langchain/pull/7546) that's supposed to fix it was merged. We'll leave the notebook using the experimental SQLDatabaseChain with memory as is. To find a better solution to using SQLDatabaseChain, we can:\n", "1. Look into the SQLDatabaseChain's implementation and the use of ConversationBufferMemory to fix the memory issue;\n", "2. Wait till the SQLDatabaseChain memory issue is really fixed.\n", "We'll update this demo when a good solution is found." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 5 }