LangChain Agents
LangChain agent tools act as a glue to map natural language human input into different sequences of actions. We are effectively using the real world knowledge in the text used to train LLMs to act as a reasoning agent.
The LangChain Agents Documentation provides everything you need to get started. Here we will dive a bit deeper into using local Python scripts in agents and look at an interesting example using SPARQL queries and the public DBPedia Knowledge Base. We will concentrate on just a few topics:
- Understanding what LangChain tools are and using pre-built tools.
- Get an overview of React reasoning. You should bookmark the original paper ReAct: Synergizing Reasoning and Acting in Language Models for reference. This paper inspired design and implementation of the agent tool code in LangChain.
- Writing custom functions for OpenAI: how to write a custom tool. We will write a tool that uses SPARQL queries to the DBPedia public Knowledge Graph.
Overview of LangChain Tools
As we have covered with many examples in this book, LangChain is a framework that provides tools for building LLM-powered applications.
Here we look at using built in LangChain agent tools, understand reactive agents, and end the chapter with a custom tool agent application.
LangChain tools are interfaces that an agent can use to interact with the world. They can be generic utilities (e.g. search), other chains, or even other agents. The interface API of a tool has a single text input and a single text output, and includes a name and description that communicate to the model what the tool does and when to use it.
Some tools can be used as-is and some tools (e.g. chains, agents) may require a base LLM to use to initialize them. In that case, you can pass in an LLM as well:
1 from langchain.agents import load_tools
2 tool_names = [...]
3 llm = ...
4 tools = load_tools(tool_names, llm=llm).
To implement your own tool, you can subclass the Tool class and implement the _call method. The _call method is called with the input text and should return the output text. The Tool superclass implements the call method, which takes care of calling the right CallbackManager methods before and after calling your _call method. When an error occurs, the _call method should when possible return a string representing an error, rather than throwing an error. This allows the error to be passed to the LLM and the LLM can decide how to handle it.
LangChain also provides pre-built tools that provide a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.
In summary, LangChain tools are interfaces that agents can use to interact with the world. They can be generic utilities or other chains or agents. Here is a list of some of the available LangChain agent tools:
- AWSLambda - A wrapper around the AWS Lambda API, invoked via the Amazon Web Services Node.js SDK. Useful for invoking server less functions with any behavior which you need to provide to an Agent.
- BingSerpAPI - A wrapper around the Bing Search API.
- BraveSearch - A wrapper around the Brave Search API.
- Calculator - Useful for getting the result of a math expression.
- GoogleCustomSearch - A wrapper around the Google Custom Search API.
- IFTTTWebHook - A wrapper around the IFTTT Web-hook API.
- OpenAI - A wrapper around the OpenAI API.
- OpenWeatherMap - A wrapper around the OpenWeatherMap API.
- Random - Useful for generating random numbers, strings, and other values.
- Wikipedia - A wrapper around the Wikipedia API.
- WolframAlpha - A wrapper around the WolframAlpha API.
Overview of ReAct Library for Implementing Reading in LMS Applications
Most of the material in this section is referenced from the paper ReAct: Synergizing Reasoning and Acting in Language Models. The ReAct framework attempts to solve the basic problem of getting LLMs to accurately perform tasks. We want an LLM to understand us and actually do what we want. I take a different but similar approach for an example in my book Safe For Humans AI A “humans-first” approach to designing and building AI systems (link for reading free online) where I use two different LLMs, one to generate answers questions and another LLM to judge how well the first model did. That example is fairly ad-hoc because I was experimenting with an idea. Here we do much the same thing using a pre-built framework.
ReAct is an extension of the idea that LLMs perform better when we ask not only for an answer but also for the reasoning steps to generate an answer. The authors of the ReAct paper refer to these reasoning steps as “reasoning traces.”
Another approach to using LLMs in applications is to ask directions for actions from an LLM, take those actions, and report the results of the actions back to the LLM. This action loop can be repeated.
The ReAct paper combines reasoning traces and action loops. To paraphrase the paper:
Large language models (LLMs) have shown impressive abilities in understanding language and making decisions. However, their capabilities for reasoning and taking action has been new work with some promising results. Here we look at using LLMs to generate both reasoning traces and task-specific actions together. This allows for better synergy between the two: reasoning traces help the model create and update action plans, while actions let it gather more information from external sources. For question answering and fact verification tasks, ReAct avoids errors by using a simple Wikipedia API and generates human-like solutions. On interactive decision making tasks, ReAct has higher success rates compared to other methods, even with limited examples.
A ReAct prompt consists of example solutions to tasks, including reasoning traces, actions, and observations of the environment. ReAct prompting is easy to design and achieves excellent performance on various tasks, from answering questions to online shopping.
The ReAct paper serves as the basis for the design and implementation of support for LangChain agent tools. We look at an example application using a custom tool in the next section.
LangChain Agent Tool Example Using DBPedia SPARQL Queries
Before we look at the the LangChain agent custom tool code, let’s look at some utility code from my Colab notebook Question Answering Example using DBPedia and SPARQL (link to shared Colab notebook). I extracted just the code we need into the file QA.py (edited to fit page width):
1 # Copyright 2021-2023 Mark Watson
2
3 import spacy
4
5 nlp_model = spacy.load("en_core_web_sm")
6
7 from SPARQLWrapper import SPARQLWrapper, JSON
8
9 sparql = SPARQLWrapper("http://dbpedia.org/sparql")
10
11 def query(query):
12 sparql.setQuery(query)
13 sparql.setReturnFormat(JSON)
14 return sparql.query().convert()["results"]["bindings"]
15
16 def entities_in_text(s):
17 doc = nlp_model(s)
18 ret = {}
19 for [ename, etype] in [[entity.text, entity.label_] for entity in doc.ents]:
20 if etype in ret:
21 ret[etype] = ret[etype] + [ename]
22 else:
23 ret[etype] = [ename]
24 return ret
25
26 # NOTE: !! note "{{" .. "}}" double curly brackets: this is to escape for Python Str\
27 ing format method:
28
29 sparql_query_template = """
30 select distinct ?s ?comment where {{
31 ?s
32 <http://www.w3.org/2000/01/rdf-schema#label>
33 '{name}'@en .
34 ?s
35 <http://www.w3.org/2000/01/rdf-schema#comment>
36 ?comment .
37 FILTER (lang(?comment) = 'en') .
38 ?s
39 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
40 {dbpedia_type} .
41 }} limit 15
42 """
43
44 def dbpedia_get_entities_by_name(name, dbpedia_type):
45 print(f"{name=} {dbpedia_type=}")
46 s_query = \
47 sparql_query_template.format(
48 name=name,
49 dbpedia_type=dbpedia_type
50 )
51 print(s_query)
52 results = query(s_query)
53 return results
54
55 entity_type_to_type_uri = {
56 "PERSON": "<http://dbpedia.org/ontology/Person>",
57 "GPE": "<http://dbpedia.org/ontology/Place>",
58 "ORG": "<http://dbpedia.org/ontology/Organisation>",
59 }
60
61 def get_context_text(query_text):
62 entities = entities_in_text(query_text)
63
64 def helper(entity_type):
65 ret = ""
66 if entity_type in entities:
67 for hname in entities[entity_type]:
68 results = dbpedia_get_entities_by_name(
69 hname,
70 entity_type_to_type_uri[entity_type]
71 )
72 for result in results:
73 ret += ret + \
74 result["comment"]["value"] + \
75 " . "
76 return ret
77
78 context_text = helper("PERSON") + \
79 helper("ORG") + \
80 helper("GPE")
81 #print("\ncontext text:\n", context_text, "\n")
82 return context_text
We use the spaCy library and a small spaCy NLP model that we set up in lines 3-5.
I have written two books dedicated to SPARQL queries as well as providing SPARQL overviews and examples in my Haskell, Common Lisp, and Hy Language books and I am not going to repeat that discusion here. You can read the semantic web and SPARQL material free online using [https://leanpub.com/lovinglisp/read#leanpub-auto-semantic-web-and-linked-data](this link).
Lines 7-14 contain Python code for querying DBPedia.
Lines 16-25 use the spaCy library to identify both the entities in a user’s query as well as the entity type.
We define a SPARQL query template in lines 30-43 that uses Python F string variables name and dbpedia_type.
The function dbpedia_get_entities_by_name defined in lines 45-54 replaces variables with values in the SPARQL query template and makes a SPARQL query to DBPedia.
The function get_context_text which is the function in this file we will directly call later is defined in lines 65-83. We get entities and entity types in line 63. We define an internal helper function in lines 65-77 that we will call once for each of three DBPedia entity types that we use in this example (people, organizations. and organizations).
Here we use spaCy so install the library and a small NLP model:
1 pip install import spacy
2 python -m spacy download en_core_web_sm
The agent custom tool example is short so I list the source file custom_func_dbpedia.py first and then we will dive into the code (edited to fit page width):
1 from QA import get_context_text
2
3 def get_context_data(query_text):
4 """
5 Method to get context text for entities from
6 DBPedia using SPARQL query
7 """
8
9 query_text_data = get_context_text(query_text)
10 return {"context_text": query_text_data}
11
12 ## Custom function example using DBPedia
13
14 from typing import Type
15 from pydantic import BaseModel, Field
16 from langchain.tools import BaseTool
17
18 class GetContextTextFromDbPediaInput(BaseModel):
19 """Inputs for get_context_data"""
20
21 query_text: str = \
22 Field(
23 description="query_text user supplied query text"
24 )
25
26 class GetContextTextFromDbPediaTool(BaseTool):
27 name = "get_context_data"
28 description =
29 """
30 Useful when you want to make a query and get
31 context text from DBPedia. You should enter
32 and text containing entity names.
33 """
34 args_schema: Type[BaseModel] = \
35 GetContextTextFromDbPediaInput
36
37 def _run(self, query_text: str):
38 text = get_context_data(query_text)
39 return text
40
41 def _arun(self, query_text: str):
42 raise NotImplementedError
43 (
44 "get_context_data does not support async"
45 )
46
47 ## Create agent
48
49 from langchain.agents import AgentType
50 from langchain.chat_models import ChatOpenAI
51 from langchain.agents import initialize_agent
52
53 llm = ChatOpenAI(model="gpt-3.5-turbo-0613",
54 temperature=0)
55
56 tools = [GetContextTextFromDbPediaTool()]
57
58 agent = initialize_agent(tools, llm,
59 agent=AgentType.OPENAI_FUNCTIONS,
60 verbose=True)
61
62 ## Run agent
63
64 agent.run(
65 """
66 What country is Berlin in and what other
67 information about the city do you have?
68 """
69 )
The class GetContextTextFromDbPediaInput defined in lines 18-24 defines a tool input variable with an English language description for the variable that the LLM can use. The class GetContextTextFromDbPediaTool defined in lines 26-45 defines the tool name, a description for the use of an LLM, and the definition of the required methon _run. Methon _run uses the utility finction get_context_data defined in the source file QA.py.
We define a GPT-3.5 model in lines 53-54. Out example only uses one tool (our custom tool). We define the tools list in line 56 and setup the agent in lines 58-60.
The example output is (edited to fit page width):
1 > Entering new AgentExecutor chain...
2
3 Invoking: `get_context_data` with `{'query_text':
4 'Berlin'}`
5
6 name='Berlin'
7 dbpedia_type='<http://dbpedia.org/ontology/Place>'
8
9 select distinct ?s ?comment where {
10 ?s
11 <http://www.w3.org/2000/01/rdf-schema#label>
12 'Berlin'@en .
13 ?s
14 <http://www.w3.org/2000/01/rdf-schema#comment>
15 ?comment .
16 FILTER (lang(?comment) = 'en') .
17 ?s
18 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
19 <http://dbpedia.org/ontology/Place> .
20 } limit 15
21
22 {'context_text': "Berlin (/bɜːrˈlɪn/ bur-LIN, German: [bɛʁˈliːn]) is the capital and\
23 largest city of Germany by both area and population. Its 3.6 million inhabitants ma
24 ke it the European Union's most populous city, according to population within city l
25 imits. One of Germany's sixteen constituent states, Berlin is surrounded by the Stat
26 e of Brandenburg and contiguous with Potsdam, Brandenburg's capital. Berlin's urban
27 area, which has a population of around 4.5 million, is the second most populous urba
28 n area in Germany after the Ruhr. The Berlin-Brandenburg capital region has around 6
29 .2 million inhabitants and is Germany's third-largest metropolitan region after the
30 Rhine-Ruhr and Rhine-Main regions. . "}
31
32 Berlin is the capital and largest city of Germany. It is located in the northeastern\
33 part of the country. Berlin has a population of approximately 3.6 million people, m
34 aking it the most populous city in the European Union. It is surrounded by the State
35 of Brandenburg and is contiguous with Potsdam, the capital of Brandenburg. The urba
36 n area of Berlin has a population of around 4.5 million, making it the second most p
37 opulous urban area in Germany after the Ruhr. The Berlin-Brandenburg capital region
38 has a population of approximately 6.2 million, making it Germany's third-largest met
39 ropolitan region after the Rhine-Ruhr and Rhine-Main regions.
40
41 > Finished chain.
Another React Agent Tool Use Example: Search and Math Expressions
The example for this section is found in langchain-book-examples/tool_search_math_example.
In this chapter, we explore the development of a multi-tool intelligent agent by leveraging the power of LangChain and OpenAI’s GPT models. With recent advancements in Large Language Models (LLMs) and agent frameworks, it’s possible to create interactive tools capable of dynamic actions like web searches or evaluating mathematical expressions. This script demonstrates how to integrate two example tools, a DuckDuckGo-based search engine and a simple calculator, into a React-style agent capable of performing multi-step reasoning.
This example highlights the practicalities of creating a modular agent that combines natural language capabilities with functional tools. It illustrates the use of:
- LLMs for conversational abilities (via OpenAI’s GPT-4 mini model).
- Third-party API tools, like DuckDuckGo search, for retrieving real-time information.
- Custom tools, such as a calculator, for handling specific operations.
By the end of this chapter, readers will understand how to define custom tools, set up a prompt template, and integrate these tools into an interactive agent capable of multi-step tasks. This example demonstrates the potential of tool-enhanced AI agents to solve both simple and complex problems dynamically.
Tools Example Implementation
1 from langchain.agents import create_react_agent, Tool, AgentExecutor
2 from langchain.agents.agent_types import AgentType
3 from langchain_community.tools import DuckDuckGoSearchRun
4 from langchain_openai import ChatOpenAI
5 from langchain_core.prompts import PromptTemplate
6 import os
7
8 openai_key = os.getenv("OPENAI_API_KEY")
9 llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.3, openai_api_key=openai_key)
10
11 ddg_api = DuckDuckGoSearchRun()
12
13 class DuckDuckGoSearchAPIWrapper:
14 def __call__(self, query):
15 results = ddg_api.invoke(query)
16 #print(f"**** {results=}")
17 return results if results else 'No results found'
18
19 class SimpleCalculator:
20 def __call__(self, expression):
21 try:
22 return eval(expression)
23 except Exception as e:
24 return f"Error in calculation: {e}"
25
26 # Initialize the tools
27 search_tool = Tool(
28 name="duckduckgo_search",
29 func=DuckDuckGoSearchAPIWrapper(),
30 description="Searches the web using DuckDuckGo"
31 )
32
33 calculator_tool = Tool(
34 name="simple_calculator",
35 func=SimpleCalculator(),
36 description="Performs simple calculations"
37 )
38
39 # Define the tool chain
40 tools = [search_tool, calculator_tool]
41 tool_names = ["duckduckgo_search", "simple_calculator"]
42
43 agent_scratchpad = "thoughts: "
44
45 template = '''Answer the following questions as best you can. You have access to the\
46 following tools:
47
48 {tools}
49
50 Use the following format:
51
52 Question: the input question you must answer
53 Thought: you should always think about what to do
54 Action: the action to take, should be one of [{tool_names}]
55 Action Input: the input to the action
56 Observation: the result of the action
57 ... (this Thought/Action/Action Input/Observation can repeat N times)
58 Thought: I now know the final answer
59 Final Answer: the final answer to the original input question
60
61 Begin!
62
63 Question: {input}
64 Thought:{agent_scratchpad}'''
65
66 prompt = PromptTemplate.from_template(template)
67 print(prompt)
68
69 # Initialize the agent with tools
70 agent = create_react_agent(llm, tools, prompt)
71
72 agent_executor = AgentExecutor(agent=agent, tools=tools)
73
74 # Example #1 input for the chain
75 input_text = "search: What is the population of Canada?"
76
77 # Run the chain
78 result = agent_executor.invoke({"input": input_text, "chat_history": agent_scratchpa\
79 d})
80
81 # Print the result
82 print(result)
83
84 # Example #2 input for the chain
85 input_text = "calculator: 250 * 4"
86
87 # Run the chain
88 result = agent_executor.invoke({"input": input_text, "chat_history": agent_scratchpa\
89 d})
90
91 # Print the result
92 print(result)
The script imports key modules from LangChain, OpenAI, and a DuckDuckGo wrapper to build the agent framework. The OpenAI API key is pulled from an environment variable to maintain security.
Defining Tools
The agent is equipped with two tools:
- DuckDuckGo Search Tool: Uses the DuckDuckGoSearchRun API to fetch real-time web search results.
- Simple Calculator Tool: Evaluates mathematical expressions using Python’s eval() function.
Each tool is encapsulated in its own class with a call() method to enable easy invocation.
The two tools are initialized using LangChain’s Tool class. Each tool is assigned a name, func, and description.
A prompt template defines how the agent should format its responses. The agent follows a structured reasoning process, using thought-action-observation loops. Note that this example is written in a general purpose style supporting chain of thought and memory but here the tool use examples are one-shot, not multiple prompts.
Sample Output
1 $ python tool_search_math_example.py
2 {'input': 'search: What is the population of Canada?', 'chat_history': 'thoughts: ',\
3 'output': 'The population of Canada is approximately 40,528,396 as of October 1, 20
4 23.'}
5 {'input': 'calculator: 250 * 4', 'chat_history': 'thoughts: ', 'output': '1000'}
LangChain Agent Tools Wrap Up
Writing custom agent tools is a great way to revisit the implementation of existing applications and improve them with the real world knowledge and reasoning abilities of LLMs. Both for practice in effectively using LLMs in your applications and also to extend your personal libraries and tools, I suggest that you look over you existing projects with an eye for either improving them using LLMs or refactoring them into reusable agent tools for your future projects.