LLM Tool Calling with Ollama
There are several example Python tool utilities in the GitHub repository https://github.com/mark-watson/Ollama_in_Action_Book in the source-code directory that we will use for function calling that start with the “tool” prefix:
1 https://github.com/mark-watson/Ollama_in_Action_Book/source-code $ ls -lh
2 total 1680
3 drwxr-xr-x 7 markw staff 224B Oct 14 14:43 autogen
4 drwxr-xr-x 6 markw staff 192B Oct 14 14:43 chains
5 drwxr-xr-x 4 markw staff 128B Aug 10 16:50 data
6 drwxr-xr-x 5 markw staff 160B Oct 14 14:43 graph
7 drwxr-xr-x 5 markw staff 160B Oct 14 14:43 judges
8 drwxr-xr-x 4 markw staff 128B Oct 14 14:43 langgraph
9 -rw-r--r-- 1 markw staff 107B Oct 14 14:43 Makefile
10 drwxr-xr-x 5 markw staff 160B Oct 14 14:43 memory
11 drwxr-xr-x 9 markw staff 288B Oct 14 13:49 OllamaCloud
12 -rw-r--r-- 1 markw staff 754B Oct 14 14:43 pyproject.toml
13 -rw-r--r-- 1 markw staff 1.1K Oct 14 14:47 README.md
14 drwxr-xr-x 4 markw staff 128B Oct 14 14:43 reasoning
15 -rw-r--r-- 1 markw staff 295B Aug 11 10:00 requirements.txt
16 drwxr-xr-x 4 markw staff 128B Aug 10 16:50 short_programs
17 drwxr-xr-x 7 markw staff 224B Oct 14 14:43 smolagents
18 drwxr-xr-x 4 markw staff 128B Oct 14 14:43 tool_examples
19 drwxr-xr-x 16 markw staff 512B Oct 14 14:46 tools
If you have not yet done so, please clone the repository for my Ollama book examples using:
1 git clone https://github.com/mark-watson/Ollama_in_Action_Book.git
Use of Python docstrings at runtime:
The Ollama Python SDK leverages docstrings as a crucial part of its runtime function calling mechanism. When defining functions that will be called by the LLM, the docstrings serve as structured metadata that gets parsed and converted into a JSON schema format. This schema describes the function’s parameters, their types, and expected behavior, which is then used by the model to understand how to properly invoke the function. The docstrings follow a specific format that includes parameter descriptions, type hints, and return value specifications, allowing the SDK to automatically generate the necessary function signatures that the LLM can understand and work with.
During runtime execution, when the LLM determines it needs to call a function, it first reads these docstring-derived schemas to understand the function’s interface. The SDK parses these docstrings using Python’s introspection capabilities (through the inspect module) and matches the LLM’s intended function call with the appropriate implementation. This system allows for a clean separation between the function’s implementation and its interface description, while maintaining human-readable documentation that serves as both API documentation and runtime function calling specifications. The docstring parsing is done lazily at runtime when the function is first accessed, and the resulting schema is typically cached to improve performance in subsequent calls.
Example Showing the Use of Tools Developed Later in this Chapter
The source file tool_examples/ollama_tools_examples.py contains simple examples of using the tools we develop later in this chapter. We will look at example code using the tools, then at the implementation of the tools. In this examples source file we first import these tools:
1 from tool_file_dir import list_directory
2 from tool_file_contents import read_file_contents
3 from tool_web_search import uri_to_markdown
4
5 import ollama
6
7 # Map function names to function objects
8 available_functions = {
9 'list_directory': list_directory,
10 'read_file_contents': read_file_contents,
11 'uri_to_markdown': uri_to_markdown,
12 }
13
14 # User prompt
15 user_prompt = "Please list the contents of the current directory, read the 'requirements.txt' file, and convert 'https://markwatson.com' to markdown."
16
17 # Initiate chat with the model
18 response = ollama.chat(
19 model='llama3.1',
20 messages=[{'role': 'user', 'content': user_prompt}],
21 tools=[list_directory, read_file_contents, uri_to_markdown],
22 )
23
24 # Process the model's response
25 for tool_call in response.message.tool_calls or []:
26 function_to_call = available_functions.get(tool_call.function.name)
27 print(f"{function_to_call=}")
28 if function_to_call:
29 result = function_to_call(**tool_call.function.arguments)
30 print(f"Output of {tool_call.function.name}: {result}")
31 else:
32 print(f"Function {tool_call.function.name} not found.")
This code demonstrates the integration of a local LLM with custom tool functions for file system operations and web content processing. It imports three utility functions for listing directories, reading file contents, and converting URLs to markdown, then maps them to a dictionary for easy access.
The main execution flow involves sending a user prompt to the Ollama hosted model (here we are using the small IBM “granite3-dense” model), which requests directory listing, file reading, and URL conversion operations. The code then processes the model’s response by iterating through any tool calls returned, executing the corresponding functions, and printing their results. Error handling is included for cases where requested functions aren’t found in the available tools dictionary.
Here is sample output from using these three tools (most output removed for brevity and blank lines added for clarity):
1 $ python ollama_tools_examples.py
2
3 function_to_call=<function read_file_contents at 0x104fac9a0>
4
5 Output of read_file_contents: {'content': 'git+https://github.com/mark-watson/Ollama_Tools.git\nrequests\nbeautifulsoup4\naisuite[ollama]\n\n', 'size': 93, 'exists': True, 'error': None}
6
7 function_to_call=<function list_directory at 0x1050389a0>
8 Output of list_directory: {'files': ['.git', '.gitignore', 'LICENSE', 'Makefile', 'README.md', 'ollama_tools_examples.py', 'requirements.txt', 'venv'], 'count': 8, 'current_dir': '/Users/markw/GITHUB/Ollama-book-examples', 'error': None}
9
10 function_to_call=<function uri_to_markdown at 0x105038c20>
11
12 Output of uri_to_markdown: {'content': 'Read My Blog on Blogspot\n\nRead My Blog on Substack\n\nConsulting\n\nFree Mentoring\n\nFun stuff\n\nMy Books\n\nOpen Source\n\n Privacy Policy\n\n# Mark Watson AI Practitioner and Consultant Specializing in Large Language Models, LangChain/Llama-Index Integrations, Deep Learning, and the Semantic Web\n\n# I am the author of 20+ books on Artificial Intelligence, Python, Common Lisp, Deep Learning, Haskell, Clojure, Java, Ruby, Hy language, and the Semantic Web. I have 55 US Patents.\n\nMy customer list includes: Google, Capital One, Babylist, Olive AI, CompassLabs, Mind AI, Disney, SAIC, Americast, PacBell, CastTV, Lutris Technology, Arctan Group, Sitescout.com, Embed.ly, and Webmind Corporation.
13
14 ...
15
16 # Fun stuff\n\nIn addition to programming and writing my hobbies are cooking,\n photography, hiking, travel, and playing the following musical instruments: guitar, didgeridoo, and American Indian flute:\n\nMy guitar playing: a boogie riff\n\nMy didgeridoo playing\n\nMy Spanish guitar riff\n\nPlaying with George (didgeridoo), Carol and Crystal (drums and percussion) and Mark (Indian flute)\n\n# Open Source\n\nMy Open Source projects are hosted on my github account so please check that out!
17
18 ...
19
20 Hosted on Cloudflare Pages\n\n © Mark Watson 1994-2024\n\nPrivacy Policy', 'title': 'Mark Watson: AI Practitioner and Author of 20+ AI Books | Mark Watson', 'error': None}
Please note that the text extracted from a web page is mostly plain text. Section heads are maintained but the format is changed to markdown format. In the last (edited for brevity) listing, the HTML H1 element with the text Fun Stuff is converted to markdown:
1 # Fun stuff
2
3 In addition to programming and writing my hobbies are cooking,
4 photography, hiking, travel, and playing the following musical
5 instruments: guitar, didgeridoo, and American Indian flute ...
You have now looked at example tool use. We will now implement the several tools in this chapter and the next. We will look at the first tool for reading and writing files in fine detail and then more briefly discuss the other tools in the https://github.com/mark-watson/Ollama_in_Action_Book repository in the source-code directory.
Tool for Reading and Writing File Contents
This tool is meant to be combined with other tools, for example a summarization tool and a file reading tool might be used to process a user prompt to summarize a specific local file on your laptop.
Here is the contents of tool utility tool_file_contents.py:
1 """
2 Provides functions for reading and writing file contents with proper error handling
3 """
4
5 from pathlib import Path
6 import json
7
8
9 def read_file_contents(file_path: str, encoding: str = "utf-8") -> str:
10 """
11 Reads contents from a file and returns the text
12
13 Args:
14 file_path (str): Path to the file to read
15 encoding (str): File encoding to use (default: utf-8)
16
17 Returns:
18 Contents of the file as a string
19 """
20 try:
21 path = Path(file_path)
22 if not path.exists():
23 return f"File not found: {file_path}"
24
25 with path.open("r", encoding=encoding) as f:
26 content = f.read()
27 return f"Contents of file '{file_path}' is:\n{content}\n"
28
29 except Exception as e:
30 return f"Error reading file '{file_path}' is: {str(e)}"
31
32
33 def write_file_contents(
34 file_path: str, content: str,
35 encoding: str = "utf-8",
36 mode: str = "w") -> str:
37 """
38 Writes content to a file and returns operation status
39
40 Args:
41 file_path (str): Path to the file to write
42 content (str): Content to write to the file
43 encoding (str): File encoding to use (default: utf-8)
44 mode (str): Write mode ('w' for write, 'a' for append)
45
46 Returns:
47 a message string
48 """
49 try:
50 path = Path(file_path)
51
52 # Create parent directories if they don't exist
53 path.parent.mkdir(parents=True, exist_ok=True)
54
55 with path.open(mode, encoding=encoding) as f:
56 bytes_written = f.write(content)
57
58 return f"File '{file_path}' written OK."
59
60 except Exception as e:
61 return f"Error writing file '{file_path}': {str(e)}"
62
63
64 # Function metadata for Ollama integration
65 read_file_contents.metadata = {
66 "name": "read_file_contents",
67 "description": "Reads contents from a file and returns the content as a string",
68 "parameters": {"file_path": "Path to the file to read"},
69 }
70
71 write_file_contents.metadata = {
72 "name": "write_file_contents",
73 "description": "Writes content to a file and returns operation status",
74 "parameters": {
75 "file_path": "Path to the file to write",
76 "content": "Content to write to the file",
77 "encoding": "File encoding (default: utf-8)",
78 "mode": 'Write mode ("w" for write, "a" for append)',
79 },
80 }
81
82 # Export the functions
83 __all__ = ["read_file_contents", "write_file_contents"]
read_file_contents
This function provides file reading capabilities with robust error handling with parameters:
- file_path (str): Path to the file to read
- encoding (str, optional): File encoding (defaults to “utf-8”)
Features:
- Uses pathlib.Path for cross-platform path handling
- Checks file existence before attempting to read
- Returns file contents with descriptive message
- Comprehensive error handling
LLM Integration:
- Includes metadata for function discovery
- Returns descriptive string responses instead of raising exceptions
write_file_contents
This function handles file writing operations with built-in safety features. The parameters are:
- file_path (str): Path to the file to write
- content (str): Content to write to the file
- encoding (str, optional): File encoding (defaults to “utf-8”)
- mode (str, optional): Write mode (‘w’ for write, ‘a’ for append)
Features:
- Automatically creates parent directories
- Supports write and append modes
- Uses context managers for safe file handling
- Returns operation status messages
LLM Integration:
- Includes detailed metadata for function calling
- Provides clear feedback about operations
Common Features of both functions:
- Type hints for better code clarity
- Detailed docstrings that are used at runtime in the tool/function calling code. The text in the doc strings is supplied as context to the LLM currently in use.
- Proper error handling
- UTF-8 default encoding
- Context managers for file operations
- Metadata for LLM function discovery
Design Benefits for LLM Integration: the utilities are optimized for LLM function calling by:
- Returning descriptive string responses
- Including metadata for function discovery
- Handling errors gracefully
- Providing clear operation feedback
- Using consistent parameter patterns
Tool for Getting File Directory Contents
This tool is similar to the last tool so here we just list the worker function from the file tool_file_dir.py:
1 def list_directory(pattern: str = "*", list_dots=None) -> Dict[str, Any]:
2 """
3 Lists files and directories in the current working directory
4
5 Args:
6 pattern (str): Glob pattern for filtering files (default: "*")
7
8 Returns:
9 string with directory name, followed by list of files in the directory
10 """
11 try:
12 current_dir = Path.cwd()
13 files = list(current_dir.glob(pattern))
14
15 # Convert Path objects to strings and sort
16 file_list = sorted([str(f.name) for f in files])
17
18 file_list = [file for file in file_list if not file.endswith("~")]
19 if not list_dots:
20 file_list = [file for file in file_list if not file.startswith(".")]
21
22 return f"Contents of current directory: [{', '.join(file_list)}]"
23
24 except Exception as e:
25 return f"Error listing directory: {str(e)}"
Tool for Accessing SQLite Databases Using Natural Language Queries
The example file tool_sqlite.py serves two purposes here:
- Test and example code: utility function _create_sample_data creates several database tables and the function main serves as an example program.
- The Python class definitions SQLiteTool and OllamaFunctionCaller are meant to be copied and used in your applications.
1 import sqlite3
2 import json
3 from typing import Dict, Any, List, Optional
4 import ollama
5 from functools import wraps
6 import re
7 from contextlib import contextmanager
8 from textwrap import dedent # for multi-line string literals
9
10 class DatabaseError(Exception):
11 """Custom exception for database operations"""
12 pass
13
14
15 def _create_sample_data(cursor): # Helper function to create sample data
16 """Create sample data for tables"""
17 sample_data = {
18 'example': [
19 ('Example 1', 10.5),
20 ('Example 2', 25.0)
21 ],
22 'users': [
23 ('Bob', 'bob@example.com'),
24 ('Susan', 'susan@test.net')
25 ],
26 'products': [
27 ('Laptop', 1200.00),
28 ('Keyboard', 75.50)
29 ]
30 }
31
32 for table, data in sample_data.items():
33 for record in data:
34 if table == 'example':
35 cursor.execute(
36 "INSERT INTO example (name, value) VALUES (?, ?) ON CONFLICT DO NOTHING",
37 record
38 )
39 elif table == 'users':
40 cursor.execute(
41 "INSERT INTO users (name, email) VALUES (?, ?) ON CONFLICT DO NOTHING",
42 record
43 )
44 elif table == 'products':
45 cursor.execute(
46 "INSERT INTO products (product_name, price) VALUES (?, ?) ON CONFLICT DO NOTHING",
47 record
48 )
49
50
51 class SQLiteTool:
52 _instance = None
53
54 def __new__(cls, *args, **kwargs):
55 if not isinstance(cls._instance, cls):
56 cls._instance = super(SQLiteTool, cls).__new__(cls)
57 return cls._instance
58
59 def __init__(self, default_db: str = "test.db"):
60 if hasattr(self, 'default_db'): # Skip initialization if already done
61 return
62 self.default_db = default_db
63 self._initialize_database()
64
65 @contextmanager
66 def get_connection(self):
67 """Context manager for database connections"""
68 conn = sqlite3.connect(self.default_db)
69 try:
70 yield conn
71 finally:
72 conn.close()
73
74 def _initialize_database(self):
75 """Initialize database with tables"""
76 tables = {
77 'example': """
78 CREATE TABLE IF NOT EXISTS example (
79 id INTEGER PRIMARY KEY,
80 name TEXT,
81 value REAL
82 );
83 """,
84 'users': """
85 CREATE TABLE IF NOT EXISTS users (
86 id INTEGER PRIMARY KEY,
87 name TEXT,
88 email TEXT UNIQUE
89 );
90 """,
91 'products': """
92 CREATE TABLE IF NOT EXISTS products (
93 id INTEGER PRIMARY KEY,
94 product_name TEXT,
95 price REAL
96 );
97 """
98 }
99
100 with self.get_connection() as conn:
101 cursor = conn.cursor()
102 for table_sql in tables.values():
103 cursor.execute(table_sql)
104 conn.commit()
105 _create_sample_data(cursor)
106 conn.commit()
107
108 def get_tables(self) -> List[str]:
109 """Get list of tables in the database"""
110 with self.get_connection() as conn:
111 cursor = conn.cursor()
112 cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
113 return [table[0] for table in cursor.fetchall()]
114
115 def get_table_schema(self, table_name: str) -> List[tuple]:
116 """Get schema for a specific table"""
117 with self.get_connection() as conn:
118 cursor = conn.cursor()
119 cursor.execute(f"PRAGMA table_info({table_name});")
120 return cursor.fetchall()
121
122 def execute_query(self, query: str) -> List[tuple]:
123 """Execute a SQL query and return results"""
124 with self.get_connection() as conn:
125 cursor = conn.cursor()
126 try:
127 cursor.execute(query)
128 return cursor.fetchall()
129 except sqlite3.Error as e:
130 raise DatabaseError(f"Query execution failed: {str(e)}")
131
132 class OllamaFunctionCaller:
133 def __init__(self, model: str = "llama3.2:latest"):
134 self.model = model
135 self.sqlite_tool = SQLiteTool()
136 self.function_definitions = self._get_function_definitions()
137
138 def _get_function_definitions(self) -> Dict:
139 return {
140 "query_database": {
141 "description": "Execute a SQL query on the database",
142 "parameters": {
143 "type": "object",
144 "properties": {
145 "query": {
146 "type": "string",
147 "description": "The SQL query to execute"
148 }
149 },
150 "required": ["query"]
151 }
152 },
153 "list_tables": {
154 "description": "List all tables in the database",
155 "parameters": {
156 "type": "object",
157 "properties": {}
158 }
159 }
160 }
161
162 def _generate_prompt(self, user_input: str) -> str:
163 prompt = dedent(f"""
164 You are a SQL assistant. Based on the user's request, generate a JSON response that calls the appropriate function.
165 Available functions: {json.dumps(self.function_definitions, indent=2)}
166
167 User request: {user_input}
168
169 Respond with a JSON object containing:
170 - "function": The function name to call
171 - "parameters": The parameters for the function
172
173 Response:
174 """).strip()
175 return prompt
176
177 def _parse_ollama_response(self, response: str) -> Dict[str, Any]:
178 try:
179 json_match = re.search(r'\{.*\}', response, re.DOTALL)
180 if not json_match:
181 raise ValueError("No valid JSON found in response")
182 return json.loads(json_match.group())
183 except json.JSONDecodeError as e:
184 raise ValueError(f"Invalid JSON in response: {str(e)}")
185
186 def process_request(self, user_input: str) -> Any:
187 try:
188 response = ollama.generate(model=self.model, prompt=self._generate_prompt(user_input))
189 function_call = self._parse_ollama_response(response.response)
190
191 if function_call["function"] == "query_database":
192 return self.sqlite_tool.execute_query(function_call["parameters"]["query"])
193 elif function_call["function"] == "list_tables":
194 return self.sqlite_tool.get_tables()
195 else:
196 raise ValueError(f"Unknown function: {function_call['function']}")
197 except Exception as e:
198 raise RuntimeError(f"Request processing failed: {str(e)}")
199
200 def main():
201 function_caller = OllamaFunctionCaller()
202 queries = [
203 "Show me all tables in the database",
204 "Get all users from the users table",
205 "What are the top 5 products by price?"
206 ]
207
208 for query in queries:
209 try:
210 print(f"\nQuery: {query}")
211 result = function_caller.process_request(query)
212 print(f"Result: {result}")
213 except Exception as e:
214 print(f"Error processing query: {str(e)}")
215
216 if __name__ == "__main__":
217 main()
This code provides a natural language interface for interacting with an SQLite database. It uses a combination of Python classes, SQLite, and Ollama for running a language model to interpret user queries and execute corresponding database operations. Below is a breakdown of the code:
- Database Setup and Error Handling: a custom exception class, DatabaseError, is defined to handle database-specific errors. The database is initialized with three tables: example, users, and products. These tables are populated with sample data for demonstration purposes.
- SQLiteTool Class: the SQLiteTool class is a singleton that manages all SQLite database operations. Key features include:–Singleton Pattern: Ensures only one instance of the class is created.–Database Initialization: Creates tables (example, users, products) if they do not already exist.–Sample Data: Populates the tables with predefined sample data.–Context Manager: Safely manages database connections using a context manager.
Utility Methods:
- get_tables: Retrieves a list of all tables in the database.
- get_table_schema: Retrieves the schema of a specific table.
- execute_query: Executes a given SQL query and returns the results.
Sample Data Creation:
A helper function, _create_sample_data, is used to populate the database with sample data. It inserts records into the example, users, and products tables. This ensures the database has some initial data for testing and demonstration.
OllamaFunctionCaller Class:
The OllamaFunctionCaller class acts as the interface between natural language queries and database operations. Key components include:
- Integration with Ollama LLM: Uses the Ollama language model to interpret natural language queries.
- Function Definitions: Defines two main functions:–query_database: Executes SQL queries on the database.–list_tables: Lists all tables in the database.
- Prompt Generation: Converts user input into a structured prompt for the language model.
- Response Parsing: Parses the language model’s response into a JSON object that specifies the function to call and its parameters.
- Request Processing: Executes the appropriate database operation based on the parsed response.
Function Definitions:
The OllamaFunctionCaller class defines two main functions that can be called based on user input:
- query_database: Executes a SQL query provided by the user and returns the results of the query.
- list_tables: Lists all tables in the database and is useful for understanding the database structure.
Request Processing Workflow:
The process_request method handles the entire workflow of processing a user query:
- Input: Takes a natural language query from the user.
- Prompt Generation: Converts the query into a structured prompt for the Ollama language model.
- Response Parsing: Parses the language model’s response into a JSON object.
- Function Execution: Calls the appropriate function (query_database or list_tables) based on the parsed response.
- Output: Returns the results of the database operation.
Main test/example function:
The main function demonstrates how the system works with sample queries. It initializes the OllamaFunctionCaller and processes a list of example queries, such as:
- “Show me all tables in the database.“
- “Get all users from the users table.“
- “What are the top 5 products by price?“
For each query, the system interprets the natural language input, executes the corresponding database operation, and prints the results.
Summary:
This code creates a natural language interface for interacting with an SQLite database. It works as follows:
- Database Management: The SQLiteTool class handles all database operations, including initialization, querying, and schema inspection.
- Natural Language Processing: The OllamaFunctionCaller uses the Ollama language model to interpret user queries and map them to database functions.
- Execution: The system executes the appropriate database operation and returns the results to the user.
This approach allows users to interact with the database using natural language instead of writing SQL queries directly, making it more user-friendly and accessible.
The output looks like this:
1 python /Users/markw/GITHUB/Ollama_in_Action_Book/source-code/tool_sqlite.py
2
3 Query: Show me all tables in the database
4 Result: ['example', 'users', 'products']
5
6 Query: Get all users from the users table
7 Result: [(1, 'Bob', 'bob@example.com'), (2, 'Susan', 'susan@test.net')]
8
9 Query: What are the top 5 products by price?
10 Result: [(1, 'Laptop', 1200.0), (3, 'Laptop', 1200.0), (2, 'Keyboard', 75.5), (4, 'Keyboard', 75.5)]
Tool for Summarizing Text
Tools that are used by LLMs can themselves also use other LLMs. The tool defined in the file tool_summarize_text.py might be triggered by a user prompt such as “summarize the text in local file test1.txt” of “summarize text from web page https://markwatson.com” where it is used by other tools like reading a local file contents, fetching a web page, etc.
We will start by looking at the file tool_summarize_text.py and then look at an example in example_chain_web_summary.py.
1 """
2 Summarize text
3 """
4
5 from ollama import ChatResponse
6 from ollama import chat
7
8
9 def summarize_text(text: str, context: str = "") -> str:
10 """
11 Summarizes text
12
13 Parameters:
14 text (str): text to summarize
15 context (str): another tool's output can at the application layer can be used set the context for this tool.
16
17 Returns:
18 a string of summarized text
19
20 """
21 prompt = "Summarize this text (and be concise), returning only the summary with NO OTHER COMMENTS:\n\n"
22 if len(text.strip()) < 50:
23 text = context
24 elif len(context) > 50:
25 prompt = f"Given this context:\n\n{context}\n\n" + prompt
26
27 summary: ChatResponse = chat(
28 model="llama3.2:latest",
29 messages=[
30 {"role": "system", "content": prompt},
31 {"role": "user", "content": text},
32 ],
33 )
34 return summary["message"]["content"]
35
36
37 # Function metadata for Ollama integration
38 summarize_text.metadata = {
39 "name": "summarize_text",
40 "description": "Summarizes input text",
41 "parameters": {"text": "string of text to summarize",
42 "context": "optional context string"},
43 }
44
45 # Export the functions
46 __all__ = ["summarize_text"]
This Python code implements a text summarization tool using the Ollama chat model. The core function summarize_text takes two parameters: the main text to summarize and an optional context string. The function operates by constructing a prompt that instructs the model to provide a concise summary without additional commentary. It includes an interesting logic where if the input text is very short (less than 50 characters), it defaults to using the context parameter instead. Additionally, if there’s substantial context provided (more than 50 characters), it prepends this context to the prompt. The function utilizes the Ollama chat model “llama3.2:latest” to generate the summary, structuring the request with a system message containing the prompt and a user message containing the text to be summarized. The program includes metadata for Ollama integration, specifying the function name, description, and parameter details, and exports the summarize_text function through all.
Here is an example of using this tool that you can find in the file example_chain_web_summary.py. Please note that this example also uses the web search tool that is discussed in the next section.
1 from tool_web_search import uri_to_markdown
2 from tool_summarize_text import summarize_text
3
4 from pprint import pprint
5
6 import ollama
7
8 # Map function names to function objects
9 available_functions = {
10 "uri_to_markdown": uri_to_markdown,
11 "summarize_text": summarize_text,
12 }
13
14 memory_context = ""
15 # User prompt
16 user_prompt = "Get the text of 'https://knowledgebooks.com' and then summarize the text."
17
18 # Initiate chat with the model
19 response = ollama.chat(
20 model='llama3.2:latest',
21 messages=[{"role": "user", "content": user_prompt}],
22 tools=[uri_to_markdown, summarize_text],
23 )
24
25 # Process the model's response
26
27 pprint(response.message.tool_calls)
28
29 for tool_call in response.message.tool_calls or []:
30 function_to_call = available_functions.get(tool_call.function.name)
31 print(
32 f"\n***** {function_to_call=}\n\nmemory_context[:70]:\n\n{memory_context[:70]}\n\n*****\n"
33 )
34 if function_to_call:
35 print()
36 if len(memory_context) > 10:
37 tool_call.function.arguments["context"] = memory_context
38 print("\n* * tool_call.function.arguments:\n")
39 pprint(tool_call.function.arguments)
40 print(f"Arguments for {function_to_call.__name__}: {tool_call.function.arguments}")
41 result = function_to_call(**tool_call.function.arguments) # , memory_context)
42 print(f"\n\n** Output of {tool_call.function.name}: {result}")
43 memory_context = memory_context + "\n\n" + result
44 else:
45 print(f"\n\n** Function {tool_call.function.name} not found.")
Here is the output edited for brevity:
1 python /Users/markw/GITHUB/Ollama_in_Action_Book/source-code/example_chain_web_summary.py
2 [ToolCall(function=Function(name='uri_to_markdown', arguments={'a_uri': 'https://knowledgebooks.com'})),
3 ToolCall(function=Function(name='summarize_text', arguments={'context': '', 'text': 'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}))]
4
5 ***** function_to_call=<function uri_to_markdown at 0x1047da200>
6
7 memory_context[:70]:
8
9
10
11 *****
12
13
14
15 * * tool_call.function.arguments:
16
17 {'a_uri': 'https://knowledgebooks.com'}
18 Arguments for uri_to_markdown: {'a_uri': 'https://knowledgebooks.com'}
19 INFO:httpx:HTTP Request: POST http://127.0.0.1:11434/api/chat "HTTP/1.1 200 OK"
20
21
22 ** Output of uri_to_markdown: Contents of URI https://knowledgebooks.com is:
23 # KnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web
24
25 KnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web
26
27 KnowledgeBooks.com
28
29 Knowledgebooks.com
30 a sole proprietorship company owned by Mark Watson
31 to promote Knowledge Management, Artificial Intelligence (AI), NLP, and Semantic Web technologies.
32
33 Site updated: December 1, 2018
34 With the experience of working on Machine Learning and Knowledge Graph applications for 30 years (at Google,
35 Capital One, SAIC, Compass Labs, etc.) I am now concerned that the leverage of deep learning and knowledge
36 representation technologies are controlled by a few large companies, mostly in China and the USA. I am proud
37 to be involved organizations like Ocean Protocol and Common Crawl that seek tp increase the availability of quality data
38 to individuals and smaller organizations.
39 Traditional knowledge management tools relied on structured data often stored in relational databases. Adding
40 new relations to this data would require changing the schemas used to store data which could negatively
41 impact exisiting systems that used that data. Relationships between data in traditional systems was
42 predefined by the structure/schema of stored data. With RDF and OWL based data modeling, relationships in
43 data are explicitly defined in the data itself. Semantic data is inherently flexible and extensible: adding
44 new data and relationships is less likely to break older systems that relied on the previous verisons of
45 data.
46 A complementary technology for knowledge management is the automated processing of unstructured text data
47 into semantic data using natural language processing (NLP) and statistical-base text analytics.
48 We will help you integrate semantic web and text analytics technologies into your organization by working
49 with your staff in a mentoring role and also help as needed with initial development. All for reasonable consulting rates
50 Knowledgebooks.com Technologies:
51
52 SAAS KnowledgeBooks Semantic NLP Portal (KBSportal.com) used for
53 in-house projects and available as a product to run on your servers.
54 Semantic Web Ontology design and development
55 Semantic Web application design and development using RDF data stores, PostgreSQL, and MongoDB.
56
57 Research
58 Natural Language Processing (NLP) using deep learning
59 Fusion of classic symbolic AI systems with deep learning models
60 Linked data, semantic web, and Ontology's
61 News ontology
62 Note: this ontology was created in 2004 using the Protege modeling tool.
63 About
64 KnowledgeBooks.com is owned as a sole proprietor business by Mark and Carol Watson.
65 Mark Watson is an author of 16 published books and a consultant specializing in the JVM platform
66 (Java, Scala, JRuby, and Clojure), artificial intelligence, and the Semantic Web.
67 Carol Watson helps prepare training data and serves as the editor for Mark's published books.
68 Privacy policy: this site collects no personal data or information on site visitors
69 Hosted on Cloudflare Pages.
70
71
72 ***** function_to_call=<function summarize_text at 0x107519260>
73
74 memory_context[:70]:
75
76
77
78 Contents of URI https://knowledgebooks.com is:
79 # KnowledgeBooks.com
80
81 *****
82
83
84
85 * * tool_call.function.arguments:
86
87 {'context': '\n'
88 '\n'
89 'Contents of URI https://knowledgebooks.com is:\n'
90 '# KnowledgeBooks.com - research on the Knowledge Management, and '
91 'the Semantic Web \n'
92 '\n'
93 'KnowledgeBooks.com - research on the Knowledge Management, and '
94 ...
95 'Carol Watson helps prepare training data and serves as the editor '
96 "for Mark's published books.\n"
97 'Privacy policy: this site collects no personal data or '
98 'information on site visitors\n'
99 'Hosted on Cloudflare Pages.\n',
100 'text': 'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}
101 Arguments for summarize_text: {'context': "\n\nContents of URI https://knowledgebooks.com is:\n# KnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web \n\nKnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web \n\nKnowledgeBooks.com \n\nKnowledgebooks.com \na sole proprietorship company owned by Mark Watson\nto promote Knowledge Management, Artificial Intelligence (AI), NLP, and Semantic Web technologies.
102
103 ...
104
105 \n\nResearch\nNatural Language Processing (NLP) using deep learning\nFusion of classic symbolic AI systems with deep learning models\nLinked data, semantic web, and Ontology's\nNews ontology\nNote: this ontology was created in 2004 using the Protege modeling tool.\nAbout\nKnowledgeBooks.com is owned as a sole proprietor business by Mark and Carol Watson.\nMark Watson is an author of 16 published books and a consultant specializing in the JVM platform\n (Java, Scala, JRuby, and Clojure), artificial intelligence, and the Semantic Web.\nCarol Watson helps prepare training data and serves as the editor for Mark's published books.\nPrivacy policy: this site collects no personal data or information on site visitors\nHosted on Cloudflare Pages.\n", 'text': 'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}
106
107
108 ** Output of summarize_text: # Knowledge Management and Semantic Web Research
109 ## About KnowledgeBooks.com
110 A sole proprietorship company by Mark Watson promoting AI, NLP, and Semantic Web technologies.
111 ### Technologies
112 - **SAAS KnowledgeBooks**: Semantic NLP Portal for in-house projects and product sales.
113 - **Semantic Web Development**: Ontology design and application development using RDF data stores.
114
115 ### Research Areas
116 - Natural Language Processing (NLP) with deep learning
117 - Fusion of symbolic AI systems with deep learning models
118 - Linked data, semantic web, and ontologies
Tool for Web Search and Fetching Web Pages
This code provides a set of functions for web searching and HTML content processing, with the main functions being uri_to_markdown, search_web, brave_search_summaries, and brave_search_text. The uri_to_markdown function fetches content from a given URI and converts HTML to markdown-style text, handling various edge cases and cleaning up the text by removing multiple blank lines and spaces while converting HTML entities. The search_web function is a placeholder that’s meant to be implemented with a preferred search API, while brave_search_summaries implements actual web searching using the Brave Search API, requiring an API key from the environment variables and returning structured results including titles, URLs, and descriptions. The brave_search_text function builds upon brave_search_summaries by fetching search results and then using uri_to_markdown to convert the content of each result URL to text, followed by summarizing the content using a separate summarize_text function. The code also includes utility functions like replace_html_tags_with_text which uses BeautifulSoup to strip HTML tags and return plain text, and includes proper error handling, logging, and type hints throughout. The module is designed to be integrated with Ollama and exports uri_to_markdown and search_web as its primary interfaces.
1 """
2 Provides functions for web searching and HTML to Markdown conversion
3 and for returning the contents of a URI as plain text (with minimal markdown)
4 """
5
6 from typing import Dict, Any
7 import requests
8 from bs4 import BeautifulSoup
9 import re
10 from urllib.parse import urlparse
11 import html
12 from ollama import chat
13 import json
14 from tool_summarize_text import summarize_text
15
16 import requests
17 import os
18 import logging
19 from pprint import pprint
20 from bs4 import BeautifulSoup
21
22 logging.basicConfig(level=logging.INFO)
23
24 api_key = os.environ.get("BRAVE_SEARCH_API_KEY")
25 if not api_key:
26 raise ValueError(
27 "API key not found. Set 'BRAVE_SEARCH_API_KEY' environment variable."
28 )
29
30
31 def replace_html_tags_with_text(html_string):
32 soup = BeautifulSoup(html_string, "html.parser")
33 return soup.get_text()
34
35
36 def uri_to_markdown(a_uri: str) -> Dict[str, Any]:
37 """
38 Fetches content from a URI and converts HTML to markdown-style text
39
40 Args:
41 a_uri (str): URI to fetch and convert
42
43 Returns:
44 web page text converted converted markdown content
45 """
46 try:
47 # Validate URI
48 parsed = urlparse(a_uri)
49 if not all([parsed.scheme, parsed.netloc]):
50 return f"Invalid URI: {a_uri}"
51
52 # Fetch content
53 headers = {
54 "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
55 }
56 response = requests.get(a_uri, headers=headers, timeout=10)
57 response.raise_for_status()
58
59 # Parse HTML
60 soup = BeautifulSoup(response.text, "html.parser")
61
62 # Get title
63 title = soup.title.string if soup.title else ""
64
65 # Get text and clean up
66 text = soup.get_text()
67
68 # Clean up the text
69 text = re.sub(r"\n\s*\n", "\n\n", text) # Remove multiple blank lines
70 text = re.sub(r" +", " ", text) # Remove multiple spaces
71 text = html.unescape(text) # Convert HTML entities
72 text = text.strip()
73
74 return f"Contents of URI {a_uri} is:\n# {title}\n\n{text}\n"
75
76 except requests.RequestException as e:
77 return f"Network error: {str(e)}"
78
79 except Exception as e:
80 return f"Error processing URI: {str(e)}"
81
82
83 def search_web(query: str, max_results: int = 5) -> str:
84 """
85 Performs a web search and returns results
86 Note: This is a placeholder. Implement with your preferred search API.
87
88 Args:
89 query (str): Search query
90 max_results (int): Maximum number of results to return
91
92 Returns:
93 Dict[str, Any]: Dictionary containing:
94 - 'results': List of search results
95 - 'count': Number of results found
96 - 'error': Error message if any, None otherwise
97 """
98
99 # Placeholder for search implementation
100 return {
101 "results": [],
102 "count": 0,
103 "error": "Web search not implemented. Please implement with your preferred search API.",
104 }
105
106
107 def brave_search_summaries(
108 query,
109 num_results=3,
110 url="https://api.search.brave.com/res/v1/web/search",
111 api_key=api_key,
112 ):
113 headers = {"X-Subscription-Token": api_key, "Content-Type": "application/json"}
114 params = {"q": query, "count": num_results}
115
116 response = requests.get(url, headers=headers, params=params)
117 ret = []
118
119 if response.status_code == 200:
120 search_results = response.json()
121 ret = [
122 {
123 "title": result.get("title"),
124 "url": result.get("url"),
125 "description": replace_html_tags_with_text(result.get("description")),
126 }
127 for result in search_results.get("web", {}).get("results", [])
128 ]
129 logging.info("Successfully retrieved results.")
130 else:
131 try:
132 error_info = response.json()
133 logging.error(f"Error {response.status_code}: {error_info.get('message')}")
134 except json.JSONDecodeError:
135 logging.error(f"Error {response.status_code}: {response.text}")
136
137 return ret
138
139 def brave_search_text(query, num_results=3):
140 summaries = brave_search_summaries(query, num_results)
141 ret = ""
142 for s in summaries:
143 url = s["url"]
144 text = uri_to_markdown(url)
145 summary = summarize_text(
146 f"Given the query:\n\n{query}\n\nthen, summarize text removing all material that is not relevant to the query and then be very concise for a very short summary:\n\n{text}\n"
147 )
148 ret += ret + summary
149 print("\n\n-----------------------------------")
150 return ret
151
152 # Function metadata for Ollama integration
153 uri_to_markdown.metadata = {
154 "name": "uri_to_markdown",
155 "description": "Converts web page content to markdown-style text",
156 "parameters": {"a_uri": "URI of the web page to convert"},
157 }
158
159 search_web.metadata = {
160 "name": "search_web",
161 "description": "Performs a web search and returns results",
162 "parameters": {
163 "query": "Search query",
164 "max_results": "Maximum number of results to return",
165 },
166 }
167
168 # Export the functions
169 __all__ = ["uri_to_markdown", "search_web"]
Tools Wrap Up
We have looked at the implementations and examples uses for several tools. In the next chapter we continue our study of tool use with the application of judging the accuracy of output generated of LLMs: basically LLMs judging the accuracy of other LLMs to reduce hallucinations, inaccurate output, etc.