1 #!/bin/zsh
 2 
 3 # Check if the directory has a requirements.txt file
 4 if [ ! -f "requirements.txt" ]; then
 5     echo "No requirements.txt file found in the current directory."
 6     exit 1
 7 fi
 8 
 9 # Create a virtual environment in the venv directory
10 python3 -m venv venv
11 
12 # Activate the virtual environment
13 source venv/bin/activate
14 
15 # Upgrade pip to the latest version
16 pip3 install --upgrade pip
17 
18 # Install dependencies from requirements.txt
19 pip3 install -r requirements.txt
20 
21 # Display installed packages
22 pip3 list
23 
24 echo "Virtual environment setup complete. Reactivate it with:"
25 echo "source venv/bin/activate"
26 echo ""

1 uv venv
2 source .venv/bin/activate
3 uv pip install -r requirements.txt
4 uv run ollama_tools_examples.py

 1 $ ollama run qwq:latest --format json
 2 >>> What are the capitals of Germany and France?
 3 { 
 4   "Germany": {
 5     "Capital": "Berlin",
 6     "Population": "83.2 million",
 7     "Area": "137,847 square miles"
 8   },
 9   "France": {
10     "Capital": "Paris",
11     "Population": "67.4 million",
12     "Area": "248,573 square miles"
13   }
14 }
15 
16 >>> /bye

1 $ ollama run llava:7b "Describe this image" markcarol.jpg
2  The image is a photograph featuring a man and a woman looking 
3 off-camera, towards the left side of the frame. In the background, there are indistinct objects that give the impression of an outdoor setting, possibly on a patio or deck.
4 
5 The focus and composition suggest that the photo was taken during the day in natural light. 

1 $ ollama run qwen2.5-coder:14b
2 >>> Send a message (/? for help)

1 $ ollama run qwq:latest       
2 >>>

 1 $ ollama run qwen2.5-coder:14b < tool_sqlite.py 
 2 This code defines a Python application that interacts with an SQLite database using SQL queries 
 3 generated by the Ollama language model. The application is structured around two main classes:
 4 
 5 1. **SQLiteTool**: Manages interactions with an SQLite database.
 6    - Handles creating sample data, managing database connections, and executing SQL queries.
 7    - Provides methods to list tables in the database, get table schemas, and execute arbitrary SQL 
 8 queries.
 9 
10 2. **OllamaFunctionCaller**: Acts as a bridge between user inputs and the SQLite database through the 
11 Ollama model.
12    - Defines functions that can be called by the Ollama model (e.g., querying the database or listing 
13 tables).
14    - Generates prompts for the Ollama model based on user input, parses the response to identify which 
15 function should be executed, and then calls the appropriate method in `SQLiteTool`.
16 
17 ...

 1 import ollama
 2 import base64
 3 
 4 def analyze_image(image_path: str, prompt: str) -> str:
 5     # Read and encode the image
 6     with open(image_path, 'rb') as img_file:
 7         image_data = base64.b64encode(img_file.read()).decode('utf-8')
 8 
 9     try:
10         # Create a stream of responses using the Ollama SDK
11         stream = ollama.generate(
12             model='llava:7b',
13             prompt=prompt,
14             images=[image_data],
15             stream=True
16         )
17 
18         # Accumulate the response
19         full_response = ""
20         for chunk in stream:
21             if 'response' in chunk:
22                 full_response += chunk['response']
23 
24         return full_response
25 
26     except Exception as e:
27         return f"Error processing image: {str(e)}"
28 
29 def main():
30     image_path = "data/sample.jpg"
31     prompt = "Please describe this image in detail, focusing on the actions of people in the picture."
32 
33     result = analyze_image(image_path, prompt)
34     print("Analysis Result:")
35     print(result)
36 
37 if __name__ == "__main__":
38     main()

1 Analysis Result:
2  The image appears to be a photograph taken inside a room that serves as a meeting or gaming space and capturing an indoor scene where five individuals are engaged in playing a tabletop card game. In the foreground, there is a table with a green surface and multiple items on it, including what looks like playing cards spread out in front of the people seated around it.
3 
4 The room has a comfortable and homely feel, with elements like a potted plant in the background on the left, which suggests that this might be a living room or a similar space repurposed for a group activity.

 1 import openai
 2 from typing import List, Dict
 3 
 4 class OllamaClient:
 5     def __init__(self, base_url: str = "http://localhost:11434/v1"):
 6         self.client = openai.OpenAI(
 7             base_url=base_url,
 8             api_key="fake-key"  # Ollama doesn't require authentication locally
 9         )
10 
11     def chat_with_context(
12         self,
13         system_context: str,
14         user_prompt: str,
15         model: str = "llama3.2:latest",
16         temperature: float = 0.7
17     ) -> str:
18         try:
19             messages = [
20                 {"role": "system", "content": system_context},
21                 {"role": "user", "content": user_prompt}
22             ]
23 
24             response = self.client.chat.completions.create(
25                 model=model,
26                 messages=messages,
27                 temperature=temperature,
28                 stream=False
29             )
30 
31             return response.choices[0].message.content
32 
33         except Exception as e:
34             return f"Error: {str(e)}"
35 
36     def chat_conversation(
37         self,
38         messages: List[Dict[str, str]],
39         model: str = "llama2"
40     ) -> str:
41         try:
42             response = self.client.chat.completions.create(
43                 model=model,
44                 messages=messages,
45                 stream=False
46             )
47 
48             return response.choices[0].message.content
49 
50         except Exception as e:
51             return f"Error: {str(e)}"
52 
53 def main():
54     # Initialize the client
55     client = OllamaClient()
56 
57     # Example 1: Single interaction with context
58     system_context = """You are a helpful AI assistant with expertise in 
59     programming and technology. Provide clear, concise answers."""
60 
61     user_prompt = "Explain the concept of recursion in programming."
62 
63     response = client.chat_with_context(
64         system_context=system_context,
65         user_prompt=user_prompt,
66         model="llama3.2:latest",
67         temperature=0.7
68     )
69 
70     print("Response with context:")
71     print(response)
72     print("\n" + "="*50 + "\n")
73 
74     # Example 2: Multi-turn conversation
75     conversation = [
76         {"role": "system", "content": "You are a helpful AI assistant."},
77         {"role": "user", "content": "What is machine learning?"},
78         {"role": "assistant", "content": "Machine learning is a subset of AI that enables systems to learn from data."},
79         {"role": "user", "content": "Can you give me a simple example?"}
80     ]
81 
82     response = client.chat_conversation(
83         messages=conversation,
84         model="llama3.2:latest"
85     )
86 
87     print("Conversation response:")
88     print(response)
89 
90 if __name__ == "__main__":
91     main()

 1 Response with context:
 2 Recursion is a fundamental concept in programming that allows a function or method to call itself repeatedly until it reaches a base case that stops the recursion.
 3 
 4 **What is Recursion?**
 5 
 6 In simple terms, recursion is a programming technique where a function invokes itself as a sub-procedure, repeating the same steps until it solves a problem ...
 7 
 8 **Key Characteristics of Recursion:**
 9 
10 1. **Base case**: A trivial case that stops the recursion.
11 2. **Recursive call**: The function calls itself with new input or parameters.
12 3. **Termination condition**: When the base case is reached, the recursion terminates.
13 
14 **How Recursion Works:**
15 
16 Here's an example to illustrate recursion:
17 
18 Imagine you have a recursive function `factorial(n)` that calculates the factorial of a number `n`. The function works as follows:
19 
20 1. If `n` is 0 or 1 (base case), return 1.
21 2. Otherwise, call itself with `n-1` as input and multiply the result by `n`.
22 3. Repeat step 2 until `n` reaches 0 or 1.
23 
24 Here's a simple recursive implementation in Python ...
25 
26 **Benefits of Recursion:**
27 
28 Recursion offers several advantages:
29 
30 * **Divide and Conquer**: Break down complex problems into smaller, more manageable sub-problems.
31 * **Elegant Code**: Recursive solutions can be concise and easy to understand.
32 * **Efficient**: Recursion can avoid explicit loops and reduce memory usage.
33 ...
34 
35 In summary, recursion is a powerful technique that allows functions to call themselves repeatedly until they solve a problem. By understanding the basics of recursion and its applications, you can write more efficient and elegant code for complex problems.
36 
37 ==================================================
38 
39 Conversation response:
40 A simple example of machine learning is a spam filter.
41 
42 Imagine we have a system that scans emails and identifies whether they are spam or not. The system learns to classify these emails as spam or not based on the following steps:
43 
44 1. Initially, it receives a large number of labeled data points (e.g., 1000 emails), where some emails are marked as "spam" and others as "not spam".
45 2. The system analyzes these examples to identify patterns and features that distinguish spam emails from non-spam messages.
46 3. Once the patterns are identified, the system can use them to classify new, unseen email data (e.g., a new email) as either spam or not spam.
47 
48 Over time, the system becomes increasingly accurate in its classification because it has learned from the examples and improvements have been made. This is essentially an example of supervised machine learning, where the system learns by being trained on labeled data.

1 https://github.com/mark-watson/Ollama_in_Action_Book/source-code $ ls tool*
2 tool_file_contents.py   tool_llm_eval.py    tool_web_search.py
3 tool_file_dir.py    tool_sqlite.py
4 tool_judge_results.py   tool_summarize_text.py

1 git clone https://github.com/mark-watson/Ollama_in_Action_Book.git

 1 from tool_file_dir import list_directory
 2 from tool_file_contents import read_file_contents
 3 from tool_web_search import uri_to_markdown
 4 
 5 import ollama
 6 
 7 # Map function names to function objects
 8 available_functions = {
 9     'list_directory': list_directory,
10     'read_file_contents': read_file_contents,
11     'uri_to_markdown': uri_to_markdown,
12 }
13 
14 # User prompt
15 user_prompt = "Please list the contents of the current directory, read the 'requirements.txt' file, and convert 'https://markwatson.com' to markdown."
16 
17 # Initiate chat with the model
18 response = ollama.chat(
19     model='llama3.1',
20     messages=[{'role': 'user', 'content': user_prompt}],
21     tools=[list_directory, read_file_contents, uri_to_markdown],
22 )
23 
24 # Process the model's response
25 for tool_call in response.message.tool_calls or []:
26     function_to_call = available_functions.get(tool_call.function.name)
27     print(f"{function_to_call=}")
28     if function_to_call:
29         result = function_to_call(**tool_call.function.arguments)
30         print(f"Output of {tool_call.function.name}: {result}")
31     else:
32         print(f"Function {tool_call.function.name} not found.")

 1 $ python ollama_tools_examples.py
 2 
 3 function_to_call=<function read_file_contents at 0x104fac9a0>
 4 
 5 Output of read_file_contents: {'content': 'git+https://github.com/mark-watson/Ollama_Tools.git\nrequests\nbeautifulsoup4\naisuite[ollama]\n\n', 'size': 93, 'exists': True, 'error': None}
 6 
 7 function_to_call=<function list_directory at 0x1050389a0>
 8 Output of list_directory: {'files': ['.git', '.gitignore', 'LICENSE', 'Makefile', 'README.md', 'ollama_tools_examples.py', 'requirements.txt', 'venv'], 'count': 8, 'current_dir': '/Users/markw/GITHUB/Ollama-book-examples', 'error': None}
 9 
10 function_to_call=<function uri_to_markdown at 0x105038c20>
11 
12 Output of uri_to_markdown: {'content': 'Read My Blog on Blogspot\n\nRead My Blog on Substack\n\nConsulting\n\nFree Mentoring\n\nFun stuff\n\nMy Books\n\nOpen Source\n\n Privacy Policy\n\n# Mark Watson AI Practitioner and Consultant Specializing in Large Language Models, LangChain/Llama-Index Integrations, Deep Learning, and the Semantic Web\n\n# I am the author of 20+ books on Artificial Intelligence, Python, Common Lisp, Deep Learning, Haskell, Clojure, Java, Ruby, Hy language, and the Semantic Web. I have 55 US Patents.\n\nMy customer list includes: Google, Capital One, Babylist, Olive AI, CompassLabs, Mind AI, Disney, SAIC, Americast, PacBell, CastTV, Lutris Technology, Arctan Group, Sitescout.com, Embed.ly, and Webmind Corporation.
13 
14  ...
15 
16  # Fun stuff\n\nIn addition to programming and writing my hobbies are cooking,\n photography, hiking, travel, and playing the following musical instruments: guitar, didgeridoo, and American Indian flute:\n\nMy guitar playing: a boogie riff\n\nMy didgeridoo playing\n\nMy Spanish guitar riff\n\nPlaying with George (didgeridoo), Carol and Crystal (drums and percussion) and Mark (Indian flute)\n\n# Open Source\n\nMy Open Source projects are hosted on my github account so please check that out!
17 
18  ...
19 
20 Hosted on Cloudflare Pages\n\n © Mark Watson 1994-2024\n\nPrivacy Policy', 'title': 'Mark Watson: AI Practitioner and Author of 20+ AI Books | Mark Watson', 'error': None}

1 # Fun stuff
2 
3 In addition to programming and writing my hobbies are cooking,
4 photography, hiking, travel, and playing the following musical
5 instruments: guitar, didgeridoo, and American Indian flute ...

 1 """
 2 Provides functions for reading and writing file contents with proper error handling
 3 """
 4 
 5 from pathlib import Path
 6 import json
 7 
 8 
 9 def read_file_contents(file_path: str, encoding: str = "utf-8") -> str:
10     """
11     Reads contents from a file and returns the text
12 
13     Args:
14         file_path (str): Path to the file to read
15         encoding (str): File encoding to use (default: utf-8)
16 
17     Returns:
18         Contents of the file as a string
19     """
20     try:
21         path = Path(file_path)
22         if not path.exists():
23             return f"File not found: {file_path}"
24 
25         with path.open("r", encoding=encoding) as f:
26             content = f.read()
27             return f"Contents of file '{file_path}' is:\n{content}\n"
28 
29     except Exception as e:
30         return f"Error reading file '{file_path}' is: {str(e)}"
31 
32 
33 def write_file_contents(
34         file_path: str, content: str,
35         encoding: str = "utf-8",
36         mode: str = "w") -> str:
37     """
38     Writes content to a file and returns operation status
39 
40     Args:
41         file_path (str): Path to the file to write
42         content (str): Content to write to the file
43         encoding (str): File encoding to use (default: utf-8)
44         mode (str): Write mode ('w' for write, 'a' for append)
45 
46     Returns:
47         a message string
48     """
49     try:
50         path = Path(file_path)
51 
52         # Create parent directories if they don't exist
53         path.parent.mkdir(parents=True, exist_ok=True)
54 
55         with path.open(mode, encoding=encoding) as f:
56             bytes_written = f.write(content)
57 
58         return f"File '{file_path}' written OK."
59 
60     except Exception as e:
61         return f"Error writing file '{file_path}': {str(e)}"
62 
63 
64 # Function metadata for Ollama integration
65 read_file_contents.metadata = {
66     "name": "read_file_contents",
67     "description": "Reads contents from a file and returns the content as a string",
68     "parameters": {"file_path": "Path to the file to read"},
69 }
70 
71 write_file_contents.metadata = {
72     "name": "write_file_contents",
73     "description": "Writes content to a file and returns operation status",
74     "parameters": {
75         "file_path": "Path to the file to write",
76         "content": "Content to write to the file",
77         "encoding": "File encoding (default: utf-8)",
78         "mode": 'Write mode ("w" for write, "a" for append)',
79     },
80 }
81 
82 # Export the functions
83 __all__ = ["read_file_contents", "write_file_contents"]

 1 def list_directory(pattern: str = "*", list_dots=None) -> Dict[str, Any]:
 2     """
 3     Lists files and directories in the current working directory
 4 
 5     Args:
 6         pattern (str): Glob pattern for filtering files (default: "*")
 7 
 8     Returns:
 9         string with directory name, followed by list of files in the directory
10     """
11     try:
12         current_dir = Path.cwd()
13         files = list(current_dir.glob(pattern))
14 
15         # Convert Path objects to strings and sort
16         file_list = sorted([str(f.name) for f in files])
17 
18         file_list = [file for file in file_list if not file.endswith("~")]
19         if not list_dots:
20             file_list = [file for file in file_list if not file.startswith(".")]
21 
22         return f"Contents of current directory: [{', '.join(file_list)}]"
23 
24     except Exception as e:
25         return f"Error listing directory: {str(e)}"

  1 import sqlite3
  2 import json
  3 from typing import Dict, Any, List, Optional
  4 import ollama
  5 from functools import wraps
  6 import re
  7 from contextlib import contextmanager
  8 from textwrap import dedent # for multi-line string literals
  9 
 10 class DatabaseError(Exception):
 11     """Custom exception for database operations"""
 12     pass
 13 
 14 
 15 def _create_sample_data(cursor):  # Helper function to create sample data
 16     """Create sample data for tables"""
 17     sample_data = {
 18         'example': [
 19             ('Example 1', 10.5),
 20             ('Example 2', 25.0)
 21         ],
 22         'users': [
 23             ('Bob', 'bob@example.com'),
 24             ('Susan', 'susan@test.net')
 25         ],
 26         'products': [
 27             ('Laptop', 1200.00),
 28             ('Keyboard', 75.50)
 29         ]
 30     }
 31 
 32     for table, data in sample_data.items():
 33         for record in data:
 34             if table == 'example':
 35                 cursor.execute(
 36                     "INSERT INTO example (name, value) VALUES (?, ?) ON CONFLICT DO NOTHING",
 37                     record
 38                 )
 39             elif table == 'users':
 40                 cursor.execute(
 41                     "INSERT INTO users (name, email) VALUES (?, ?) ON CONFLICT DO NOTHING",
 42                     record
 43                 )
 44             elif table == 'products':
 45                 cursor.execute(
 46                     "INSERT INTO products (product_name, price) VALUES (?, ?) ON CONFLICT DO NOTHING",
 47                     record
 48                 )
 49 
 50 
 51 class SQLiteTool:
 52     _instance = None
 53 
 54     def __new__(cls, *args, **kwargs):
 55         if not isinstance(cls._instance, cls):
 56             cls._instance = super(SQLiteTool, cls).__new__(cls)
 57         return cls._instance
 58 
 59     def __init__(self, default_db: str = "test.db"):
 60         if hasattr(self, 'default_db'):  # Skip initialization if already done
 61             return
 62         self.default_db = default_db
 63         self._initialize_database()
 64 
 65     @contextmanager
 66     def get_connection(self):
 67         """Context manager for database connections"""
 68         conn = sqlite3.connect(self.default_db)
 69         try:
 70             yield conn
 71         finally:
 72             conn.close()
 73 
 74     def _initialize_database(self):
 75         """Initialize database with tables"""
 76         tables = {
 77             'example': """
 78                 CREATE TABLE IF NOT EXISTS example (
 79                     id INTEGER PRIMARY KEY,
 80                     name TEXT,
 81                     value REAL
 82                 );
 83             """,
 84             'users': """
 85                 CREATE TABLE IF NOT EXISTS users (
 86                     id INTEGER PRIMARY KEY,
 87                     name TEXT,
 88                     email TEXT UNIQUE
 89                 );
 90             """,
 91             'products': """
 92                 CREATE TABLE IF NOT EXISTS products (
 93                     id INTEGER PRIMARY KEY,
 94                     product_name TEXT,
 95                     price REAL
 96                 );
 97             """
 98         }
 99 
100         with self.get_connection() as conn:
101             cursor = conn.cursor()
102             for table_sql in tables.values():
103                 cursor.execute(table_sql)
104             conn.commit()
105             _create_sample_data(cursor)
106             conn.commit()
107 
108     def get_tables(self) -> List[str]:
109         """Get list of tables in the database"""
110         with self.get_connection() as conn:
111             cursor = conn.cursor()
112             cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
113             return [table[0] for table in cursor.fetchall()]
114 
115     def get_table_schema(self, table_name: str) -> List[tuple]:
116         """Get schema for a specific table"""
117         with self.get_connection() as conn:
118             cursor = conn.cursor()
119             cursor.execute(f"PRAGMA table_info({table_name});")
120             return cursor.fetchall()
121 
122     def execute_query(self, query: str) -> List[tuple]:
123         """Execute a SQL query and return results"""
124         with self.get_connection() as conn:
125             cursor = conn.cursor()
126             try:
127                 cursor.execute(query)
128                 return cursor.fetchall()
129             except sqlite3.Error as e:
130                 raise DatabaseError(f"Query execution failed: {str(e)}")
131 
132 class OllamaFunctionCaller:
133     def __init__(self, model: str = "llama3.2:latest"):
134         self.model = model
135         self.sqlite_tool = SQLiteTool()
136         self.function_definitions = self._get_function_definitions()
137 
138     def _get_function_definitions(self) -> Dict:
139         return {
140             "query_database": {
141                 "description": "Execute a SQL query on the database",
142                 "parameters": {
143                     "type": "object",
144                     "properties": {
145                         "query": {
146                             "type": "string",
147                             "description": "The SQL query to execute"
148                         }
149                     },
150                     "required": ["query"]
151                 }
152             },
153             "list_tables": {
154                 "description": "List all tables in the database",
155                 "parameters": {
156                     "type": "object",
157                     "properties": {}
158                 }
159             }
160         }
161 
162     def _generate_prompt(self, user_input: str) -> str:
163         prompt = dedent(f"""
164             You are a SQL assistant. Based on the user's request, generate a JSON response that calls the appropriate function.
165             Available functions: {json.dumps(self.function_definitions, indent=2)}
166 
167             User request: {user_input}
168 
169             Respond with a JSON object containing:
170             - "function": The function name to call
171             - "parameters": The parameters for the function
172 
173             Response:
174         """).strip()
175         return prompt
176 
177     def _parse_ollama_response(self, response: str) -> Dict[str, Any]:
178         try:
179             json_match = re.search(r'\{.*\}', response, re.DOTALL)
180             if not json_match:
181                 raise ValueError("No valid JSON found in response")
182             return json.loads(json_match.group())
183         except json.JSONDecodeError as e:
184             raise ValueError(f"Invalid JSON in response: {str(e)}")
185 
186     def process_request(self, user_input: str) -> Any:
187         try:
188             response = ollama.generate(model=self.model, prompt=self._generate_prompt(user_input))
189             function_call = self._parse_ollama_response(response.response)
190 
191             if function_call["function"] == "query_database":
192                 return self.sqlite_tool.execute_query(function_call["parameters"]["query"])
193             elif function_call["function"] == "list_tables":
194                 return self.sqlite_tool.get_tables()
195             else:
196                 raise ValueError(f"Unknown function: {function_call['function']}")
197         except Exception as e:
198             raise RuntimeError(f"Request processing failed: {str(e)}")
199 
200 def main():
201     function_caller = OllamaFunctionCaller()
202     queries = [
203         "Show me all tables in the database",
204         "Get all users from the users table",
205         "What are the top 5 products by price?"
206     ]
207 
208     for query in queries:
209         try:
210             print(f"\nQuery: {query}")
211             result = function_caller.process_request(query)
212             print(f"Result: {result}")
213         except Exception as e:
214             print(f"Error processing query: {str(e)}")
215 
216 if __name__ == "__main__":
217     main()

 1 python /Users/markw/GITHUB/Ollama_in_Action_Book/source-code/tool_sqlite.py 
 2 
 3 Query: Show me all tables in the database
 4 Result: ['example', 'users', 'products']
 5 
 6 Query: Get all users from the users table
 7 Result: [(1, 'Bob', 'bob@example.com'), (2, 'Susan', 'susan@test.net')]
 8 
 9 Query: What are the top 5 products by price?
10 Result: [(1, 'Laptop', 1200.0), (3, 'Laptop', 1200.0), (2, 'Keyboard', 75.5), (4, 'Keyboard', 75.5)]

 1 """
 2 Summarize text
 3 """
 4 
 5 from ollama import ChatResponse
 6 from ollama import chat
 7 
 8 
 9 def summarize_text(text: str, context: str = "") -> str:
10     """
11     Summarizes text
12 
13     Parameters:
14         text (str): text to summarize
15         context (str): another tool's output can at the application layer can be used set the context for this tool.
16 
17     Returns:
18         a string of summarized text
19 
20     """
21     prompt = "Summarize this text (and be concise), returning only the summary with NO OTHER COMMENTS:\n\n"
22     if len(text.strip()) < 50:
23         text = context
24     elif len(context) > 50:
25         prompt = f"Given this context:\n\n{context}\n\n" + prompt
26 
27     summary: ChatResponse = chat(
28         model="llama3.2:latest",
29         messages=[
30             {"role": "system", "content": prompt},
31             {"role": "user", "content": text},
32         ],
33     )
34     return summary["message"]["content"]
35 
36 
37 # Function metadata for Ollama integration
38 summarize_text.metadata = {
39     "name": "summarize_text",
40     "description": "Summarizes input text",
41     "parameters": {"text": "string of text to summarize",
42                    "context": "optional context string"},
43 }
44 
45 # Export the functions
46 __all__ = ["summarize_text"]

 1 from tool_web_search import uri_to_markdown
 2 from tool_summarize_text import summarize_text
 3 
 4 from pprint import pprint
 5 
 6 import ollama
 7 
 8 # Map function names to function objects
 9 available_functions = {
10     "uri_to_markdown": uri_to_markdown,
11     "summarize_text": summarize_text,
12 }
13 
14 memory_context = ""
15 # User prompt
16 user_prompt = "Get the text of 'https://knowledgebooks.com' and then summarize the text."
17 
18 # Initiate chat with the model
19 response = ollama.chat(
20     model='llama3.2:latest',
21     messages=[{"role": "user", "content": user_prompt}],
22     tools=[uri_to_markdown, summarize_text],
23 )
24 
25 # Process the model's response
26 
27 pprint(response.message.tool_calls)
28 
29 for tool_call in response.message.tool_calls or []:
30     function_to_call = available_functions.get(tool_call.function.name)
31     print(
32         f"\n***** {function_to_call=}\n\nmemory_context[:70]:\n\n{memory_context[:70]}\n\n*****\n"
33     )
34     if function_to_call:
35         print()
36         if len(memory_context) > 10:
37             tool_call.function.arguments["context"] = memory_context
38         print("\n* * tool_call.function.arguments:\n")
39         pprint(tool_call.function.arguments)
40         print(f"Arguments for {function_to_call.__name__}: {tool_call.function.arguments}")
41         result = function_to_call(**tool_call.function.arguments)  # , memory_context)
42         print(f"\n\n** Output of {tool_call.function.name}: {result}")
43         memory_context = memory_context + "\n\n" + result
44     else:
45         print(f"\n\n** Function {tool_call.function.name} not found.")

  1 python /Users/markw/GITHUB/Ollama_in_Action_Book/source-code/example_chain_web_summary.py 
  2 [ToolCall(function=Function(name='uri_to_markdown', arguments={'a_uri': 'https://knowledgebooks.com'})),
  3  ToolCall(function=Function(name='summarize_text', arguments={'context': '', 'text': 'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}))]
  4 
  5 ***** function_to_call=<function uri_to_markdown at 0x1047da200>
  6 
  7 memory_context[:70]:
  8 
  9 
 10 
 11 *****
 12 
 13 
 14 
 15 * * tool_call.function.arguments:
 16 
 17 {'a_uri': 'https://knowledgebooks.com'}
 18 Arguments for uri_to_markdown: {'a_uri': 'https://knowledgebooks.com'}
 19 INFO:httpx:HTTP Request: POST http://127.0.0.1:11434/api/chat "HTTP/1.1 200 OK"
 20 
 21 
 22 ** Output of uri_to_markdown: Contents of URI https://knowledgebooks.com is:
 23 # KnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web 
 24 
 25 KnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web 
 26 
 27 KnowledgeBooks.com 
 28 
 29 Knowledgebooks.com 
 30 a sole proprietorship company owned by Mark Watson
 31 to promote Knowledge Management, Artificial Intelligence (AI), NLP, and Semantic Web technologies.
 32 
 33 Site updated: December 1, 2018
 34 With the experience of working on Machine Learning and Knowledge Graph applications for 30 years (at Google,
 35  Capital One, SAIC, Compass Labs, etc.) I am now concerned that the leverage of deep learning and knowledge
 36  representation technologies are controlled by a few large companies, mostly in China and the USA. I am proud
 37  to be involved organizations like Ocean Protocol and Common Crawl that seek tp increase the availability of quality data
 38  to individuals and smaller organizations.
 39 Traditional knowledge management tools relied on structured data often stored in relational databases. Adding
 40  new relations to this data would require changing the schemas used to store data which could negatively
 41  impact exisiting systems that used that data. Relationships between data in traditional systems was
 42  predefined by the structure/schema of stored data. With RDF and OWL based data modeling, relationships in
 43  data are explicitly defined in the data itself. Semantic data is inherently flexible and extensible: adding
 44  new data and relationships is less likely to break older systems that relied on the previous verisons of
 45  data.
 46 A complementary technology for knowledge management is the automated processing of unstructured text data
 47  into semantic data using natural language processing (NLP) and statistical-base text analytics.
 48 We will help you integrate semantic web and text analytics technologies into your organization by working
 49  with your staff in a mentoring role and also help as needed with initial development. All for reasonable consulting rates
 50 Knowledgebooks.com Technologies:
 51 
 52 SAAS KnowledgeBooks Semantic NLP Portal (KBSportal.com) used for
 53  in-house projects and available as a product to run on your servers.
 54 Semantic Web Ontology design and development
 55 Semantic Web application design and development using RDF data stores, PostgreSQL, and MongoDB.
 56 
 57 Research
 58 Natural Language Processing (NLP) using deep learning
 59 Fusion of classic symbolic AI systems with deep learning models
 60 Linked data, semantic web, and Ontology's
 61 News ontology
 62 Note: this ontology was created in 2004 using the Protege modeling tool.
 63 About
 64 KnowledgeBooks.com is owned as a sole proprietor business by Mark and Carol Watson.
 65 Mark Watson is an author of 16 published books and a consultant specializing in the JVM platform
 66  (Java, Scala, JRuby, and Clojure), artificial intelligence, and the Semantic Web.
 67 Carol Watson helps prepare training data and serves as the editor for Mark's published books.
 68 Privacy policy: this site collects no personal data or information on site visitors
 69 Hosted on Cloudflare Pages.
 70 
 71 
 72 ***** function_to_call=<function summarize_text at 0x107519260>
 73 
 74 memory_context[:70]:
 75 
 76 
 77 
 78 Contents of URI https://knowledgebooks.com is:
 79 # KnowledgeBooks.com 
 80 
 81 *****
 82 
 83 
 84 
 85 * * tool_call.function.arguments:
 86 
 87 {'context': '\n'
 88             '\n'
 89             'Contents of URI https://knowledgebooks.com is:\n'
 90             '# KnowledgeBooks.com - research on the Knowledge Management, and '
 91             'the Semantic Web \n'
 92             '\n'
 93             'KnowledgeBooks.com - research on the Knowledge Management, and '
 94 ...
 95             'Carol Watson helps prepare training data and serves as the editor '
 96             "for Mark's published books.\n"
 97             'Privacy policy: this site collects no personal data or '
 98             'information on site visitors\n'
 99             'Hosted on Cloudflare Pages.\n',
100  'text': 'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}
101 Arguments for summarize_text: {'context': "\n\nContents of URI https://knowledgebooks.com is:\n# KnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web \n\nKnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web \n\nKnowledgeBooks.com \n\nKnowledgebooks.com \na sole proprietorship company owned by Mark Watson\nto promote Knowledge Management, Artificial Intelligence (AI), NLP, and Semantic Web technologies.
102 
103 ...
104 
105 \n\nResearch\nNatural Language Processing (NLP) using deep learning\nFusion of classic symbolic AI systems with deep learning models\nLinked data, semantic web, and Ontology's\nNews ontology\nNote: this ontology was created in 2004 using the Protege modeling tool.\nAbout\nKnowledgeBooks.com is owned as a sole proprietor business by Mark and Carol Watson.\nMark Watson is an author of 16 published books and a consultant specializing in the JVM platform\n (Java, Scala, JRuby, and Clojure), artificial intelligence, and the Semantic Web.\nCarol Watson helps prepare training data and serves as the editor for Mark's published books.\nPrivacy policy: this site collects no personal data or information on site visitors\nHosted on Cloudflare Pages.\n", 'text': 'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}
106 
107 
108 ** Output of summarize_text: # Knowledge Management and Semantic Web Research
109 ## About KnowledgeBooks.com
110 A sole proprietorship company by Mark Watson promoting AI, NLP, and Semantic Web technologies.
111 ### Technologies
112 - **SAAS KnowledgeBooks**: Semantic NLP Portal for in-house projects and product sales.
113 - **Semantic Web Development**: Ontology design and application development using RDF data stores.
114 
115 ### Research Areas
116 - Natural Language Processing (NLP) with deep learning
117 - Fusion of symbolic AI systems with deep learning models
118 - Linked data, semantic web, and ontologies

  1 """
  2 Provides functions for web searching and HTML to Markdown conversion
  3 and for returning the contents of a URI as plain text (with minimal markdown)
  4 """
  5 
  6 from typing import Dict, Any
  7 import requests
  8 from bs4 import BeautifulSoup
  9 import re
 10 from urllib.parse import urlparse
 11 import html
 12 from ollama import chat
 13 import json
 14 from tool_summarize_text import summarize_text
 15 
 16 import requests
 17 import os
 18 import logging
 19 from pprint import pprint
 20 from bs4 import BeautifulSoup
 21 
 22 logging.basicConfig(level=logging.INFO)
 23 
 24 api_key = os.environ.get("BRAVE_SEARCH_API_KEY")
 25 if not api_key:
 26     raise ValueError(
 27         "API key not found. Set 'BRAVE_SEARCH_API_KEY' environment variable."
 28     )
 29 
 30 
 31 def replace_html_tags_with_text(html_string):
 32     soup = BeautifulSoup(html_string, "html.parser")
 33     return soup.get_text()
 34 
 35 
 36 def uri_to_markdown(a_uri: str) -> Dict[str, Any]:
 37     """
 38     Fetches content from a URI and converts HTML to markdown-style text
 39 
 40     Args:
 41         a_uri (str): URI to fetch and convert
 42 
 43     Returns:
 44         web page text converted converted markdown content
 45     """
 46     try:
 47         # Validate URI
 48         parsed = urlparse(a_uri)
 49         if not all([parsed.scheme, parsed.netloc]):
 50             return f"Invalid URI: {a_uri}"
 51 
 52         # Fetch content
 53         headers = {
 54             "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
 55         }
 56         response = requests.get(a_uri, headers=headers, timeout=10)
 57         response.raise_for_status()
 58 
 59         # Parse HTML
 60         soup = BeautifulSoup(response.text, "html.parser")
 61 
 62         # Get title
 63         title = soup.title.string if soup.title else ""
 64 
 65         # Get text and clean up
 66         text = soup.get_text()
 67 
 68         # Clean up the text
 69         text = re.sub(r"\n\s*\n", "\n\n", text)  # Remove multiple blank lines
 70         text = re.sub(r" +", " ", text)  # Remove multiple spaces
 71         text = html.unescape(text)  # Convert HTML entities
 72         text = text.strip()
 73 
 74         return f"Contents of URI {a_uri} is:\n# {title}\n\n{text}\n"
 75 
 76     except requests.RequestException as e:
 77         return f"Network error: {str(e)}"
 78 
 79     except Exception as e:
 80         return f"Error processing URI: {str(e)}"
 81 
 82 
 83 def search_web(query: str, max_results: int = 5) -> str:
 84     """
 85     Performs a web search and returns results
 86     Note: This is a placeholder. Implement with your preferred search API.
 87 
 88     Args:
 89         query (str): Search query
 90         max_results (int): Maximum number of results to return
 91 
 92     Returns:
 93         Dict[str, Any]: Dictionary containing:
 94             - 'results': List of search results
 95             - 'count': Number of results found
 96             - 'error': Error message if any, None otherwise
 97     """
 98 
 99     # Placeholder for search implementation
100     return {
101         "results": [],
102         "count": 0,
103         "error": "Web search not implemented. Please implement with your preferred search API.",
104     }
105 
106 
107 def brave_search_summaries(
108     query,
109     num_results=3,
110     url="https://api.search.brave.com/res/v1/web/search",
111     api_key=api_key,
112 ):
113     headers = {"X-Subscription-Token": api_key, "Content-Type": "application/json"}
114     params = {"q": query, "count": num_results}
115 
116     response = requests.get(url, headers=headers, params=params)
117     ret = []
118 
119     if response.status_code == 200:
120         search_results = response.json()
121         ret = [
122             {
123                 "title": result.get("title"),
124                 "url": result.get("url"),
125                 "description": replace_html_tags_with_text(result.get("description")),
126             }
127             for result in search_results.get("web", {}).get("results", [])
128         ]
129         logging.info("Successfully retrieved results.")
130     else:
131         try:
132             error_info = response.json()
133             logging.error(f"Error {response.status_code}: {error_info.get('message')}")
134         except json.JSONDecodeError:
135             logging.error(f"Error {response.status_code}: {response.text}")
136 
137     return ret
138 
139 def brave_search_text(query, num_results=3):
140     summaries = brave_search_summaries(query, num_results)
141     ret = ""
142     for s in summaries:
143         url = s["url"]
144         text = uri_to_markdown(url)
145         summary = summarize_text(
146             f"Given the query:\n\n{query}\n\nthen, summarize text removing all material that is not relevant to the query and then be very concise for a very short summary:\n\n{text}\n"
147         )
148         ret += ret + summary
149     print("\n\n-----------------------------------")
150     return ret
151 
152 # Function metadata for Ollama integration
153 uri_to_markdown.metadata = {
154     "name": "uri_to_markdown",
155     "description": "Converts web page content to markdown-style text",
156     "parameters": {"a_uri": "URI of the web page to convert"},
157 }
158 
159 search_web.metadata = {
160     "name": "search_web",
161     "description": "Performs a web search and returns results",
162     "parameters": {
163         "query": "Search query",
164         "max_results": "Maximum number of results to return",
165     },
166 }
167 
168 # Export the functions
169 __all__ = ["uri_to_markdown", "search_web"]

 1 """
 2 Judge results from LLM generation from prompts
 3 """
 4 
 5 from typing import Optional, Dict, Any
 6 from pathlib import Path
 7 import json
 8 import re
 9 from pprint import pprint
10 
11 import ollama
12 
13 client = ollama.Client()
14 
15 def judge_results(original_prompt: str, llm_gen_results: str) -> Dict[str, str]:
16     """
17     Takes an original prompt to a LLM and the output results
18 
19     Args:
20         original_prompt (str): original prompt to a LLM
21         llm_gen_results (str): output from the LLM that this function judges for accuracy
22 
23     Returns:
24         result: str: string that is one character with one of these values:
25             - 'B': Bad result
26             - 'G': A Good result
27     """
28     try:
29         messages = [
30             {"role": "system", "content": "Always judge this output for correctness."},
31             {"role": "user", "content": f"Evaluate this output:\n\n{llm_gen_results}\n\nfor this prompt:\n\n{original_prompt}\n\nDouble check your work and explain your thinking in a few sentences. End your output with a Y or N answer"},
32         ]
33 
34         response = client.chat(
35             model="qwen2.5-coder:14b", # "llama3.2:latest",
36             messages=messages,
37         )
38 
39         r = response.message.content.strip()
40         print(f"\n\noriginal COT response:\n\n{r}\n\n")
41 
42         # look at the end of the response for the Y or N judgement
43         s = r.lower()
44         # remove all non-alphabetic characters:
45         s = re.sub(r'[^a-zA-Z]', '', s).strip()
46 
47         return {'judgement': s[-1].upper(), 'reasoning': r[1:].strip()}
48 
49     except Exception as e:
50         print(f"\n\n***** {e=}\n\n")
51         return {'judgement': 'E', 'reasoning': str(e)}  # on any error, assign 'E' result

 1 $ cd OllamaEx
 2 $ python example_judge.py 
 3 
 4 ==================================================
 5  Judge output from a LLM
 6 ==================================================
 7 
 8 ==================================================
 9  First test: should be Y, or good
10 ==================================================
11 
12 
13 original COT response:
14 
15 The given output correctly calculates the absolute value of age differences for each pair:
16 
17 - Sally (55) and John (18): \( |55 - 18| = 37 \)
18 - Sally (55) and Mary (31): \( |55 - 31| = 24 \)
19 - John (18) and Mary (31): \( |31 - 18| = 13 \)
20 
21 These calculations are accurate, matching the prompt's requirements. Therefore, the answer is Y.
22 
23 
24 
25 ** JUDGEMENT ***
26 
27 judgement={'judgement': 'Y', 'reasoning': "The given output correctly calculates the absolute value of age differences for each pair:\n\n- Sally (55) and John (18): \\( |55 - 18| = 37 \\)\n- Sally (55) and Mary (31): \\( |55 - 31| = 24 \\)\n- John (18) and Mary (31): \\( |31 - 18| = 13 \\)\n\nThese calculations are accurate, matching the prompt's requirements. Therefore, the answer is Y."}
28 
29 ==================================================
30  Second test: should be N, or bad
31 ==================================================
32 
33 
34 original COT response:
35 
36 Let's evaluate the given calculations step by step:
37 
38 1. Sally (55) - John (18) = 37. The difference is calculated as 55 - 18, which equals 37.
39 2. Sally (55) - Mary (31) = 24. The difference is calculated as 55 - 31, which equals 24.
40 3. John (18) - Mary (31) = -13. However, the absolute value of this difference is |18 - 31| = 13.
41 
42 The given output shows:
43 - Sally and John: 55 - 18 = 31. This should be 37.
44 - Sally and Mary: 55 - 31 = 24. This is correct.
45 - John and Mary: 31 - 18 = 10. This should be 13.
46 
47 The output contains errors in the first and third calculations. Therefore, the answer is:
48 
49 N
50 
51 ** JUDGEMENT ***
52 
53 judgement={'judgement': 'N', 'reasoning': "et's evaluate the given calculations step by step:\n\n1. Sally (55) - John (18) = 37. The difference is calculated as 55 - 18, which equals 37.\n2. Sally (55) - Mary (31) = 24. The difference is calculated as 55 - 31, which equals 24.\n3. John (18) - Mary (31) = -13. However, the absolute value of this difference is |18 - 31| = 13.\n\nThe given output shows:\n- Sally and John: 55 - 18 = 31. This should be 37.\n- Sally and Mary: 55 - 31 = 24. This is correct.\n- John and Mary: 31 - 18 = 10. This should be 13.\n\nThe output contains errors in the first and third calculations. Therefore, the answer is:\n\nN"}

  1 import json
  2 from typing import List, Dict, Optional, Iterator
  3 import ollama
  4 from ollama import GenerateResponse
  5 
  6 
  7 def clean_json_response(response: str) -> str:
  8     """
  9     Cleans the response string by removing markdown code blocks and other formatting
 10     """
 11     # Remove markdown code block indicators
 12     response = response.replace("json", "").replace("```", "")
 13     # Strip whitespace
 14     response = response.strip()
 15     return response
 16 
 17 def evaluate_llm_conversation(
 18     chat_history: List[Dict[str, str]],
 19     evaluation_criteria: Optional[List[str]] = None,
 20     model: str = "llama3.1" # older model that is good at generating JSON
 21 ) -> Dict[str, any]:
 22     """
 23     Evaluates a chat history using Ollama to run the evaluation model.
 24 
 25     Args:
 26         chat_history: List of dictionaries containing the conversation
 27         evaluation_criteria: Optional list of specific criteria to evaluate
 28         model: Ollama model to use for evaluation
 29 
 30     Returns:
 31         Dictionary containing evaluation results
 32     """
 33     if evaluation_criteria is None:
 34         evaluation_criteria = [
 35             "Response accuracy",
 36             "Coherence and clarity",
 37             "Helpfulness",
 38             "Task completion",
 39             "Natural conversation flow"
 40         ]
 41 
 42     # Format chat history for evaluation
 43     formatted_chat = "\n".join([
 44         f"{'User' if msg['role'] == 'user' else 'Assistant'}: {msg['content']}"
 45         for msg in chat_history
 46     ])
 47 
 48     # Create evaluation prompt
 49     evaluation_prompt = f"""
 50     Please evaluate the following conversation between a user and an AI assistant.
 51     Focus on these criteria: {', '.join(evaluation_criteria)}
 52 
 53     Conversation:
 54     {formatted_chat}
 55 
 56     Provide a structured evaluation with:
 57     1. Scores (1-10) for each criterion
 58     2. Brief explanation for each score
 59     3. Overall assessment
 60     4. Suggestions for improvement
 61 
 62     Format your response as JSON.
 63     """
 64 
 65     try:
 66         # Get evaluation from Ollama
 67         response: GenerateResponse | Iterator[GenerateResponse] = ollama.generate(
 68             model=model,
 69             prompt=evaluation_prompt,
 70             system="You are an expert AI evaluator. Provide detailed, objective assessments in JSON format."
 71         )
 72 
 73         response_clean: str = clean_json_response(response['response'])
 74 
 75         # Parse the response to extract JSON
 76         try:
 77             evaluation_result = json.loads(response_clean)
 78         except json.JSONDecodeError:
 79             # Fallback if response isn't proper JSON
 80             evaluation_result = {
 81                 "error": "Could not parse evaluation as JSON",
 82                 "raw_response": response_clean
 83             }
 84 
 85         return evaluation_result
 86 
 87     except Exception as e:
 88         return {
 89             "error": f"Evaluation failed: {str(e)}",
 90             "status": "failed"
 91         }
 92 
 93 # Example usage
 94 if __name__ == "__main__":
 95     # Sample chat history
 96     sample_chat = [
 97         {"role": "user", "content": "What's the capital of France?"},
 98         {"role": "assistant", "content": "The capital of France is Paris."},
 99         {"role": "user", "content": "Tell me more about it."},
100         {"role": "assistant", "content": "Paris is the largest city in France and serves as the country's political, economic, and cultural center. It's known for landmarks like the Eiffel Tower, Louvre Museum, and Notre-Dame Cathedral."}
101     ]
102 
103     # Run evaluation
104     result = evaluate_llm_conversation(sample_chat)
105     print(json.dumps(result, indent=2))

 1 $ cd OllamaEx 
 2 $ python tool_llm_eval.py 
 3 {
 4   "evaluation": {
 5     "responseAccuracy": {
 6       "score": 9,
 7       "explanation": "The assistant correctly answered the user's question about the capital of France, and provided accurate information when the user asked for more details."
 8     },
 9     "coherenceAndClarity": {
10       "score": 8,
11       "explanation": "The assistant's responses were clear and easy to understand. However, there was a slight shift in tone from a simple answer to a more formal description."
12     },
13     "helpfulness": {
14       "score": 9,
15       "explanation": "The assistant provided relevant information that helped the user gain a better understanding of Paris. The response was thorough and answered the user's follow-up question."
16     },
17     "taskCompletion": {
18       "score": 10,
19       "explanation": "The assistant completed both tasks: providing the capital of France and elaborating on it with additional context."
20     },
21     "naturalConversationFlow": {
22       "score": 7,
23       "explanation": "While the responses were clear, they felt a bit abrupt. The assistant could have maintained a more conversational tone or encouraged further discussion."
24     }
25   },
26   "overallAssessment": {
27     "score": 8.5,
28     "explanation": "The assistant demonstrated strong technical knowledge and was able to provide accurate information on demand. However, there were some minor lapses in natural conversation flow and coherence."
29   },
30   "suggestionsForImprovement": [
31     {
32       "improvementArea": "NaturalConversationFlow",
33       "description": "Consider using more conversational language or prompts to engage users further."
34     },
35     {
36       "improvementArea": "CoherenceAndClarity",
37       "description": "Use transitional phrases and maintain a consistent tone throughout the conversation."
38     }
39   ]
40 }

 1 You are a fair judge and an expert at identifying false hallucinations and you are tasked with evaluating the accuracy of an AI-generated answer to a given context. Analyze the provided INPUT, CONTEXT, and OUTPUT to determine if the OUTPUT contains any hallucinations or false information.
 2 
 3 Guidelines:
 4 1. The OUTPUT must not contradict any information given in the CONTEXT.
 5 2. The OUTPUT must not introduce new information beyond what's provided in the CONTEXT.
 6 3. The OUTPUT should not contradict well-established facts or general knowledge.
 7 4. Check that the OUTPUT doesn't oversimplify or generalize information in a way that changes its meaning or accuracy.
 8 
 9 Analyze the text thoroughly and assign a hallucination score between 0 and 1, where:
10 - 0.0: The OUTPUT is unfaithful or is incorrect to the CONTEXT and the user's INPUT
11 - 1.0: The OUTPUT is entirely accurate abd faithful to the CONTEXT and the user's INPUT
12 
13 INPUT:
14 {input}
15 
16 CONTEXT:
17 {context}
18 
19 OUTPUT:
20 {output}
21 
22 Provide your judgement in JSON format:
23 {{
24     "score": <your score between 0.0 and 1.0>,
25     "reason": [
26         <list your reasoning as Python strings>
27     ]
28 }}

 1 """
 2 Provides functions detecting hallucinations by other LLMs
 3 """
 4 
 5 from typing import Optional, Dict, Any
 6 from pathlib import Path
 7 from pprint import pprint
 8 import json
 9 from ollama import ChatResponse
10 from ollama import chat
11 
12 def read_anti_hallucination_template() -> str:
13     """
14     Reads the anti-hallucination template file and returns the content
15     """
16     template_path = Path(__file__).parent / "templates" / "anti_hallucination.txt"
17     with template_path.open("r", encoding="utf-8") as f:
18         content = f.read()
19         return content
20 
21 TEMPLATE = read_anti_hallucination_template()
22 
23 def detect_hallucination(user_input: str, context: str, output: str) -> str:
24     """
25     Given user input, context, and LLM output, detect hallucination
26 
27     Args:
28         user_input (str): User's input text prompt
29         context (str): Context text for LLM
30         output (str): LLM's output text that is to be evaluated as being a hallucination)
31 
32     Returns: JSON data:
33      {
34        "score": <your score between 0.0 and 1.0>,
35        "reason": [
36          <list your reasoning as bullet points>
37        ]
38      }
39     """
40     prompt = TEMPLATE.format(input=user_input, context=context, output=output)
41     response: ChatResponse = chat(
42         model="llama3.2:latest",
43         messages=[
44             {"role": "system", "content": prompt},
45             {"role": "user", "content": output},
46         ],
47     )
48     try:
49         return json.loads(response.message.content)
50     except json.JSONDecodeError:
51         print(f"Error decoding JSON: {response.message.content}")
52     return {"score": 0.0, "reason": ["Error decoding JSON"]}
53 
54 
55 # Export the functions
56 __all__ = ["detect_hallucination"]
57 
58 ## Test only code:
59 
60 def main():
61     def separator(title: str):
62         """Prints a section separator"""
63         print(f"\n{'=' * 50}")
64         print(f" {title}")
65         print('=' * 50)
66 
67     # Test file writing
68     separator("Detect hallucination from a LLM")
69 
70     test_prompt = "Sally is 55, John is 18, and Mary is 31. What are pairwise combinations of the absolute value of age differences?"
71     test_context = "Double check all math results."
72     test_output = "Sally and John:  55 - 18 = 31. Sally and Mary:  55 - 31 = 24. John and Mary:  31 - 18 = 10."
73     judgement = detect_hallucination(test_prompt, test_context, test_output)
74     print(f"\n** JUDGEMENT ***\n")
75     pprint(judgement)
76 
77 if __name__ == "__main__":
78     try:
79         main()
80     except Exception as e:
81         print(f"An error occurred: {str(e)}")

 1 python /Users/markw/GITHUB/OllamaExamples/tool_anti_hallucination.py 
 2 
 3 ==================================================
 4  Detect hallucination from a LLM
 5 ==================================================
 6 
 7 ** JUDGEMENT ***
 8 
 9 {'reason': ['The OUTPUT claims that the absolute value of age differences are '
10             '31, 24, and 10 for Sally and John, Sally and Mary, and John and '
11             'Mary respectively. However, this contradicts the CONTEXT, as the '
12             'CONTEXT asks to double-check math results.',
13             'The OUTPUT does not introduce new information, but it provides '
14             'incorrect calculations: Sally and John: 55 - 18 = 37, Sally and '
15             'Mary: 55 - 31 = 24, John and Mary: 31 - 18 = 13. Therefore, the '
16             'actual output should be recalculated to ensure accuracy.',
17             'The OUTPUT oversimplifies the age differences by not considering '
18             "the order of subtraction (i.e., John's age subtracted from "
19             "Sally's or Mary's). However, this is already identified as a "
20             'contradiction in point 1.'],
21  'score': 0.0}

1 python3.11 -m venv venv
2 source venv/bin/activate
3 python3.11 -m pip install -r requirements.txt
4 python3.11 smolagents_test.py

 1 smolagents
 2 litellm[proxy]
 3 requests
 4 beautifulsoup4
 5 ollama
 6 langchain
 7 langchain-community
 8 langchain-ollama
 9 langgraph
10 rdflib
11 kuzu
12 langchain_openai
13 tabulate

 1 """
 2 smolagents example program (slightly modified)
 3 """
 4 
 5 from smolagents.agents import ToolCallingAgent
 6 from smolagents import tool, LiteLLMModel
 7 from typing import Optional
 8 
 9 model = LiteLLMModel(
10     model_id="ollama_chat/llama3.2:latest",
11     api_base="http://localhost:11434",
12     api_key="your-api-key" # not used
13 )
14 
15 @tool
16 def get_weather(location: str, celsius: Optional[bool] = False) -> str:
17     """
18     Get weather in the next days at given location.
19     Secretly this tool does not care about the location, it hates the weather everywhere.
20 
21     Args:
22         location: the location
23         celsius: the temperature
24     """
25     return "The weather is UNGODLY with torrential rains and temperatures below -10°C"
26 
27 agent = ToolCallingAgent(tools=[get_weather], model=model)
28 
29 print(agent.run("What's the weather like in Paris?"))

 1 """
 2 Wrapper for book example tools for smloagents compatibility
 3 """
 4 from pathlib import Path
 5 
 6 from smolagents import tool, LiteLLMModel
 7 from typing import Optional
 8 from pprint import pprint
 9 
10 from tool_file_dir import list_directory
11 
12 @tool
13 def sa_list_directory(list_dots: Optional[bool]=None) -> str:
14     """
15     Lists files and directories in the current working directory
16 
17     Args:
18         list_dots: optional boolean (if true, include dot files)
19 
20     Returns:
21         string with directory name, followed by list of files in the directory
22     """
23     lst = list_directory()
24     pprint(lst)
25     return lst
26 
27 @tool
28 def read_file_contents(file_path: str) -> str:
29     """
30     Reads contents from a file and returns the text
31 
32     Args:
33         file_path: Path to the file to read
34 
35     Returns:
36         Contents of the file as a string
37     """
38     try:
39         path = Path(file_path)
40         if not path.exists():
41             return f"File not found: {file_path}"
42 
43         with path.open("r", encoding="utf-8") as f:
44             content = f.read()
45             return f"Contents of file '{file_path}' is:\n{content}\n"
46 
47     except Exception as e:
48         return f"Error reading file '{file_path}' is: {str(e)}"
49 
50 @tool
51 def summarize_directory() -> str:
52     """
53     Summarizes the files and directories in the current working directory
54 
55     Returns:
56         string with directory name, followed by summary of files in the directory
57     """
58     lst = list_directory()
59     num_files = len(lst)
60     num_dirs = len([x for x in lst if x[1] == 'directory'])
61     num_files = num_files - num_dirs
62     return f"Current directory contains {num_files} files and {num_dirs} directories."

 1 from smolagents.agents import ToolCallingAgent
 2 from smolagents import tool, LiteLLMModel
 3 from typing import Optional
 4 
 5 from smolagents_tools import sa_list_directory
 6 from smolagents_tools import summarize_directory
 7 from smolagents_tools import read_file_contents
 8 
 9 model = LiteLLMModel(
10     model_id="ollama_chat/llama3.2:latest",
11     api_base="http://localhost:11434",
12     api_key="your-api-key" # not used
13 )
14 
15 agent = ToolCallingAgent(tools=[sa_list_directory,
16                                 summarize_directory,
17                                 read_file_contents],
18                          model=model)
19 
20 print(agent.run("What are the files in the current directory? Describe the current directory"))
21 
22 print(agent.run("Which Python scripts evaluate the performance of LLMs?"))

  1 python smolagents_agent_test1.py 
  2 ╭────────────────────────────────── New run ───────────────────────────────────╮
  3 │                                                                              │
  4 │ List the Python programs in the current directory, and then tell me which    │
  5 │ Python programs in the current directory evaluate the performance of LLMs?   │
  6 │                                                                              │
  7 ╰─ LiteLLMModel - ollama_chat/llama3.2:latest ─────────────────────────────────╯
  8 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  9 ╭──────────────────────────────────────────────────────────────────────────────╮
 10 │ Calling tool: 'sa_list_directory' with arguments: {'list_dots': True}        │
 11 ╰──────────────────────────────────────────────────────────────────────────────╯
 12 ('Contents of current directory: [Makefile, README.md, __pycache__, data, '
 13  'example_chain_read_summary.py, example_chain_web_summary.py, '
 14  'example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, '
 15  'langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, '
 16  'short_programs, smolagents_agent_test1.py, smolagents_test.py, '
 17  'smolagents_tools.py, templates, tool_anti_hallucination.py, '
 18  'tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, '
 19  'tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, '
 20  'tool_web_search.py, venv]')
 21 Observations: Contents of current directory: [Makefile, README.md, __pycache__, 
 22 data, example_chain_read_summary.py, example_chain_web_summary.py, 
 23 example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, 
 24 langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, 
 25 short_programs, smolagents_agent_test1.py, smolagents_test.py, 
 26 smolagents_tools.py, templates, tool_anti_hallucination.py, 
 27 tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, 
 28 tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, 
 29 venv]
 30 [Step 0: Duration 4.49 seconds| Input tokens: 1,347 | Output tokens: 79]
 31 
 32 ...
 33 
 34 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 35 ╭──────────────────────────────────────────────────────────────────────────────╮
 36 │ Calling tool: 'sa_summarize_directory' with arguments: {}                    │
 37 ╰──────────────────────────────────────────────────────────────────────────────╯
 38 lst='Contents of current directory: [Makefile, README.md, __pycache__, data, example_chain_read_summary.py, example_chain_web_summary.py, example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, short_programs, smolagents_agent_test1.py, smolagents_test.py, smolagents_tools.py, templates, tool_anti_hallucination.py, tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, venv]'
 39 response.message.content="Based on the file names provided, here's a summary of the contents and my educated guesses for their purposes:\n\n1. **Makefile**: A build script used to automate compilation, installation, or other tasks.\n2. **README.md**: A markdown file providing an introduction to the project, its purpose, and how to get started with it.\n3. **__pycache__**: This is a hidden directory generated by Python's bytecode compiler. It likely contains compiled versions of Python code in the current directory.\n4. **data**: A directory containing data used for training or testing models, simulations, or other computational tasks.\n5. **example_chain_read_summary.py**: A script that generates summaries from reading chains (e.g., text from a document). Its purpose is likely related to natural language processing (NLP) or text analysis.\n6. **example_chain_web_summary.py**: Similar to the previous one, but this script seems to be focused on web-based applications or online content summarization.\n7. **example_judge.py**: A script that evaluates judges or other aspects of a system. Its purpose might involve testing, validation, or quality control in a specific domain (e.g., AI, programming).\n8. **graph_kuzu_from_text.py**: A script that generates graphs from text data. This could be related to graph analysis, network science, or text mining.\n9. **graph_kuzu_property_example.py**: Another script related to graph generation, possibly used for demonstrating specific properties or characteristics of the generated graphs.\n10. **langgraph_agent_test.py**: A test script for a language graph agent, which likely involves machine learning, natural language processing, or other AI-related tasks.\n11. **ollama_tools_examples.py**: A file containing examples of OLLAMA (Open Language Learning with Multi-Agent) tools and their usage. This seems to be related to AI-powered language understanding and generation.\n12. **requirements.txt**: A text file specifying the dependencies required to run the project, including Python packages and other software.\n13. **short_programs**: A directory containing short programs or scripts that demonstrate specific techniques, concepts, or examples in the field of AI or programming.\n14. **smolagents_agent_test1.py**: Another test script for a small agent (SMOLAgent), which is likely involved in autonomous decision-making or other complex system simulations.\n15. **smolagents_test.py**: A general-purpose test file for SMOLAgents, used to verify their functionality and performance.\n16. **smolagents_tools.py**: A script containing tools or utilities specifically designed for working with SMOLAgents or related projects.\n17. **templates**: A directory containing templates for generating documents, reports, or other content in a specific format (e.g., LaTeX).\n18. **tool_anti_hallucination.py**: A script that addresses issues of hallucination (i.e., when an AI model provides false information) and aims to mitigate these errors.\n19. **tool_file_contents.py**, **tool_file_dir.py**, **tool_judge_results.py**, and other similar tool scripts: These tools seem to be designed for various computational tasks, such as:\n\t* `tool_file_contents.py`: Analyzing the contents of files.\n\t* `tool_file_dir.py`: Examining or manipulating directory structures.\n\t* `tool_judge_results.py`: Evaluating the performance or outcomes of a system or model.\n\n20. **tool_llm_eval.py**: A script for evaluating Large Language Models (LLMs) and their capabilities, likely involving text analysis, sentiment detection, or other NLP tasks.\n21. **tool_sqlite.py**: A tool that interacts with SQLite databases, possibly used for data storage, management, or querying.\n22. **tool_summarize_text.py**: A script designed to summarize long pieces of text into shorter versions, possibly using machine learning algorithms.\n23. **tool_web_search.py**: A tool that performs web searches or retrieves information from online sources, which could involve natural language processing (NLP) and web scraping techniques.\n\n24. **venv**: A directory generated by Python's virtual environment module, used to isolate dependencies and manage a specific Python environment for the project.\n\nKeep in mind that this is an educated guess based on common file name conventions and the context provided. The actual purposes of these files might differ depending on the specific project or domain they are related to."
 40 Observations: Summary of directory:Based on the file names provided, here's a 
 41 summary of the contents and my educated guesses for their purposes:
 42 
 43 1. **Makefile**: A build script used to automate compilation, installation, or 
 44 other tasks.
 45 2. **README.md**: A markdown file providing an introduction to the project, its 
 46 purpose, and how to get started with it.
 47 3. **__pycache__**: This is a hidden directory generated by Python's bytecode 
 48 compiler. It likely contains compiled versions of Python code in the current 
 49 directory.
 50 4. **data**: A directory containing data used for training or testing models, 
 51 simulations, or other computational tasks.
 52 5. **example_chain_read_summary.py**: A script that generates summaries from 
 53 reading chains (e.g., text from a document). Its purpose is likely related to 
 54 natural language processing (NLP) or text analysis.
 55 6. **example_chain_web_summary.py**: Similar to the previous one, but this 
 56 script seems to be focused on web-based applications or online content 
 57 summarization.
 58 7. **example_judge.py**: A script that evaluates judges or other aspects of a 
 59 system. Its purpose might involve testing, validation, or quality control in a 
 60 specific domain (e.g., AI, programming).
 61 8. **graph_kuzu_from_text.py**: A script that generates graphs from text data. 
 62 This could be related to graph analysis, network science, or text mining.
 63 9. **graph_kuzu_property_example.py**: Another script related to graph 
 64 generation, possibly used for demonstrating specific properties or 
 65 characteristics of the generated graphs.
 66 10. **langgraph_agent_test.py**: A test script for a language graph agent, which
 67 likely involves machine learning, natural language processing, or other 
 68 AI-related tasks.
 69 11. **ollama_tools_examples.py**: A file containing examples of OLLAMA (Open 
 70 Language Learning with Multi-Agent) tools and their usage. This seems to be 
 71 related to AI-powered language understanding and generation.
 72 12. **requirements.txt**: A text file specifying the dependencies required to 
 73 run the project, including Python packages and other software.
 74 13. **short_programs**: A directory containing short programs or scripts that 
 75 demonstrate specific techniques, concepts, or examples in the field of AI or 
 76 programming.
 77 14. **smolagents_agent_test1.py**: Another test script for a small agent 
 78 (SMOLAgent), which is likely involved in autonomous decision-making or other 
 79 complex system simulations.
 80 15. **smolagents_test.py**: A general-purpose test file for SMOLAgents, used to 
 81 verify their functionality and performance.
 82 16. **smolagents_tools.py**: A script containing tools or utilities specifically
 83 designed for working with SMOLAgents or related projects.
 84 17. **templates**: A directory containing templates for generating documents, 
 85 reports, or other content in a specific format (e.g., LaTeX).
 86 18. **tool_anti_hallucination.py**: A script that addresses issues of 
 87 hallucination (i.e., when an AI model provides false information) and aims to 
 88 mitigate these errors.
 89 19. **tool_file_contents.py**, **tool_file_dir.py**, **tool_judge_results.py**, 
 90 and other similar tool scripts: These tools seem to be designed for various 
 91 computational tasks, such as:
 92         * `tool_file_contents.py`: Analyzing the contents of files.
 93         * `tool_file_dir.py`: Examining or manipulating directory structures.
 94         * `tool_judge_results.py`: Evaluating the performance or outcomes of a 
 95 system or model.
 96 
 97 20. **tool_llm_eval.py**: A script for evaluating Large Language Models (LLMs) 
 98 and their capabilities, likely involving text analysis, sentiment detection, or 
 99 other NLP tasks.
100 21. **tool_sqlite.py**: A tool that interacts with SQLite databases, possibly 
101 used for data storage, management, or querying.
102 22. **tool_summarize_text.py**: A script designed to summarize long pieces of 
103 text into shorter versions, possibly using machine learning algorithms.
104 23. **tool_web_search.py**: A tool that performs web searches or retrieves 
105 information from online sources, which could involve natural language processing
106 (NLP) and web scraping techniques.
107 
108 24. **venv**: A directory generated by Python's virtual environment module, used
109 to isolate dependencies and manage a specific Python environment for the 
110 project.
111 
112 Keep in mind that this is an educated guess based on common file name 
113 conventions and the context provided. The actual purposes of these files might 
114 differ depending on the specific project or domain they are related to.
115 [Step 3: Duration 21.53 seconds| Input tokens: 6,779 | Output tokens: 133]
116 
117 ...
118 
119 Reached max iterations.
120 Final answer: [{'id': '9630af1b-498f-4604-ab1b-e4139607cb02', 'type': 
121 'function', 'function': {'name': 'sa_list_directory', 'arguments': {'list_dots':
122 True}}}, {'id': '2143745d-5ec1-4711-8813-228398bf36f8', 'type': 'function', 
123 'function': {'name': 'sa_summarize_directory', 'arguments': {}}}]
124 
125 ...
126 
127 The Python programs in the current directory are:
128 1. example_chain_read_summary.py
129 2. example_chain_web_summary.py
130 3. example_judge.py
131 4. graph_kuzu_from_text.py
132 5. graph_kuzu_property_example.py
133 6. langgraph_agent_test.py
134 7. ollama_tools_examples.py
135 8. tool_anti_hallucination.py
136 9. tool_file_contents.py
137 10. tool_file_dir.py
138 11. tool_judge_results.py
139 12. tool_llm_eval.py
140 13. tool_summarize_text.py
141 14. smolagents_agent_test1.py
142 15. smolagents_test.py
143 
144 These Python programs evaluate the performance of LLMs:
145 1. tool_anti_hallucination.py
146 2. tool_llm_eval.py
147 3. tool_summarize_text.py

  1 python smolagents_agent_test1.py 
  2 ╭────────────────────────────────── New run ───────────────────────────────────╮
  3 │                                                                              │
  4 │ What are the files in the current directory? Describe the current directory  │
  5 │                                                                              │
  6 ╰─ LiteLLMModel - ollama_chat/llama3.2:latest ─────────────────────────────────╯
  7 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  8 ╭──────────────────────────────────────────────────────────────────────────────╮
  9 │ Calling tool: 'sa_list_directory' with arguments: {'list_dots': True}        │
 10 ╰──────────────────────────────────────────────────────────────────────────────╯
 11 ('Contents of current directory: [Makefile, README.md, __pycache__, data, '
 12  'example_chain_read_summary.py, example_chain_web_summary.py, '
 13  'example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, '
 14  'langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, '
 15  'short_programs, smolagents_agent_test1.py, smolagents_test.py, '
 16  'smolagents_tools.py, templates, tool_anti_hallucination.py, '
 17  'tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, '
 18  'tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, '
 19  'tool_web_search.py, venv]')
 20 Observations: Contents of current directory: [Makefile, README.md, __pycache__, 
 21 data, example_chain_read_summary.py, example_chain_web_summary.py, 
 22 example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, 
 23 langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, 
 24 short_programs, smolagents_agent_test1.py, smolagents_test.py, 
 25 smolagents_tools.py, templates, tool_anti_hallucination.py, 
 26 tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, 
 27 tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, 
 28 venv]
 29 [Step 0: Duration 3.06 seconds| Input tokens: 1,332 | Output tokens: 19]
 30 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 31 ╭──────────────────────────────────────────────────────────────────────────────╮
 32 │ Calling tool: 'sa_list_directory' with arguments: {'list_dots': True}        │
 33 ╰──────────────────────────────────────────────────────────────────────────────╯
 34 ('Contents of current directory: [Makefile, README.md, __pycache__, data, '
 35  'example_chain_read_summary.py, example_chain_web_summary.py, '
 36  'example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, '
 37  'langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, '
 38  'short_programs, smolagents_agent_test1.py, smolagents_test.py, '
 39  'smolagents_tools.py, templates, tool_anti_hallucination.py, '
 40  'tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, '
 41  'tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, '
 42  'tool_web_search.py, venv]')
 43 Observations: Contents of current directory: [Makefile, README.md, __pycache__, 
 44 data, example_chain_read_summary.py, example_chain_web_summary.py, 
 45 example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, 
 46 langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, 
 47 short_programs, smolagents_agent_test1.py, smolagents_test.py, 
 48 smolagents_tools.py, templates, tool_anti_hallucination.py, 
 49 tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, 
 50 tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, 
 51 venv]
 52 [Step 1: Duration 1.58 seconds| Input tokens: 2,900 | Output tokens: 38]
 53 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 54 ╭──────────────────────────────────────────────────────────────────────────────╮
 55 │ Calling tool: 'sa_summarize_directory' with arguments: {}                    │
 56 ╰──────────────────────────────────────────────────────────────────────────────╯
 57 lst='Contents of current directory: [Makefile, README.md, __pycache__, data, example_chain_read_summary.py, example_chain_web_summary.py, example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, short_programs, smolagents_agent_test1.py, smolagents_test.py, smolagents_tools.py, templates, tool_anti_hallucination.py, tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, venv]'
 58 response.message.content="Based on the provided list of files in the current directory, here's a summary and an educated guess about the major purposes of each file:\n\n1. **Makefile**: A build script for automating compilation and other tasks.\n\n2. **README.md**: A document providing information about the project, its purpose, usage, and installation instructions.\n\n3. **__pycache__** (hidden directory): Cache files generated by Python's compiler to speed up importing modules.\n\n4. **data**: Directory containing data used for testing or training models.\n\n5. **example_chain_read_summary.py**, **example_chain_web_summary.py**: Example scripts demonstrating how to summarize text from chain-related input, possibly related to natural language processing (NLP) or machine learning (ML).\n\n6. **example_judge.py**: An example script for evaluating the performance of a model or algorithm.\n\n7. **graph_kuzu_from_text.py**, **graph_kuzu_property_example.py**: Scripts that manipulate graphs generated from text data using the Kuzu graph library, possibly used in NLP or ML applications.\n\n8. **langgraph_agent_test.py**: A test file for a language graph agent, which is likely an AI model designed to process and understand languages.\n\n9. **ollama_tools_examples.py**: An example script showcasing how to use Ollama, a tool for generating text data.\n\n10. **requirements.txt**: A list of dependencies required to run the project, including libraries and tools.\n\n11. **short_programs**: Directory containing short programs or scripts that demonstrate specific tasks or algorithms.\n\n12. **smolagents_agent_test1.py**, **smolagents_test.py**, **smolagents_tools.py**: Test files for a small agents framework, possibly an AI model designed to make decisions in complex environments.\n\n13. **templates**: A directory containing templates used for generating text or code in certain contexts.\n\n14. **tool_anti_hallucination.py**, **tool_file_contents.py**, **tool_file_dir.py**, **tool_judge_results.py**, **tool_llm_eval.py**, **tool_sqlite.py**, **tool_summarize_text.py**, **tool_web_search.py**: Various tool scripts that provide functionality for tasks like:\n   - Anti-hallucination (removing fake data from generated text)\n   - Evaluating file contents\n   - File directory manipulation\n   - Judging results\n   - LLM (Large Language Model) evaluation\n   - SQLite database interactions\n   - Text summarization\n   - Web search functionality\n\n15. **venv**: A virtual environment script used to create and manage a separate Python environment for the project.\n\nThese are educated guesses based on common naming conventions and directory structures in software development projects, particularly those related to AI, NLP, and machine learning."
 59 Observations: Summary of directory:Based on the provided list of files in the 
 60 current directory, here's a summary and an educated guess about the major 
 61 purposes of each file:
 62 
 63 1. **Makefile**: A build script for automating compilation and other tasks.
 64 
 65 2. **README.md**: A document providing information about the project, its 
 66 purpose, usage, and installation instructions.
 67 
 68 3. **__pycache__** (hidden directory): Cache files generated by Python's 
 69 compiler to speed up importing modules.
 70 
 71 4. **data**: Directory containing data used for testing or training models.
 72 
 73 5. **example_chain_read_summary.py**, **example_chain_web_summary.py**: Example 
 74 scripts demonstrating how to summarize text from chain-related input, possibly 
 75 related to natural language processing (NLP) or machine learning (ML).
 76 
 77 6. **example_judge.py**: An example script for evaluating the performance of a 
 78 model or algorithm.
 79 
 80 7. **graph_kuzu_from_text.py**, **graph_kuzu_property_example.py**: Scripts that
 81 manipulate graphs generated from text data using the Kuzu graph library, 
 82 possibly used in NLP or ML applications.
 83 
 84 8. **langgraph_agent_test.py**: A test file for a language graph agent, which is
 85 likely an AI model designed to process and understand languages.
 86 
 87 9. **ollama_tools_examples.py**: An example script showcasing how to use Ollama,
 88 a tool for generating text data.
 89 
 90 10. **requirements.txt**: A list of dependencies required to run the project, 
 91 including libraries and tools.
 92 
 93 11. **short_programs**: Directory containing short programs or scripts that 
 94 demonstrate specific tasks or algorithms.
 95 
 96 12. **smolagents_agent_test1.py**, **smolagents_test.py**, 
 97 **smolagents_tools.py**: Test files for a small agents framework, possibly an AI
 98 model designed to make decisions in complex environments.
 99 
100 13. **templates**: A directory containing templates used for generating text or 
101 code in certain contexts.
102 
103 14. **tool_anti_hallucination.py**, **tool_file_contents.py**, 
104 **tool_file_dir.py**, **tool_judge_results.py**, **tool_llm_eval.py**, 
105 **tool_sqlite.py**, **tool_summarize_text.py**, **tool_web_search.py**: Various 
106 tool scripts that provide functionality for tasks like:
107    - Anti-hallucination (removing fake data from generated text)
108    - Evaluating file contents
109    - File directory manipulation
110    - Judging results
111    - LLM (Large Language Model) evaluation
112    - SQLite database interactions
113    - Text summarization
114    - Web search functionality
115 
116 15. **venv**: A virtual environment script used to create and manage a separate 
117 Python environment for the project.
118 
119 These are educated guesses based on common naming conventions and directory 
120 structures in software development projects, particularly those related to AI, 
121 NLP, and machine learning.
122 [Step 2: Duration 13.79 seconds| Input tokens: 4,706 | Output tokens: 54]
123 
124 ...
125 
126 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
127 ╭──────────────────────────────────────────────────────────────────────────────╮
128 │ Calling tool: 'sa_summarize_directory' with arguments: {}                    │
129 ╰──────────────────────────────────────────────────────────────────────────────╯
130 lst='Contents of current directory: [Makefile, README.md, __pycache__, data, example_chain_read_summary.py, example_chain_web_summary.py, example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, short_programs, smolagents_agent_test1.py, smolagents_test.py, smolagents_tools.py, templates, tool_anti_hallucination.py, tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, venv]'
131 response.message.content="Based on the names and locations of the files in the current directory, here's a summary of their contents and a educated guess about their purposes:\n\n1. **Makefile**: A build script for automating tasks such as compiling or running code. It likely contains instructions on how to build and install the project.\n\n2. **README.md**: A Markdown document that serves as an introduction or guide for users of the project. It may include information on how to use the tools, dependencies required, and contributing to the project.\n\n3. **__pycache__**: An empty directory that contains compiled Python files (`.cpyc` and `.pyo`) generated by the PyInstaller build process for a Python application.\n\n4. **data**: A directory containing data used for testing or training purposes. It might include CSV, JSON, or other formats of datasets.\n\n5. **example_chain_read_summary.py** and **example_chain_web_summary.py**: Example scripts demonstrating how to use tools related to text summarization, possibly for natural language processing (NLP) tasks.\n\n6. **example_judge.py**: An example script that likely demonstrates the usage of a judging tool or an evaluation framework for the project.\n\n7. **graph_kuzu_from_text.py** and **graph_kuzu_property_example.py**: Scripts related to graph-based tools, possibly using Kuzu, a library for graph algorithms. These scripts might illustrate how to work with graphs in Python.\n\n8. **langgraph_agent_test.py**: A script that tests the functionality of a language graph agent.\n\n9. **ollama_tools_examples.py**: An example script demonstrating the usage of OLLAMA tools ( likely Open-source Language Model-based Agent).\n\n10. **requirements.txt**: A text file listing the dependencies required to run the project, such as Python packages or other software libraries.\n\n11. **short_programs**: A directory containing short programs for demonstration purposes.\n\n12. **smolagents_agent_test1.py** and **smolagents_test.py**: Scripts related to testing SmoLA (Small Model-based Language Agent), a library that allows the use of small models in language agents.\n\n13. **smolagents_tools.py**: An example script demonstrating the usage of SmoLA tools.\n\n14. **templates**: A directory containing template files used for generating documentation or other text output.\n\n15. **tool_anti_hallucination.py**, **tool_file_contents.py**, **tool_file_dir.py**, **tool_judge_results.py**, and **tool_llm_eval.py**: Scripts related to various tools, possibly used for data analysis, model evaluation, or language understanding tasks.\n\n16. **tool_sqlite.py** and **tool_summarize_text.py**: Scripts that interface with SQLite databases or are used for text summarization.\n\n17. **tool_web_search.py** and **tool_web_search.py** ( likely a duplicate): These scripts might be related to web search-related tools, possibly interfacing with APIs or web scraping techniques.\n\n18. **venv**: A directory containing the virtual environment configuration files created by `python -m venv` command, allowing the project to isolate its dependencies from the system Python environment.\n\nNote: The exact purposes of some files may vary depending on the specific implementation and requirements of the project. This summary is based on common file name conventions and their typical associations with certain tasks or tools in the field of NLP and agent-based systems."
132 Observations: Summary of directory:Based on the names and locations of the files
133 in the current directory, here's a summary of their contents and a educated 
134 guess about their purposes:
135 
136 1. **Makefile**: A build script for automating tasks such as compiling or 
137 running code. It likely contains instructions on how to build and install the 
138 project.
139 
140 2. **README.md**: A Markdown document that serves as an introduction or guide 
141 for users of the project. It may include information on how to use the tools, 
142 dependencies required, and contributing to the project.
143 
144 3. **__pycache__**: An empty directory that contains compiled Python files 
145 (`.cpyc` and `.pyo`) generated by the PyInstaller build process for a Python 
146 application.
147 
148 4. **data**: A directory containing data used for testing or training purposes. 
149 It might include CSV, JSON, or other formats of datasets.
150 
151 5. **example_chain_read_summary.py** and **example_chain_web_summary.py**: 
152 Example scripts demonstrating how to use tools related to text summarization, 
153 possibly for natural language processing (NLP) tasks.
154 
155 6. **example_judge.py**: An example script that likely demonstrates the usage of
156 a judging tool or an evaluation framework for the project.
157 
158 7. **graph_kuzu_from_text.py** and **graph_kuzu_property_example.py**: Scripts 
159 related to graph-based tools, possibly using Kuzu, a library for graph 
160 algorithms. These scripts might illustrate how to work with graphs in Python.
161 
162 8. **langgraph_agent_test.py**: A script that tests the functionality of a 
163 language graph agent.
164 
165 9. **ollama_tools_examples.py**: An example script demonstrating the usage of 
166 OLLAMA tools ( likely Open-source Language Model-based Agent).
167 
168 10. **requirements.txt**: A text file listing the dependencies required to run 
169 the project, such as Python packages or other software libraries.
170 
171 11. **short_programs**: A directory containing short programs for demonstration 
172 purposes.
173 
174 12. **smolagents_agent_test1.py** and **smolagents_test.py**: Scripts related to
175 testing SmoLA (Small Model-based Language Agent), a library that allows the use 
176 of small models in language agents.
177 
178 13. **smolagents_tools.py**: An example script demonstrating the usage of SmoLA 
179 tools.
180 
181 14. **templates**: A directory containing template files used for generating 
182 documentation or other text output.
183 
184 15. **tool_anti_hallucination.py**, **tool_file_contents.py**, 
185 **tool_file_dir.py**, **tool_judge_results.py**, and **tool_llm_eval.py**: 
186 Scripts related to various tools, possibly used for data analysis, model 
187 evaluation, or language understanding tasks.
188 
189 16. **tool_sqlite.py** and **tool_summarize_text.py**: Scripts that interface 
190 with SQLite databases or are used for text summarization.
191 
192 17. **tool_web_search.py** and **tool_web_search.py** ( likely a duplicate): 
193 These scripts might be related to web search-related tools, possibly interfacing
194 with APIs or web scraping techniques.
195 
196 18. **venv**: A directory containing the virtual environment configuration files
197 created by `python -m venv` command, allowing the project to isolate its 
198 dependencies from the system Python environment.
199 
200 Note: The exact purposes of some files may vary depending on the specific 
201 implementation and requirements of the project. This summary is based on common 
202 file name conventions and their typical associations with certain tasks or tools
203 in the field of NLP and agent-based systems.
204 [Step 5: Duration 18.89 seconds| Input tokens: 10,726 | Output tokens: 102]
205 Reached max iterations.
206 Final answer: The current directory appears to be a repository for various 
207 projects related to natural language processing (NLP) and agent-based systems. 
208 The directory contains a mix of files and subdirectories that suggest it is used
209 for building, testing, and demonstrating NLP-related tools and applications.
210 
211 Here are some key observations about the files in the current directory:
212 
213 1. **Makefile**: A build script for automating tasks such as compiling or 
214 running code.
215 2. **README.md**: A Markdown document serving as an introduction to the project.
216 3. **__pycache__**: An empty directory containing compiled Python files 
217 generated by the PyInstaller build process.
218 
219 4. **data**: A directory containing data used for testing or training purposes, 
220 including CSV, JSON, and other formats of datasets.
221 
222 5. **Example scripts**: Various example scripts demonstrating how to use tools 
223 related to text summarization, judging, graph-based algorithms, language graph 
224 agents, and more.
225 
226 6. **Requirements file**: A text file listing the dependencies required to run 
227 the project.
228 
229 7. **Short programs**: A directory containing short programs for demonstration 
230 purposes.
231 
232 8. **Testing scripts**: Scripts related to testing various NLP-related tools and
233 libraries, including SmoLA and OLLAMA.
234 
235 9. **Tool scripts**: Various scripts related to data analysis, model evaluation,
236 language understanding, web search, and other tasks.
237 
238 10. **Virtual environment**: A directory containing the virtual environment 
239 configuration files created by `python -m venv` command.
240 
241 The current directory seems to be a collection of example projects, tools, and 
242 scripts for working with NLP-related technologies. It is likely used as a 
243 demonstration or testing environment for exploring new ideas and techniques.
244 [Step 6: Duration 0.00 seconds| Input tokens: 12,332 | Output tokens: 442]
245 The current directory appears to be a repository for various projects related to natural language processing (NLP) and agent-based systems. The directory contains a mix of files and subdirectories that suggest it is used for building, testing, and demonstrating NLP-related tools and applications.
246 
247 Here are some key observations about the files in the current directory:
248 
249 1. **Makefile**: A build script for automating tasks such as compiling or running code.
250 2. **README.md**: A Markdown document serving as an introduction to the project.
251 3. **__pycache__**: An empty directory containing compiled Python files generated by the PyInstaller build process.
252 
253 4. **data**: A directory containing data used for testing or training purposes, including CSV, JSON, and other formats of datasets.
254 
255 5. **Example scripts**: Various example scripts demonstrating how to use tools related to text summarization, judging, graph-based algorithms, language graph agents, and more.
256 
257 6. **Requirements file**: A text file listing the dependencies required to run the project.
258 
259 7. **Short programs**: A directory containing short programs for demonstration purposes.
260 
261 8. **Testing scripts**: Scripts related to testing various NLP-related tools and libraries, including SmoLA and OLLAMA.
262 
263 9. **Tool scripts**: Various scripts related to data analysis, model evaluation, language understanding, web search, and other tasks.
264 
265 10. **Virtual environment**: A directory containing the virtual environment configuration files created by `python -m venv` command.
266 
267 The current directory seems to be a collection of example projects, tools, and scripts for working with NLP-related technologies. It is likely used as a demonstration or testing environment for exploring new ideas and techniques.