1 #!/bin/zsh
 2 
 3 # Check if the directory has a requirements.txt file
 4 if [ ! -f "requirements.txt" ]; then
 5     echo "No requirements.txt file found in the current directory."
 6     exit 1
 7 fi
 8 
 9 # Create a virtual environment in the venv directory
10 python3 -m venv venv
11 
12 # Activate the virtual environment
13 source venv/bin/activate
14 
15 # Upgrade pip to the latest version
16 pip3 install --upgrade pip
17 
18 # Install dependencies from requirements.txt
19 pip3 install -r requirements.txt
20 
21 # Display installed packages
22 pip3 list
23 
24 echo "Virtual environment setup complete. Reactivate it with:"
25 echo "source venv/bin/activate"
26 echo ""

1 uv venv
2 source .venv/bin/activate
3 uv pip install -r requirements.txt
4 uv run ollama_tools_examples.py

 1 $ ollama run qwq:latest --format json
 2 >>> What are the capitals of Germany and France?
 3 { 
 4   "Germany": {
 5     "Capital": "Berlin",
 6     "Population": "83.2 million",
 7     "Area": "137,847 square miles"
 8   },
 9   "France": {
10     "Capital": "Paris",
11     "Population": "67.4 million",
12     "Area": "248,573 square miles"
13   }
14 }
15 
16 >>> /bye

1 $ ollama run llava:7b "Describe this image" markcarol.jpg
2  The image is a photograph featuring a man and a woman looking 
3 off-camera, towards the left side of the frame. In the background, there are indisti\
4 nct objects that give the impression of an outdoor setting, possibly on a patio or d
5 eck.
6 
7 The focus and composition suggest that the photo was taken during the day in natural\
8  light. 

1 $ ollama run qwen2.5-coder:14b
2 >>> Send a message (/? for help)

1 $ ollama run qwq:latest       
2 >>>

 1 $ ollama run qwen2.5-coder:14b < tool_sqlite.py 
 2 This code defines a Python application that interacts with an SQLite database using \
 3 SQL queries 
 4 generated by the Ollama language model. The application is structured around two mai\
 5 n classes:
 6 
 7 1. **SQLiteTool**: Manages interactions with an SQLite database.
 8    - Handles creating sample data, managing database connections, and executing SQL \
 9 queries.
10    - Provides methods to list tables in the database, get table schemas, and execute\
11  arbitrary SQL 
12 queries.
13 
14 2. **OllamaFunctionCaller**: Acts as a bridge between user inputs and the SQLite dat\
15 abase through the 
16 Ollama model.
17    - Defines functions that can be called by the Ollama model (e.g., querying the da\
18 tabase or listing 
19 tables).
20    - Generates prompts for the Ollama model based on user input, parses the response\
21  to identify which 
22 function should be executed, and then calls the appropriate method in `SQLiteTool`.
23 
24 ...

 1 import ollama
 2 import base64
 3 
 4 def analyze_image(image_path: str, prompt: str) -> str:
 5     # Read and encode the image
 6     with open(image_path, 'rb') as img_file:
 7         image_data = base64.b64encode(img_file.read()).decode('utf-8')
 8 
 9     try:
10         # Create a stream of responses using the Ollama SDK
11         stream = ollama.generate(
12             model='llava:7b',
13             prompt=prompt,
14             images=[image_data],
15             stream=True
16         )
17 
18         # Accumulate the response
19         full_response = ""
20         for chunk in stream:
21             if 'response' in chunk:
22                 full_response += chunk['response']
23 
24         return full_response
25 
26     except Exception as e:
27         return f"Error processing image: {str(e)}"
28 
29 def main():
30     image_path = "data/sample.jpg"
31     prompt = "Please describe this image in detail, focusing on the actions of peopl\
32 e in the picture."
33 
34     result = analyze_image(image_path, prompt)
35     print("Analysis Result:")
36     print(result)
37 
38 if __name__ == "__main__":
39     main()

 1 Analysis Result:
 2  The image appears to be a photograph taken inside a room that serves as a meeting o\
 3 r gaming space and capturing an indoor scene where five individuals are engaged in p
 4 laying a tabletop card game. In the foreground, there is a table with a green surfac
 5 e and multiple items on it, including what looks like playing cards spread out in fr
 6 ont of the people seated around it.
 7 
 8 The room has a comfortable and homely feel, with elements like a potted plant in the\
 9  background on the left, which suggests that this might be a living room or a simila
10 r space repurposed for a group activity.

 1 import openai
 2 from typing import List, Dict
 3 
 4 class OllamaClient:
 5     def __init__(self, base_url: str = "http://localhost:11434/v1"):
 6         self.client = openai.OpenAI(
 7             base_url=base_url,
 8             api_key="fake-key"  # Ollama doesn't require authentication locally
 9         )
10 
11     def chat_with_context(
12         self,
13         system_context: str,
14         user_prompt: str,
15         model: str = "llama3.2:latest",
16         temperature: float = 0.7
17     ) -> str:
18         try:
19             messages = [
20                 {"role": "system", "content": system_context},
21                 {"role": "user", "content": user_prompt}
22             ]
23 
24             response = self.client.chat.completions.create(
25                 model=model,
26                 messages=messages,
27                 temperature=temperature,
28                 stream=False
29             )
30 
31             return response.choices[0].message.content
32 
33         except Exception as e:
34             return f"Error: {str(e)}"
35 
36     def chat_conversation(
37         self,
38         messages: List[Dict[str, str]],
39         model: str = "llama2"
40     ) -> str:
41         try:
42             response = self.client.chat.completions.create(
43                 model=model,
44                 messages=messages,
45                 stream=False
46             )
47 
48             return response.choices[0].message.content
49 
50         except Exception as e:
51             return f"Error: {str(e)}"
52 
53 def main():
54     # Initialize the client
55     client = OllamaClient()
56 
57     # Example 1: Single interaction with context
58     system_context = """You are a helpful AI assistant with expertise in 
59     programming and technology. Provide clear, concise answers."""
60 
61     user_prompt = "Explain the concept of recursion in programming."
62 
63     response = client.chat_with_context(
64         system_context=system_context,
65         user_prompt=user_prompt,
66         model="llama3.2:latest",
67         temperature=0.7
68     )
69 
70     print("Response with context:")
71     print(response)
72     print("\n" + "="*50 + "\n")
73 
74     # Example 2: Multi-turn conversation
75     conversation = [
76         {"role": "system", "content": "You are a helpful AI assistant."},
77         {"role": "user", "content": "What is machine learning?"},
78         {"role": "assistant", "content": "Machine learning is a subset of AI that en\
79 ables systems to learn from data."},
80         {"role": "user", "content": "Can you give me a simple example?"}
81     ]
82 
83     response = client.chat_conversation(
84         messages=conversation,
85         model="llama3.2:latest"
86     )
87 
88     print("Conversation response:")
89     print(response)
90 
91 if __name__ == "__main__":
92     main()

 1 Response with context:
 2 Recursion is a fundamental concept in programming that allows a function or method t\
 3 o call itself repeatedly until it reaches a base case that stops the recursion.
 4 
 5 **What is Recursion?**
 6 
 7 In simple terms, recursion is a programming technique where a function invokes itsel\
 8 f as a sub-procedure, repeating the same steps until it solves a problem ...
 9 
10 **Key Characteristics of Recursion:**
11 
12 1. **Base case**: A trivial case that stops the recursion.
13 2. **Recursive call**: The function calls itself with new input or parameters.
14 3. **Termination condition**: When the base case is reached, the recursion terminate\
15 s.
16 
17 **How Recursion Works:**
18 
19 Here's an example to illustrate recursion:
20 
21 Imagine you have a recursive function `factorial(n)` that calculates the factorial o\
22 f a number `n`. The function works as follows:
23 
24 1. If `n` is 0 or 1 (base case), return 1.
25 2. Otherwise, call itself with `n-1` as input and multiply the result by `n`.
26 3. Repeat step 2 until `n` reaches 0 or 1.
27 
28 Here's a simple recursive implementation in Python ...
29 
30 **Benefits of Recursion:**
31 
32 Recursion offers several advantages:
33 
34 * **Divide and Conquer**: Break down complex problems into smaller, more manageable \
35 sub-problems.
36 * **Elegant Code**: Recursive solutions can be concise and easy to understand.
37 * **Efficient**: Recursion can avoid explicit loops and reduce memory usage.
38 ...
39 
40 In summary, recursion is a powerful technique that allows functions to call themselv\
41 es repeatedly until they solve a problem. By understanding the basics of recursion a
42 nd its applications, you can write more efficient and elegant code for complex probl
43 ems.
44 
45 ==================================================
46 
47 Conversation response:
48 A simple example of machine learning is a spam filter.
49 
50 Imagine we have a system that scans emails and identifies whether they are spam or n\
51 ot. The system learns to classify these emails as spam or not based on the following
52  steps:
53 
54 1. Initially, it receives a large number of labeled data points (e.g., 1000 emails),\
55  where some emails are marked as "spam" and others as "not spam".
56 2. The system analyzes these examples to identify patterns and features that disting\
57 uish spam emails from non-spam messages.
58 3. Once the patterns are identified, the system can use them to classify new, unseen\
59  email data (e.g., a new email) as either spam or not spam.
60 
61 Over time, the system becomes increasingly accurate in its classification because it\
62  has learned from the examples and improvements have been made. This is essentially 
63 an example of supervised machine learning, where the system learns by being trained 
64 on labeled data.

1 OllamaExamples $ ls tool*
2 tool_file_contents.py   tool_llm_eval.py    tool_web_search.py
3 tool_file_dir.py    tool_sqlite.py
4 tool_judge_results.py   tool_summarize_text.py

1 git clone https://github.com/mark-watson/OllamaExamples.git

 1 from tool_file_dir import list_directory
 2 from tool_file_contents import read_file_contents
 3 from tool_web_search import uri_to_markdown
 4 
 5 import ollama
 6 
 7 # Map function names to function objects
 8 available_functions = {
 9     'list_directory': list_directory,
10     'read_file_contents': read_file_contents,
11     'uri_to_markdown': uri_to_markdown,
12 }
13 
14 # User prompt
15 user_prompt = "Please list the contents of the current directory, read the 'requirem\
16 ents.txt' file, and convert 'https://markwatson.com' to markdown."
17 
18 # Initiate chat with the model
19 response = ollama.chat(
20     model='llama3.1',
21     messages=[{'role': 'user', 'content': user_prompt}],
22     tools=[list_directory, read_file_contents, uri_to_markdown],
23 )
24 
25 # Process the model's response
26 for tool_call in response.message.tool_calls or []:
27     function_to_call = available_functions.get(tool_call.function.name)
28     print(f"{function_to_call=}")
29     if function_to_call:
30         result = function_to_call(**tool_call.function.arguments)
31         print(f"Output of {tool_call.function.name}: {result}")
32     else:
33         print(f"Function {tool_call.function.name} not found.")

 1 $ python ollama_tools_examples.py
 2 
 3 function_to_call=<function read_file_contents at 0x104fac9a0>
 4 
 5 Output of read_file_contents: {'content': 'git+https://github.com/mark-watson/Ollama\
 6 _Tools.git\nrequests\nbeautifulsoup4\naisuite[ollama]\n\n', 'size': 93, 'exists': Tr
 7 ue, 'error': None}
 8 
 9 function_to_call=<function list_directory at 0x1050389a0>
10 Output of list_directory: {'files': ['.git', '.gitignore', 'LICENSE', 'Makefile', 'R\
11 EADME.md', 'ollama_tools_examples.py', 'requirements.txt', 'venv'], 'count': 8, 'cur
12 rent_dir': '/Users/markw/GITHUB/Ollama-book-examples', 'error': None}
13 
14 function_to_call=<function uri_to_markdown at 0x105038c20>
15 
16 Output of uri_to_markdown: {'content': 'Read My Blog on Blogspot\n\nRead My Blog on \
17 Substack\n\nConsulting\n\nFree Mentoring\n\nFun stuff\n\nMy Books\n\nOpen Source\n\n
18  Privacy Policy\n\n# Mark Watson AI Practitioner and Consultant Specializing in Larg
19 e Language Models, LangChain/Llama-Index Integrations, Deep Learning, and the Semant
20 ic Web\n\n# I am the author of 20+ books on Artificial Intelligence, Python, Common 
21 Lisp, Deep Learning, Haskell, Clojure, Java, Ruby, Hy language, and the Semantic Web
22 . I have 55 US Patents.\n\nMy customer list includes: Google, Capital One, Babylist,
23  Olive AI, CompassLabs, Mind AI, Disney, SAIC, Americast, PacBell, CastTV, Lutris Te
24 chnology, Arctan Group, Sitescout.com, Embed.ly, and Webmind Corporation.
25 
26  ...
27 
28  # Fun stuff\n\nIn addition to programming and writing my hobbies are cooking,\n pho\
29 tography, hiking, travel, and playing the following musical instruments: guitar, did
30 geridoo, and American Indian flute:\n\nMy guitar playing: a boogie riff\n\nMy didger
31 idoo playing\n\nMy Spanish guitar riff\n\nPlaying with George (didgeridoo), Carol an
32 d Crystal (drums and percussion) and Mark (Indian flute)\n\n# Open Source\n\nMy Open
33  Source projects are hosted on my github account so please check that out!
34 
35  ...
36 
37 Hosted on Cloudflare Pages\n\n © Mark Watson 1994-2024\n\nPrivacy Policy', 'title': \
38 'Mark Watson: AI Practitioner and Author of 20+ AI Books | Mark Watson', 'error': No
39 ne}

1 # Fun stuff
2 
3 In addition to programming and writing my hobbies are cooking,
4 photography, hiking, travel, and playing the following musical
5 instruments: guitar, didgeridoo, and American Indian flute ...

 1 """
 2 Provides functions for reading and writing file contents with proper error handling
 3 """
 4 
 5 from pathlib import Path
 6 import json
 7 
 8 
 9 def read_file_contents(file_path: str, encoding: str = "utf-8") -> str:
10     """
11     Reads contents from a file and returns the text
12 
13     Args:
14         file_path (str): Path to the file to read
15         encoding (str): File encoding to use (default: utf-8)
16 
17     Returns:
18         Contents of the file as a string
19     """
20     try:
21         path = Path(file_path)
22         if not path.exists():
23             return f"File not found: {file_path}"
24 
25         with path.open("r", encoding=encoding) as f:
26             content = f.read()
27             return f"Contents of file '{file_path}' is:\n{content}\n"
28 
29     except Exception as e:
30         return f"Error reading file '{file_path}' is: {str(e)}"
31 
32 
33 def write_file_contents(
34         file_path: str, content: str,
35         encoding: str = "utf-8",
36         mode: str = "w") -> str:
37     """
38     Writes content to a file and returns operation status
39 
40     Args:
41         file_path (str): Path to the file to write
42         content (str): Content to write to the file
43         encoding (str): File encoding to use (default: utf-8)
44         mode (str): Write mode ('w' for write, 'a' for append)
45 
46     Returns:
47         a message string
48     """
49     try:
50         path = Path(file_path)
51 
52         # Create parent directories if they don't exist
53         path.parent.mkdir(parents=True, exist_ok=True)
54 
55         with path.open(mode, encoding=encoding) as f:
56             bytes_written = f.write(content)
57 
58         return f"File '{file_path}' written OK."
59 
60     except Exception as e:
61         return f"Error writing file '{file_path}': {str(e)}"
62 
63 
64 # Function metadata for Ollama integration
65 read_file_contents.metadata = {
66     "name": "read_file_contents",
67     "description": "Reads contents from a file and returns the content as a string",
68     "parameters": {"file_path": "Path to the file to read"},
69 }
70 
71 write_file_contents.metadata = {
72     "name": "write_file_contents",
73     "description": "Writes content to a file and returns operation status",
74     "parameters": {
75         "file_path": "Path to the file to write",
76         "content": "Content to write to the file",
77         "encoding": "File encoding (default: utf-8)",
78         "mode": 'Write mode ("w" for write, "a" for append)',
79     },
80 }
81 
82 # Export the functions
83 __all__ = ["read_file_contents", "write_file_contents"]

 1 def list_directory(pattern: str = "*", list_dots=None) -> Dict[str, Any]:
 2     """
 3     Lists files and directories in the current working directory
 4 
 5     Args:
 6         pattern (str): Glob pattern for filtering files (default: "*")
 7 
 8     Returns:
 9         string with directory name, followed by list of files in the directory
10     """
11     try:
12         current_dir = Path.cwd()
13         files = list(current_dir.glob(pattern))
14 
15         # Convert Path objects to strings and sort
16         file_list = sorted([str(f.name) for f in files])
17 
18         file_list = [file for file in file_list if not file.endswith("~")]
19         if not list_dots:
20             file_list = [file for file in file_list if not file.startswith(".")]
21 
22         return f"Contents of current directory: [{', '.join(file_list)}]"
23 
24     except Exception as e:
25         return f"Error listing directory: {str(e)}"

  1 import sqlite3
  2 import json
  3 from typing import Dict, Any, List, Optional
  4 import ollama
  5 from functools import wraps
  6 import re
  7 from contextlib import contextmanager
  8 from textwrap import dedent # for multi-line string literals
  9 
 10 class DatabaseError(Exception):
 11     """Custom exception for database operations"""
 12     pass
 13 
 14 
 15 def _create_sample_data(cursor):  # Helper function to create sample data
 16     """Create sample data for tables"""
 17     sample_data = {
 18         'example': [
 19             ('Example 1', 10.5),
 20             ('Example 2', 25.0)
 21         ],
 22         'users': [
 23             ('Bob', 'bob@example.com'),
 24             ('Susan', 'susan@test.net')
 25         ],
 26         'products': [
 27             ('Laptop', 1200.00),
 28             ('Keyboard', 75.50)
 29         ]
 30     }
 31 
 32     for table, data in sample_data.items():
 33         for record in data:
 34             if table == 'example':
 35                 cursor.execute(
 36                     "INSERT INTO example (name, value) VALUES (?, ?) ON CONFLICT DO \
 37 NOTHING",
 38                     record
 39                 )
 40             elif table == 'users':
 41                 cursor.execute(
 42                     "INSERT INTO users (name, email) VALUES (?, ?) ON CONFLICT DO NO\
 43 THING",
 44                     record
 45                 )
 46             elif table == 'products':
 47                 cursor.execute(
 48                     "INSERT INTO products (product_name, price) VALUES (?, ?) ON CON\
 49 FLICT DO NOTHING",
 50                     record
 51                 )
 52 
 53 
 54 class SQLiteTool:
 55     _instance = None
 56 
 57     def __new__(cls, *args, **kwargs):
 58         if not isinstance(cls._instance, cls):
 59             cls._instance = super(SQLiteTool, cls).__new__(cls)
 60         return cls._instance
 61 
 62     def __init__(self, default_db: str = "test.db"):
 63         if hasattr(self, 'default_db'):  # Skip initialization if already done
 64             return
 65         self.default_db = default_db
 66         self._initialize_database()
 67 
 68     @contextmanager
 69     def get_connection(self):
 70         """Context manager for database connections"""
 71         conn = sqlite3.connect(self.default_db)
 72         try:
 73             yield conn
 74         finally:
 75             conn.close()
 76 
 77     def _initialize_database(self):
 78         """Initialize database with tables"""
 79         tables = {
 80             'example': """
 81                 CREATE TABLE IF NOT EXISTS example (
 82                     id INTEGER PRIMARY KEY,
 83                     name TEXT,
 84                     value REAL
 85                 );
 86             """,
 87             'users': """
 88                 CREATE TABLE IF NOT EXISTS users (
 89                     id INTEGER PRIMARY KEY,
 90                     name TEXT,
 91                     email TEXT UNIQUE
 92                 );
 93             """,
 94             'products': """
 95                 CREATE TABLE IF NOT EXISTS products (
 96                     id INTEGER PRIMARY KEY,
 97                     product_name TEXT,
 98                     price REAL
 99                 );
100             """
101         }
102 
103         with self.get_connection() as conn:
104             cursor = conn.cursor()
105             for table_sql in tables.values():
106                 cursor.execute(table_sql)
107             conn.commit()
108             _create_sample_data(cursor)
109             conn.commit()
110 
111     def get_tables(self) -> List[str]:
112         """Get list of tables in the database"""
113         with self.get_connection() as conn:
114             cursor = conn.cursor()
115             cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
116             return [table[0] for table in cursor.fetchall()]
117 
118     def get_table_schema(self, table_name: str) -> List[tuple]:
119         """Get schema for a specific table"""
120         with self.get_connection() as conn:
121             cursor = conn.cursor()
122             cursor.execute(f"PRAGMA table_info({table_name});")
123             return cursor.fetchall()
124 
125     def execute_query(self, query: str) -> List[tuple]:
126         """Execute a SQL query and return results"""
127         with self.get_connection() as conn:
128             cursor = conn.cursor()
129             try:
130                 cursor.execute(query)
131                 return cursor.fetchall()
132             except sqlite3.Error as e:
133                 raise DatabaseError(f"Query execution failed: {str(e)}")
134 
135 class OllamaFunctionCaller:
136     def __init__(self, model: str = "llama3.2:latest"):
137         self.model = model
138         self.sqlite_tool = SQLiteTool()
139         self.function_definitions = self._get_function_definitions()
140 
141     def _get_function_definitions(self) -> Dict:
142         return {
143             "query_database": {
144                 "description": "Execute a SQL query on the database",
145                 "parameters": {
146                     "type": "object",
147                     "properties": {
148                         "query": {
149                             "type": "string",
150                             "description": "The SQL query to execute"
151                         }
152                     },
153                     "required": ["query"]
154                 }
155             },
156             "list_tables": {
157                 "description": "List all tables in the database",
158                 "parameters": {
159                     "type": "object",
160                     "properties": {}
161                 }
162             }
163         }
164 
165     def _generate_prompt(self, user_input: str) -> str:
166         prompt = dedent(f"""
167             You are a SQL assistant. Based on the user's request, generate a JSON re\
168 sponse that calls the appropriate function.
169             Available functions: {json.dumps(self.function_definitions, indent=2)}
170 
171             User request: {user_input}
172 
173             Respond with a JSON object containing:
174             - "function": The function name to call
175             - "parameters": The parameters for the function
176 
177             Response:
178         """).strip()
179         return prompt
180 
181     def _parse_ollama_response(self, response: str) -> Dict[str, Any]:
182         try:
183             json_match = re.search(r'\{.*\}', response, re.DOTALL)
184             if not json_match:
185                 raise ValueError("No valid JSON found in response")
186             return json.loads(json_match.group())
187         except json.JSONDecodeError as e:
188             raise ValueError(f"Invalid JSON in response: {str(e)}")
189 
190     def process_request(self, user_input: str) -> Any:
191         try:
192             response = ollama.generate(model=self.model, prompt=self._generate_promp\
193 t(user_input))
194             function_call = self._parse_ollama_response(response.response)
195 
196             if function_call["function"] == "query_database":
197                 return self.sqlite_tool.execute_query(function_call["parameters"]["q\
198 uery"])
199             elif function_call["function"] == "list_tables":
200                 return self.sqlite_tool.get_tables()
201             else:
202                 raise ValueError(f"Unknown function: {function_call['function']}")
203         except Exception as e:
204             raise RuntimeError(f"Request processing failed: {str(e)}")
205 
206 def main():
207     function_caller = OllamaFunctionCaller()
208     queries = [
209         "Show me all tables in the database",
210         "Get all users from the users table",
211         "What are the top 5 products by price?"
212     ]
213 
214     for query in queries:
215         try:
216             print(f"\nQuery: {query}")
217             result = function_caller.process_request(query)
218             print(f"Result: {result}")
219         except Exception as e:
220             print(f"Error processing query: {str(e)}")
221 
222 if __name__ == "__main__":
223     main()

 1 python /Users/markw/GITHUB/OllamaExamples/tool_sqlite.py 
 2 
 3 Query: Show me all tables in the database
 4 Result: ['example', 'users', 'products']
 5 
 6 Query: Get all users from the users table
 7 Result: [(1, 'Bob', 'bob@example.com'), (2, 'Susan', 'susan@test.net')]
 8 
 9 Query: What are the top 5 products by price?
10 Result: [(1, 'Laptop', 1200.0), (3, 'Laptop', 1200.0), (2, 'Keyboard', 75.5), (4, 'K\
11 eyboard', 75.5)]

 1 """
 2 Summarize text
 3 """
 4 
 5 from ollama import ChatResponse
 6 from ollama import chat
 7 
 8 
 9 def summarize_text(text: str, context: str = "") -> str:
10     """
11     Summarizes text
12 
13     Parameters:
14         text (str): text to summarize
15         context (str): another tool's output can at the application layer can be use\
16 d set the context for this tool.
17 
18     Returns:
19         a string of summarized text
20 
21     """
22     prompt = "Summarize this text (and be concise), returning only the summary with \
23 NO OTHER COMMENTS:\n\n"
24     if len(text.strip()) < 50:
25         text = context
26     elif len(context) > 50:
27         prompt = f"Given this context:\n\n{context}\n\n" + prompt
28 
29     summary: ChatResponse = chat(
30         model="llama3.2:latest",
31         messages=[
32             {"role": "system", "content": prompt},
33             {"role": "user", "content": text},
34         ],
35     )
36     return summary["message"]["content"]
37 
38 
39 # Function metadata for Ollama integration
40 summarize_text.metadata = {
41     "name": "summarize_text",
42     "description": "Summarizes input text",
43     "parameters": {"text": "string of text to summarize",
44                    "context": "optional context string"},
45 }
46 
47 # Export the functions
48 __all__ = ["summarize_text"]

 1 from tool_web_search import uri_to_markdown
 2 from tool_summarize_text import summarize_text
 3 
 4 from pprint import pprint
 5 
 6 import ollama
 7 
 8 # Map function names to function objects
 9 available_functions = {
10     "uri_to_markdown": uri_to_markdown,
11     "summarize_text": summarize_text,
12 }
13 
14 memory_context = ""
15 # User prompt
16 user_prompt = "Get the text of 'https://knowledgebooks.com' and then summarize the t\
17 ext."
18 
19 # Initiate chat with the model
20 response = ollama.chat(
21     model='llama3.2:latest',
22     messages=[{"role": "user", "content": user_prompt}],
23     tools=[uri_to_markdown, summarize_text],
24 )
25 
26 # Process the model's response
27 
28 pprint(response.message.tool_calls)
29 
30 for tool_call in response.message.tool_calls or []:
31     function_to_call = available_functions.get(tool_call.function.name)
32     print(
33         f"\n***** {function_to_call=}\n\nmemory_context[:70]:\n\n{memory_context[:70\
34 ]}\n\n*****\n"
35     )
36     if function_to_call:
37         print()
38         if len(memory_context) > 10:
39             tool_call.function.arguments["context"] = memory_context
40         print("\n* * tool_call.function.arguments:\n")
41         pprint(tool_call.function.arguments)
42         print(f"Arguments for {function_to_call.__name__}: {tool_call.function.argum\
43 ents}")
44         result = function_to_call(**tool_call.function.arguments)  # , memory_contex\
45 t)
46         print(f"\n\n** Output of {tool_call.function.name}: {result}")
47         memory_context = memory_context + "\n\n" + result
48     else:
49         print(f"\n\n** Function {tool_call.function.name} not found.")

  1 python /Users/markw/GITHUB/OllamaExamples/example_chain_web_summary.py 
  2 [ToolCall(function=Function(name='uri_to_markdown', arguments={'a_uri': 'https://kno\
  3 wledgebooks.com'})),
  4  ToolCall(function=Function(name='summarize_text', arguments={'context': '', 'text':\
  5  'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}))]
  6 
  7 ***** function_to_call=<function uri_to_markdown at 0x1047da200>
  8 
  9 memory_context[:70]:
 10 
 11 
 12 
 13 *****
 14 
 15 
 16 
 17 * * tool_call.function.arguments:
 18 
 19 {'a_uri': 'https://knowledgebooks.com'}
 20 Arguments for uri_to_markdown: {'a_uri': 'https://knowledgebooks.com'}
 21 INFO:httpx:HTTP Request: POST http://127.0.0.1:11434/api/chat "HTTP/1.1 200 OK"
 22 
 23 
 24 ** Output of uri_to_markdown: Contents of URI https://knowledgebooks.com is:
 25 # KnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web 
 26 
 27 KnowledgeBooks.com - research on the Knowledge Management, and the Semantic Web 
 28 
 29 KnowledgeBooks.com 
 30 
 31 Knowledgebooks.com 
 32 a sole proprietorship company owned by Mark Watson
 33 to promote Knowledge Management, Artificial Intelligence (AI), NLP, and Semantic Web\
 34  technologies.
 35 
 36 Site updated: December 1, 2018
 37 With the experience of working on Machine Learning and Knowledge Graph applications \
 38 for 30 years (at Google,
 39  Capital One, SAIC, Compass Labs, etc.) I am now concerned that the leverage of deep\
 40  learning and knowledge
 41  representation technologies are controlled by a few large companies, mostly in Chin\
 42 a and the USA. I am proud
 43  to be involved organizations like Ocean Protocol and Common Crawl that seek tp incr\
 44 ease the availability of quality data
 45  to individuals and smaller organizations.
 46 Traditional knowledge management tools relied on structured data often stored in rel\
 47 ational databases. Adding
 48  new relations to this data would require changing the schemas used to store data wh\
 49 ich could negatively
 50  impact exisiting systems that used that data. Relationships between data in traditi\
 51 onal systems was
 52  predefined by the structure/schema of stored data. With RDF and OWL based data mode\
 53 ling, relationships in
 54  data are explicitly defined in the data itself. Semantic data is inherently flexibl\
 55 e and extensible: adding
 56  new data and relationships is less likely to break older systems that relied on the\
 57  previous verisons of
 58  data.
 59 A complementary technology for knowledge management is the automated processing of u\
 60 nstructured text data
 61  into semantic data using natural language processing (NLP) and statistical-base tex\
 62 t analytics.
 63 We will help you integrate semantic web and text analytics technologies into your or\
 64 ganization by working
 65  with your staff in a mentoring role and also help as needed with initial developmen\
 66 t. All for reasonable consulting rates
 67 Knowledgebooks.com Technologies:
 68 
 69 SAAS KnowledgeBooks Semantic NLP Portal (KBSportal.com) used for
 70  in-house projects and available as a product to run on your servers.
 71 Semantic Web Ontology design and development
 72 Semantic Web application design and development using RDF data stores, PostgreSQL, a\
 73 nd MongoDB.
 74 
 75 Research
 76 Natural Language Processing (NLP) using deep learning
 77 Fusion of classic symbolic AI systems with deep learning models
 78 Linked data, semantic web, and Ontology's
 79 News ontology
 80 Note: this ontology was created in 2004 using the Protege modeling tool.
 81 About
 82 KnowledgeBooks.com is owned as a sole proprietor business by Mark and Carol Watson.
 83 Mark Watson is an author of 16 published books and a consultant specializing in the \
 84 JVM platform
 85  (Java, Scala, JRuby, and Clojure), artificial intelligence, and the Semantic Web.
 86 Carol Watson helps prepare training data and serves as the editor for Mark's publish\
 87 ed books.
 88 Privacy policy: this site collects no personal data or information on site visitors
 89 Hosted on Cloudflare Pages.
 90 
 91 
 92 ***** function_to_call=<function summarize_text at 0x107519260>
 93 
 94 memory_context[:70]:
 95 
 96 
 97 
 98 Contents of URI https://knowledgebooks.com is:
 99 # KnowledgeBooks.com 
100 
101 *****
102 
103 
104 
105 * * tool_call.function.arguments:
106 
107 {'context': '\n'
108             '\n'
109             'Contents of URI https://knowledgebooks.com is:\n'
110             '# KnowledgeBooks.com - research on the Knowledge Management, and '
111             'the Semantic Web \n'
112             '\n'
113             'KnowledgeBooks.com - research on the Knowledge Management, and '
114 ...
115             'Carol Watson helps prepare training data and serves as the editor '
116             "for Mark's published books.\n"
117             'Privacy policy: this site collects no personal data or '
118             'information on site visitors\n'
119             'Hosted on Cloudflare Pages.\n',
120  'text': 'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}
121 Arguments for summarize_text: {'context': "\n\nContents of URI https://knowledgebook\
122 s.com is:\n# KnowledgeBooks.com - research on the Knowledge Management, and the Sema
123 ntic Web \n\nKnowledgeBooks.com - research on the Knowledge Management, and the Sema
124 ntic Web \n\nKnowledgeBooks.com \n\nKnowledgebooks.com \na sole proprietorship compa
125 ny owned by Mark Watson\nto promote Knowledge Management, Artificial Intelligence (A
126 I), NLP, and Semantic Web technologies.
127 
128 ...
129 
130 \n\nResearch\nNatural Language Processing (NLP) using deep learning\nFusion of class\
131 ic symbolic AI systems with deep learning models\nLinked data, semantic web, and Ont
132 ology's\nNews ontology\nNote: this ontology was created in 2004 using the Protege mo
133 deling tool.\nAbout\nKnowledgeBooks.com is owned as a sole proprietor business by Ma
134 rk and Carol Watson.\nMark Watson is an author of 16 published books and a consultan
135 t specializing in the JVM platform\n (Java, Scala, JRuby, and Clojure), artificial i
136 ntelligence, and the Semantic Web.\nCarol Watson helps prepare training data and ser
137 ves as the editor for Mark's published books.\nPrivacy policy: this site collects no
138  personal data or information on site visitors\nHosted on Cloudflare Pages.\n", 'tex
139 t': 'uri_to_markdown(a_uri = "https://knowledgebooks.com")'}
140 
141 
142 ** Output of summarize_text: # Knowledge Management and Semantic Web Research
143 ## About KnowledgeBooks.com
144 A sole proprietorship company by Mark Watson promoting AI, NLP, and Semantic Web tec\
145 hnologies.
146 ### Technologies
147 - **SAAS KnowledgeBooks**: Semantic NLP Portal for in-house projects and product sal\
148 es.
149 - **Semantic Web Development**: Ontology design and application development using RD\
150 F data stores.
151 
152 ### Research Areas
153 - Natural Language Processing (NLP) with deep learning
154 - Fusion of symbolic AI systems with deep learning models
155 - Linked data, semantic web, and ontologies

  1 """
  2 Provides functions for web searching and HTML to Markdown conversion
  3 and for returning the contents of a URI as plain text (with minimal markdown)
  4 """
  5 
  6 from typing import Dict, Any
  7 import requests
  8 from bs4 import BeautifulSoup
  9 import re
 10 from urllib.parse import urlparse
 11 import html
 12 from ollama import chat
 13 import json
 14 from tool_summarize_text import summarize_text
 15 
 16 import requests
 17 import os
 18 import logging
 19 from pprint import pprint
 20 from bs4 import BeautifulSoup
 21 
 22 logging.basicConfig(level=logging.INFO)
 23 
 24 api_key = os.environ.get("BRAVE_SEARCH_API_KEY")
 25 if not api_key:
 26     raise ValueError(
 27         "API key not found. Set 'BRAVE_SEARCH_API_KEY' environment variable."
 28     )
 29 
 30 
 31 def replace_html_tags_with_text(html_string):
 32     soup = BeautifulSoup(html_string, "html.parser")
 33     return soup.get_text()
 34 
 35 
 36 def uri_to_markdown(a_uri: str) -> Dict[str, Any]:
 37     """
 38     Fetches content from a URI and converts HTML to markdown-style text
 39 
 40     Args:
 41         a_uri (str): URI to fetch and convert
 42 
 43     Returns:
 44         web page text converted converted markdown content
 45     """
 46     try:
 47         # Validate URI
 48         parsed = urlparse(a_uri)
 49         if not all([parsed.scheme, parsed.netloc]):
 50             return f"Invalid URI: {a_uri}"
 51 
 52         # Fetch content
 53         headers = {
 54             "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537\
 55 .36"
 56         }
 57         response = requests.get(a_uri, headers=headers, timeout=10)
 58         response.raise_for_status()
 59 
 60         # Parse HTML
 61         soup = BeautifulSoup(response.text, "html.parser")
 62 
 63         # Get title
 64         title = soup.title.string if soup.title else ""
 65 
 66         # Get text and clean up
 67         text = soup.get_text()
 68 
 69         # Clean up the text
 70         text = re.sub(r"\n\s*\n", "\n\n", text)  # Remove multiple blank lines
 71         text = re.sub(r" +", " ", text)  # Remove multiple spaces
 72         text = html.unescape(text)  # Convert HTML entities
 73         text = text.strip()
 74 
 75         return f"Contents of URI {a_uri} is:\n# {title}\n\n{text}\n"
 76 
 77     except requests.RequestException as e:
 78         return f"Network error: {str(e)}"
 79 
 80     except Exception as e:
 81         return f"Error processing URI: {str(e)}"
 82 
 83 
 84 def search_web(query: str, max_results: int = 5) -> str:
 85     """
 86     Performs a web search and returns results
 87     Note: This is a placeholder. Implement with your preferred search API.
 88 
 89     Args:
 90         query (str): Search query
 91         max_results (int): Maximum number of results to return
 92 
 93     Returns:
 94         Dict[str, Any]: Dictionary containing:
 95             - 'results': List of search results
 96             - 'count': Number of results found
 97             - 'error': Error message if any, None otherwise
 98     """
 99 
100     # Placeholder for search implementation
101     return {
102         "results": [],
103         "count": 0,
104         "error": "Web search not implemented. Please implement with your preferred s\
105 earch API.",
106     }
107 
108 
109 def brave_search_summaries(
110     query,
111     num_results=3,
112     url="https://api.search.brave.com/res/v1/web/search",
113     api_key=api_key,
114 ):
115     headers = {"X-Subscription-Token": api_key, "Content-Type": "application/json"}
116     params = {"q": query, "count": num_results}
117 
118     response = requests.get(url, headers=headers, params=params)
119     ret = []
120 
121     if response.status_code == 200:
122         search_results = response.json()
123         ret = [
124             {
125                 "title": result.get("title"),
126                 "url": result.get("url"),
127                 "description": replace_html_tags_with_text(result.get("description")\
128 ),
129             }
130             for result in search_results.get("web", {}).get("results", [])
131         ]
132         logging.info("Successfully retrieved results.")
133     else:
134         try:
135             error_info = response.json()
136             logging.error(f"Error {response.status_code}: {error_info.get('message')\
137 }")
138         except json.JSONDecodeError:
139             logging.error(f"Error {response.status_code}: {response.text}")
140 
141     return ret
142 
143 def brave_search_text(query, num_results=3):
144     summaries = brave_search_summaries(query, num_results)
145     ret = ""
146     for s in summaries:
147         url = s["url"]
148         text = uri_to_markdown(url)
149         summary = summarize_text(
150             f"Given the query:\n\n{query}\n\nthen, summarize text removing all mater\
151 ial that is not relevant to the query and then be very concise for a very short summ
152 ary:\n\n{text}\n"
153         )
154         ret += ret + summary
155     print("\n\n-----------------------------------")
156     return ret
157 
158 # Function metadata for Ollama integration
159 uri_to_markdown.metadata = {
160     "name": "uri_to_markdown",
161     "description": "Converts web page content to markdown-style text",
162     "parameters": {"a_uri": "URI of the web page to convert"},
163 }
164 
165 search_web.metadata = {
166     "name": "search_web",
167     "description": "Performs a web search and returns results",
168     "parameters": {
169         "query": "Search query",
170         "max_results": "Maximum number of results to return",
171     },
172 }
173 
174 # Export the functions
175 __all__ = ["uri_to_markdown", "search_web"]

 1 """
 2 Judge results from LLM generation from prompts
 3 """
 4 
 5 from typing import Optional, Dict, Any
 6 from pathlib import Path
 7 import json
 8 import re
 9 from pprint import pprint
10 
11 import ollama
12 
13 client = ollama.Client()
14 
15 def judge_results(original_prompt: str, llm_gen_results: str) -> Dict[str, str]:
16     """
17     Takes an original prompt to a LLM and the output results
18 
19     Args:
20         original_prompt (str): original prompt to a LLM
21         llm_gen_results (str): output from the LLM that this function judges for acc\
22 uracy
23 
24     Returns:
25         result: str: string that is one character with one of these values:
26             - 'B': Bad result
27             - 'G': A Good result
28     """
29     try:
30         messages = [
31             {"role": "system", "content": "Always judge this output for correctness.\
32 "},
33             {"role": "user", "content": f"Evaluate this output:\n\n{llm_gen_results}\
34 \n\nfor this prompt:\n\n{original_prompt}\n\nDouble check your work and explain your
35  thinking in a few sentences. End your output with a Y or N answer"},
36         ]
37 
38         response = client.chat(
39             model="qwen2.5-coder:14b", # "llama3.2:latest",
40             messages=messages,
41         )
42 
43         r = response.message.content.strip()
44         print(f"\n\noriginal COT response:\n\n{r}\n\n")
45 
46         # look at the end of the response for the Y or N judgement
47         s = r.lower()
48         # remove all non-alphabetic characters:
49         s = re.sub(r'[^a-zA-Z]', '', s).strip()
50 
51         return {'judgement': s[-1].upper(), 'reasoning': r[1:].strip()}
52 
53     except Exception as e:
54         print(f"\n\n***** {e=}\n\n")
55         return {'judgement': 'E', 'reasoning': str(e)}  # on any error, assign 'E' r\
56 esult

 1 $ cd OllamaEx
 2 $ python example_judge.py 
 3 
 4 ==================================================
 5  Judge output from a LLM
 6 ==================================================
 7 
 8 ==================================================
 9  First test: should be Y, or good
10 ==================================================
11 
12 
13 original COT response:
14 
15 The given output correctly calculates the absolute value of age differences for each\
16  pair:
17 
18 - Sally (55) and John (18): \( |55 - 18| = 37 \)
19 - Sally (55) and Mary (31): \( |55 - 31| = 24 \)
20 - John (18) and Mary (31): \( |31 - 18| = 13 \)
21 
22 These calculations are accurate, matching the prompt's requirements. Therefore, the \
23 answer is Y.
24 
25 
26 
27 ** JUDGEMENT ***
28 
29 judgement={'judgement': 'Y', 'reasoning': "The given output correctly calculates the\
30  absolute value of age differences for each pair:\n\n- Sally (55) and John (18): \\(
31  |55 - 18| = 37 \\)\n- Sally (55) and Mary (31): \\( |55 - 31| = 24 \\)\n- John (18)
32  and Mary (31): \\( |31 - 18| = 13 \\)\n\nThese calculations are accurate, matching 
33 the prompt's requirements. Therefore, the answer is Y."}
34 
35 ==================================================
36  Second test: should be N, or bad
37 ==================================================
38 
39 
40 original COT response:
41 
42 Let's evaluate the given calculations step by step:
43 
44 1. Sally (55) - John (18) = 37. The difference is calculated as 55 - 18, which equal\
45 s 37.
46 2. Sally (55) - Mary (31) = 24. The difference is calculated as 55 - 31, which equal\
47 s 24.
48 3. John (18) - Mary (31) = -13. However, the absolute value of this difference is |1\
49 8 - 31| = 13.
50 
51 The given output shows:
52 - Sally and John: 55 - 18 = 31. This should be 37.
53 - Sally and Mary: 55 - 31 = 24. This is correct.
54 - John and Mary: 31 - 18 = 10. This should be 13.
55 
56 The output contains errors in the first and third calculations. Therefore, the answe\
57 r is:
58 
59 N
60 
61 ** JUDGEMENT ***
62 
63 judgement={'judgement': 'N', 'reasoning': "et's evaluate the given calculations step\
64  by step:\n\n1. Sally (55) - John (18) = 37. The difference is calculated as 55 - 18
65 , which equals 37.\n2. Sally (55) - Mary (31) = 24. The difference is calculated as 
66 55 - 31, which equals 24.\n3. John (18) - Mary (31) = -13. However, the absolute val
67 ue of this difference is |18 - 31| = 13.\n\nThe given output shows:\n- Sally and Joh
68 n: 55 - 18 = 31. This should be 37.\n- Sally and Mary: 55 - 31 = 24. This is correct
69 .\n- John and Mary: 31 - 18 = 10. This should be 13.\n\nThe output contains errors i
70 n the first and third calculations. Therefore, the answer is:\n\nN"}

  1 import json
  2 from typing import List, Dict, Optional, Iterator
  3 import ollama
  4 from ollama import GenerateResponse
  5 
  6 
  7 def clean_json_response(response: str) -> str:
  8     """
  9     Cleans the response string by removing markdown code blocks and other formatting
 10     """
 11     # Remove markdown code block indicators
 12     response = response.replace("json", "").replace("```", "")
 13     # Strip whitespace
 14     response = response.strip()
 15     return response
 16 
 17 def evaluate_llm_conversation(
 18     chat_history: List[Dict[str, str]],
 19     evaluation_criteria: Optional[List[str]] = None,
 20     model: str = "llama3.1" # older model that is good at generating JSON
 21 ) -> Dict[str, any]:
 22     """
 23     Evaluates a chat history using Ollama to run the evaluation model.
 24 
 25     Args:
 26         chat_history: List of dictionaries containing the conversation
 27         evaluation_criteria: Optional list of specific criteria to evaluate
 28         model: Ollama model to use for evaluation
 29 
 30     Returns:
 31         Dictionary containing evaluation results
 32     """
 33     if evaluation_criteria is None:
 34         evaluation_criteria = [
 35             "Response accuracy",
 36             "Coherence and clarity",
 37             "Helpfulness",
 38             "Task completion",
 39             "Natural conversation flow"
 40         ]
 41 
 42     # Format chat history for evaluation
 43     formatted_chat = "\n".join([
 44         f"{'User' if msg['role'] == 'user' else 'Assistant'}: {msg['content']}"
 45         for msg in chat_history
 46     ])
 47 
 48     # Create evaluation prompt
 49     evaluation_prompt = f"""
 50     Please evaluate the following conversation between a user and an AI assistant.
 51     Focus on these criteria: {', '.join(evaluation_criteria)}
 52 
 53     Conversation:
 54     {formatted_chat}
 55 
 56     Provide a structured evaluation with:
 57     1. Scores (1-10) for each criterion
 58     2. Brief explanation for each score
 59     3. Overall assessment
 60     4. Suggestions for improvement
 61 
 62     Format your response as JSON.
 63     """
 64 
 65     try:
 66         # Get evaluation from Ollama
 67         response: GenerateResponse | Iterator[GenerateResponse] = ollama.generate(
 68             model=model,
 69             prompt=evaluation_prompt,
 70             system="You are an expert AI evaluator. Provide detailed, objective asse\
 71 ssments in JSON format."
 72         )
 73 
 74         response_clean: str = clean_json_response(response['response'])
 75 
 76         # Parse the response to extract JSON
 77         try:
 78             evaluation_result = json.loads(response_clean)
 79         except json.JSONDecodeError:
 80             # Fallback if response isn't proper JSON
 81             evaluation_result = {
 82                 "error": "Could not parse evaluation as JSON",
 83                 "raw_response": response_clean
 84             }
 85 
 86         return evaluation_result
 87 
 88     except Exception as e:
 89         return {
 90             "error": f"Evaluation failed: {str(e)}",
 91             "status": "failed"
 92         }
 93 
 94 # Example usage
 95 if __name__ == "__main__":
 96     # Sample chat history
 97     sample_chat = [
 98         {"role": "user", "content": "What's the capital of France?"},
 99         {"role": "assistant", "content": "The capital of France is Paris."},
100         {"role": "user", "content": "Tell me more about it."},
101         {"role": "assistant", "content": "Paris is the largest city in France and se\
102 rves as the country's political, economic, and cultural center. It's known for landm
103 arks like the Eiffel Tower, Louvre Museum, and Notre-Dame Cathedral."}
104     ]
105 
106     # Run evaluation
107     result = evaluate_llm_conversation(sample_chat)
108     print(json.dumps(result, indent=2))

 1 $ cd OllamaEx 
 2 $ python tool_llm_eval.py 
 3 {
 4   "evaluation": {
 5     "responseAccuracy": {
 6       "score": 9,
 7       "explanation": "The assistant correctly answered the user's question about the\
 8  capital of France, and provided accurate information when the user asked for more d
 9 etails."
10     },
11     "coherenceAndClarity": {
12       "score": 8,
13       "explanation": "The assistant's responses were clear and easy to understand. H\
14 owever, there was a slight shift in tone from a simple answer to a more formal descr
15 iption."
16     },
17     "helpfulness": {
18       "score": 9,
19       "explanation": "The assistant provided relevant information that helped the us\
20 er gain a better understanding of Paris. The response was thorough and answered the 
21 user's follow-up question."
22     },
23     "taskCompletion": {
24       "score": 10,
25       "explanation": "The assistant completed both tasks: providing the capital of F\
26 rance and elaborating on it with additional context."
27     },
28     "naturalConversationFlow": {
29       "score": 7,
30       "explanation": "While the responses were clear, they felt a bit abrupt. The as\
31 sistant could have maintained a more conversational tone or encouraged further discu
32 ssion."
33     }
34   },
35   "overallAssessment": {
36     "score": 8.5,
37     "explanation": "The assistant demonstrated strong technical knowledge and was ab\
38 le to provide accurate information on demand. However, there were some minor lapses 
39 in natural conversation flow and coherence."
40   },
41   "suggestionsForImprovement": [
42     {
43       "improvementArea": "NaturalConversationFlow",
44       "description": "Consider using more conversational language or prompts to enga\
45 ge users further."
46     },
47     {
48       "improvementArea": "CoherenceAndClarity",
49       "description": "Use transitional phrases and maintain a consistent tone throug\
50 hout the conversation."
51     }
52   ]
53 }

 1 You are a fair judge and an expert at identifying false hallucinations and you are t\
 2 asked with evaluating the accuracy of an AI-generated answer to a given context. Ana
 3 lyze the provided INPUT, CONTEXT, and OUTPUT to determine if the OUTPUT contains any
 4  hallucinations or false information.
 5 
 6 Guidelines:
 7 1. The OUTPUT must not contradict any information given in the CONTEXT.
 8 2. The OUTPUT must not introduce new information beyond what's provided in the CONTE\
 9 XT.
10 3. The OUTPUT should not contradict well-established facts or general knowledge.
11 4. Check that the OUTPUT doesn't oversimplify or generalize information in a way tha\
12 t changes its meaning or accuracy.
13 
14 Analyze the text thoroughly and assign a hallucination score between 0 and 1, where:
15 - 0.0: The OUTPUT is unfaithful or is incorrect to the CONTEXT and the user's INPUT
16 - 1.0: The OUTPUT is entirely accurate abd faithful to the CONTEXT and the user's IN\
17 PUT
18 
19 INPUT:
20 {input}
21 
22 CONTEXT:
23 {context}
24 
25 OUTPUT:
26 {output}
27 
28 Provide your judgement in JSON format:
29 {{
30     "score": <your score between 0.0 and 1.0>,
31     "reason": [
32         <list your reasoning as Python strings>
33     ]
34 }}

 1 """
 2 Provides functions detecting hallucinations by other LLMs
 3 """
 4 
 5 from typing import Optional, Dict, Any
 6 from pathlib import Path
 7 from pprint import pprint
 8 import json
 9 from ollama import ChatResponse
10 from ollama import chat
11 
12 def read_anti_hallucination_template() -> str:
13     """
14     Reads the anti-hallucination template file and returns the content
15     """
16     template_path = Path(__file__).parent / "templates" / "anti_hallucination.txt"
17     with template_path.open("r", encoding="utf-8") as f:
18         content = f.read()
19         return content
20 
21 TEMPLATE = read_anti_hallucination_template()
22 
23 def detect_hallucination(user_input: str, context: str, output: str) -> str:
24     """
25     Given user input, context, and LLM output, detect hallucination
26 
27     Args:
28         user_input (str): User's input text prompt
29         context (str): Context text for LLM
30         output (str): LLM's output text that is to be evaluated as being a hallucina\
31 tion)
32 
33     Returns: JSON data:
34      {
35        "score": <your score between 0.0 and 1.0>,
36        "reason": [
37          <list your reasoning as bullet points>
38        ]
39      }
40     """
41     prompt = TEMPLATE.format(input=user_input, context=context, output=output)
42     response: ChatResponse = chat(
43         model="llama3.2:latest",
44         messages=[
45             {"role": "system", "content": prompt},
46             {"role": "user", "content": output},
47         ],
48     )
49     try:
50         return json.loads(response.message.content)
51     except json.JSONDecodeError:
52         print(f"Error decoding JSON: {response.message.content}")
53     return {"score": 0.0, "reason": ["Error decoding JSON"]}
54 
55 
56 # Export the functions
57 __all__ = ["detect_hallucination"]
58 
59 ## Test only code:
60 
61 def main():
62     def separator(title: str):
63         """Prints a section separator"""
64         print(f"\n{'=' * 50}")
65         print(f" {title}")
66         print('=' * 50)
67 
68     # Test file writing
69     separator("Detect hallucination from a LLM")
70 
71     test_prompt = "Sally is 55, John is 18, and Mary is 31. What are pairwise combin\
72 ations of the absolute value of age differences?"
73     test_context = "Double check all math results."
74     test_output = "Sally and John:  55 - 18 = 31. Sally and Mary:  55 - 31 = 24. Joh\
75 n and Mary:  31 - 18 = 10."
76     judgement = detect_hallucination(test_prompt, test_context, test_output)
77     print(f"\n** JUDGEMENT ***\n")
78     pprint(judgement)
79 
80 if __name__ == "__main__":
81     try:
82         main()
83     except Exception as e:
84         print(f"An error occurred: {str(e)}")

 1 python /Users/markw/GITHUB/OllamaExamples/tool_anti_hallucination.py 
 2 
 3 ==================================================
 4  Detect hallucination from a LLM
 5 ==================================================
 6 
 7 ** JUDGEMENT ***
 8 
 9 {'reason': ['The OUTPUT claims that the absolute value of age differences are '
10             '31, 24, and 10 for Sally and John, Sally and Mary, and John and '
11             'Mary respectively. However, this contradicts the CONTEXT, as the '
12             'CONTEXT asks to double-check math results.',
13             'The OUTPUT does not introduce new information, but it provides '
14             'incorrect calculations: Sally and John: 55 - 18 = 37, Sally and '
15             'Mary: 55 - 31 = 24, John and Mary: 31 - 18 = 13. Therefore, the '
16             'actual output should be recalculated to ensure accuracy.',
17             'The OUTPUT oversimplifies the age differences by not considering '
18             "the order of subtraction (i.e., John's age subtracted from "
19             "Sally's or Mary's). However, this is already identified as a "
20             'contradiction in point 1.'],
21  'score': 0.0}

1 python3.11 -m venv venv
2 source venv/bin/activate
3 python3.11 -m pip install -r requirements.txt
4 python3.11 smolagents_test.py

 1 smolagents
 2 litellm[proxy]
 3 requests
 4 beautifulsoup4
 5 ollama
 6 langchain
 7 langchain-community
 8 langchain-ollama
 9 langgraph
10 rdflib
11 kuzu
12 langchain_openai
13 tabulate

 1 """
 2 smolagents example program (slightly modified)
 3 """
 4 
 5 from smolagents.agents import ToolCallingAgent
 6 from smolagents import tool, LiteLLMModel
 7 from typing import Optional
 8 
 9 model = LiteLLMModel(
10     model_id="ollama_chat/llama3.2:latest",
11     api_base="http://localhost:11434",
12     api_key="your-api-key" # not used
13 )
14 
15 @tool
16 def get_weather(location: str, celsius: Optional[bool] = False) -> str:
17     """
18     Get weather in the next days at given location.
19     Secretly this tool does not care about the location, it hates the weather everyw\
20 here.
21 
22     Args:
23         location: the location
24         celsius: the temperature
25     """
26     return "The weather is UNGODLY with torrential rains and temperatures below -10°\
27 C"
28 
29 agent = ToolCallingAgent(tools=[get_weather], model=model)
30 
31 print(agent.run("What's the weather like in Paris?"))

 1 """
 2 Wrapper for book example tools for smloagents compatibility
 3 """
 4 from pathlib import Path
 5 
 6 from smolagents import tool, LiteLLMModel
 7 from typing import Optional
 8 from pprint import pprint
 9 
10 from tool_file_dir import list_directory
11 
12 @tool
13 def sa_list_directory(list_dots: Optional[bool]=None) -> str:
14     """
15     Lists files and directories in the current working directory
16 
17     Args:
18         list_dots: optional boolean (if true, include dot files)
19 
20     Returns:
21         string with directory name, followed by list of files in the directory
22     """
23     lst = list_directory()
24     pprint(lst)
25     return lst
26 
27 @tool
28 def read_file_contents(file_path: str) -> str:
29     """
30     Reads contents from a file and returns the text
31 
32     Args:
33         file_path: Path to the file to read
34 
35     Returns:
36         Contents of the file as a string
37     """
38     try:
39         path = Path(file_path)
40         if not path.exists():
41             return f"File not found: {file_path}"
42 
43         with path.open("r", encoding="utf-8") as f:
44             content = f.read()
45             return f"Contents of file '{file_path}' is:\n{content}\n"
46 
47     except Exception as e:
48         return f"Error reading file '{file_path}' is: {str(e)}"
49 
50 @tool
51 def summarize_directory() -> str:
52     """
53     Summarizes the files and directories in the current working directory
54 
55     Returns:
56         string with directory name, followed by summary of files in the directory
57     """
58     lst = list_directory()
59     num_files = len(lst)
60     num_dirs = len([x for x in lst if x[1] == 'directory'])
61     num_files = num_files - num_dirs
62     return f"Current directory contains {num_files} files and {num_dirs} directories\
63 ."

 1 from smolagents.agents import ToolCallingAgent
 2 from smolagents import tool, LiteLLMModel
 3 from typing import Optional
 4 
 5 from smolagents_tools import sa_list_directory
 6 from smolagents_tools import summarize_directory
 7 from smolagents_tools import read_file_contents
 8 
 9 model = LiteLLMModel(
10     model_id="ollama_chat/llama3.2:latest",
11     api_base="http://localhost:11434",
12     api_key="your-api-key" # not used
13 )
14 
15 agent = ToolCallingAgent(tools=[sa_list_directory,
16                                 summarize_directory,
17                                 read_file_contents],
18                          model=model)
19 
20 print(agent.run("What are the files in the current directory? Describe the current d\
21 irectory"))
22 
23 print(agent.run("Which Python scripts evaluate the performance of LLMs?"))

  1 python smolagents_agent_test1.py 
  2 ╭────────────────────────────────── New run ───────────────────────────────────╮
  3 │                                                                              │
  4 │ List the Python programs in the current directory, and then tell me which    │
  5 │ Python programs in the current directory evaluate the performance of LLMs?   │
  6 │                                                                              │
  7 ╰─ LiteLLMModel - ollama_chat/llama3.2:latest ─────────────────────────────────╯
  8 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  9 ╭──────────────────────────────────────────────────────────────────────────────╮
 10 │ Calling tool: 'sa_list_directory' with arguments: {'list_dots': True}        │
 11 ╰──────────────────────────────────────────────────────────────────────────────╯
 12 ('Contents of current directory: [Makefile, README.md, __pycache__, data, '
 13  'example_chain_read_summary.py, example_chain_web_summary.py, '
 14  'example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, '
 15  'langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, '
 16  'short_programs, smolagents_agent_test1.py, smolagents_test.py, '
 17  'smolagents_tools.py, templates, tool_anti_hallucination.py, '
 18  'tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, '
 19  'tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, '
 20  'tool_web_search.py, venv]')
 21 Observations: Contents of current directory: [Makefile, README.md, __pycache__, 
 22 data, example_chain_read_summary.py, example_chain_web_summary.py, 
 23 example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, 
 24 langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, 
 25 short_programs, smolagents_agent_test1.py, smolagents_test.py, 
 26 smolagents_tools.py, templates, tool_anti_hallucination.py, 
 27 tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, 
 28 tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, 
 29 venv]
 30 [Step 0: Duration 4.49 seconds| Input tokens: 1,347 | Output tokens: 79]
 31 
 32 ...
 33 
 34 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 35 ╭──────────────────────────────────────────────────────────────────────────────╮
 36 │ Calling tool: 'sa_summarize_directory' with arguments: {}                    │
 37 ╰──────────────────────────────────────────────────────────────────────────────╯
 38 lst='Contents of current directory: [Makefile, README.md, __pycache__, data, example\
 39 _chain_read_summary.py, example_chain_web_summary.py, example_judge.py, graph_kuzu_f
 40 rom_text.py, graph_kuzu_property_example.py, langgraph_agent_test.py, ollama_tools_e
 41 xamples.py, requirements.txt, short_programs, smolagents_agent_test1.py, smolagents_
 42 test.py, smolagents_tools.py, templates, tool_anti_hallucination.py, tool_file_conte
 43 nts.py, tool_file_dir.py, tool_judge_results.py, tool_llm_eval.py, tool_sqlite.py, t
 44 ool_summarize_text.py, tool_web_search.py, venv]'
 45 response.message.content="Based on the file names provided, here's a summary of the \
 46 contents and my educated guesses for their purposes:\n\n1. **Makefile**: A build scr
 47 ipt used to automate compilation, installation, or other tasks.\n2. **README.md**: A
 48  markdown file providing an introduction to the project, its purpose, and how to get
 49  started with it.\n3. **__pycache__**: This is a hidden directory generated by Pytho
 50 n's bytecode compiler. It likely contains compiled versions of Python code in the cu
 51 rrent directory.\n4. **data**: A directory containing data used for training or test
 52 ing models, simulations, or other computational tasks.\n5. **example_chain_read_summ
 53 ary.py**: A script that generates summaries from reading chains (e.g., text from a d
 54 ocument). Its purpose is likely related to natural language processing (NLP) or text
 55  analysis.\n6. **example_chain_web_summary.py**: Similar to the previous one, but th
 56 is script seems to be focused on web-based applications or online content summarizat
 57 ion.\n7. **example_judge.py**: A script that evaluates judges or other aspects of a 
 58 system. Its purpose might involve testing, validation, or quality control in a speci
 59 fic domain (e.g., AI, programming).\n8. **graph_kuzu_from_text.py**: A script that g
 60 enerates graphs from text data. This could be related to graph analysis, network sci
 61 ence, or text mining.\n9. **graph_kuzu_property_example.py**: Another script related
 62  to graph generation, possibly used for demonstrating specific properties or charact
 63 eristics of the generated graphs.\n10. **langgraph_agent_test.py**: A test script fo
 64 r a language graph agent, which likely involves machine learning, natural language p
 65 rocessing, or other AI-related tasks.\n11. **ollama_tools_examples.py**: A file cont
 66 aining examples of OLLAMA (Open Language Learning with Multi-Agent) tools and their 
 67 usage. This seems to be related to AI-powered language understanding and generation.
 68 \n12. **requirements.txt**: A text file specifying the dependencies required to run 
 69 the project, including Python packages and other software.\n13. **short_programs**: 
 70 A directory containing short programs or scripts that demonstrate specific technique
 71 s, concepts, or examples in the field of AI or programming.\n14. **smolagents_agent_
 72 test1.py**: Another test script for a small agent (SMOLAgent), which is likely invol
 73 ved in autonomous decision-making or other complex system simulations.\n15. **smolag
 74 ents_test.py**: A general-purpose test file for SMOLAgents, used to verify their fun
 75 ctionality and performance.\n16. **smolagents_tools.py**: A script containing tools 
 76 or utilities specifically designed for working with SMOLAgents or related projects.\
 77 n17. **templates**: A directory containing templates for generating documents, repor
 78 ts, or other content in a specific format (e.g., LaTeX).\n18. **tool_anti_hallucinat
 79 ion.py**: A script that addresses issues of hallucination (i.e., when an AI model pr
 80 ovides false information) and aims to mitigate these errors.\n19. **tool_file_conten
 81 ts.py**, **tool_file_dir.py**, **tool_judge_results.py**, and other similar tool scr
 82 ipts: These tools seem to be designed for various computational tasks, such as:\n\t*
 83  `tool_file_contents.py`: Analyzing the contents of files.\n\t* `tool_file_dir.py`: 
 84 Examining or manipulating directory structures.\n\t* `tool_judge_results.py`: Evalua
 85 ting the performance or outcomes of a system or model.\n\n20. **tool_llm_eval.py**: 
 86 A script for evaluating Large Language Models (LLMs) and their capabilities, likely 
 87 involving text analysis, sentiment detection, or other NLP tasks.\n21. **tool_sqlite
 88 .py**: A tool that interacts with SQLite databases, possibly used for data storage, 
 89 management, or querying.\n22. **tool_summarize_text.py**: A script designed to summa
 90 rize long pieces of text into shorter versions, possibly using machine learning algo
 91 rithms.\n23. **tool_web_search.py**: A tool that performs web searches or retrieves 
 92 information from online sources, which could involve natural language processing (NL
 93 P) and web scraping techniques.\n\n24. **venv**: A directory generated by Python's v
 94 irtual environment module, used to isolate dependencies and manage a specific Python
 95  environment for the project.\n\nKeep in mind that this is an educated guess based o
 96 n common file name conventions and the context provided. The actual purposes of thes
 97 e files might differ depending on the specific project or domain they are related to
 98 ."
 99 Observations: Summary of directory:Based on the file names provided, here's a 
100 summary of the contents and my educated guesses for their purposes:
101 
102 1. **Makefile**: A build script used to automate compilation, installation, or 
103 other tasks.
104 2. **README.md**: A markdown file providing an introduction to the project, its 
105 purpose, and how to get started with it.
106 3. **__pycache__**: This is a hidden directory generated by Python's bytecode 
107 compiler. It likely contains compiled versions of Python code in the current 
108 directory.
109 4. **data**: A directory containing data used for training or testing models, 
110 simulations, or other computational tasks.
111 5. **example_chain_read_summary.py**: A script that generates summaries from 
112 reading chains (e.g., text from a document). Its purpose is likely related to 
113 natural language processing (NLP) or text analysis.
114 6. **example_chain_web_summary.py**: Similar to the previous one, but this 
115 script seems to be focused on web-based applications or online content 
116 summarization.
117 7. **example_judge.py**: A script that evaluates judges or other aspects of a 
118 system. Its purpose might involve testing, validation, or quality control in a 
119 specific domain (e.g., AI, programming).
120 8. **graph_kuzu_from_text.py**: A script that generates graphs from text data. 
121 This could be related to graph analysis, network science, or text mining.
122 9. **graph_kuzu_property_example.py**: Another script related to graph 
123 generation, possibly used for demonstrating specific properties or 
124 characteristics of the generated graphs.
125 10. **langgraph_agent_test.py**: A test script for a language graph agent, which
126 likely involves machine learning, natural language processing, or other 
127 AI-related tasks.
128 11. **ollama_tools_examples.py**: A file containing examples of OLLAMA (Open 
129 Language Learning with Multi-Agent) tools and their usage. This seems to be 
130 related to AI-powered language understanding and generation.
131 12. **requirements.txt**: A text file specifying the dependencies required to 
132 run the project, including Python packages and other software.
133 13. **short_programs**: A directory containing short programs or scripts that 
134 demonstrate specific techniques, concepts, or examples in the field of AI or 
135 programming.
136 14. **smolagents_agent_test1.py**: Another test script for a small agent 
137 (SMOLAgent), which is likely involved in autonomous decision-making or other 
138 complex system simulations.
139 15. **smolagents_test.py**: A general-purpose test file for SMOLAgents, used to 
140 verify their functionality and performance.
141 16. **smolagents_tools.py**: A script containing tools or utilities specifically
142 designed for working with SMOLAgents or related projects.
143 17. **templates**: A directory containing templates for generating documents, 
144 reports, or other content in a specific format (e.g., LaTeX).
145 18. **tool_anti_hallucination.py**: A script that addresses issues of 
146 hallucination (i.e., when an AI model provides false information) and aims to 
147 mitigate these errors.
148 19. **tool_file_contents.py**, **tool_file_dir.py**, **tool_judge_results.py**, 
149 and other similar tool scripts: These tools seem to be designed for various 
150 computational tasks, such as:
151         * `tool_file_contents.py`: Analyzing the contents of files.
152         * `tool_file_dir.py`: Examining or manipulating directory structures.
153         * `tool_judge_results.py`: Evaluating the performance or outcomes of a 
154 system or model.
155 
156 20. **tool_llm_eval.py**: A script for evaluating Large Language Models (LLMs) 
157 and their capabilities, likely involving text analysis, sentiment detection, or 
158 other NLP tasks.
159 21. **tool_sqlite.py**: A tool that interacts with SQLite databases, possibly 
160 used for data storage, management, or querying.
161 22. **tool_summarize_text.py**: A script designed to summarize long pieces of 
162 text into shorter versions, possibly using machine learning algorithms.
163 23. **tool_web_search.py**: A tool that performs web searches or retrieves 
164 information from online sources, which could involve natural language processing
165 (NLP) and web scraping techniques.
166 
167 24. **venv**: A directory generated by Python's virtual environment module, used
168 to isolate dependencies and manage a specific Python environment for the 
169 project.
170 
171 Keep in mind that this is an educated guess based on common file name 
172 conventions and the context provided. The actual purposes of these files might 
173 differ depending on the specific project or domain they are related to.
174 [Step 3: Duration 21.53 seconds| Input tokens: 6,779 | Output tokens: 133]
175 
176 ...
177 
178 Reached max iterations.
179 Final answer: [{'id': '9630af1b-498f-4604-ab1b-e4139607cb02', 'type': 
180 'function', 'function': {'name': 'sa_list_directory', 'arguments': {'list_dots':
181 True}}}, {'id': '2143745d-5ec1-4711-8813-228398bf36f8', 'type': 'function', 
182 'function': {'name': 'sa_summarize_directory', 'arguments': {}}}]
183 
184 ...
185 
186 The Python programs in the current directory are:
187 1. example_chain_read_summary.py
188 2. example_chain_web_summary.py
189 3. example_judge.py
190 4. graph_kuzu_from_text.py
191 5. graph_kuzu_property_example.py
192 6. langgraph_agent_test.py
193 7. ollama_tools_examples.py
194 8. tool_anti_hallucination.py
195 9. tool_file_contents.py
196 10. tool_file_dir.py
197 11. tool_judge_results.py
198 12. tool_llm_eval.py
199 13. tool_summarize_text.py
200 14. smolagents_agent_test1.py
201 15. smolagents_test.py
202 
203 These Python programs evaluate the performance of LLMs:
204 1. tool_anti_hallucination.py
205 2. tool_llm_eval.py
206 3. tool_summarize_text.py

  1 python smolagents_agent_test1.py 
  2 ╭────────────────────────────────── New run ───────────────────────────────────╮
  3 │                                                                              │
  4 │ What are the files in the current directory? Describe the current directory  │
  5 │                                                                              │
  6 ╰─ LiteLLMModel - ollama_chat/llama3.2:latest ─────────────────────────────────╯
  7 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  8 ╭──────────────────────────────────────────────────────────────────────────────╮
  9 │ Calling tool: 'sa_list_directory' with arguments: {'list_dots': True}        │
 10 ╰──────────────────────────────────────────────────────────────────────────────╯
 11 ('Contents of current directory: [Makefile, README.md, __pycache__, data, '
 12  'example_chain_read_summary.py, example_chain_web_summary.py, '
 13  'example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, '
 14  'langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, '
 15  'short_programs, smolagents_agent_test1.py, smolagents_test.py, '
 16  'smolagents_tools.py, templates, tool_anti_hallucination.py, '
 17  'tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, '
 18  'tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, '
 19  'tool_web_search.py, venv]')
 20 Observations: Contents of current directory: [Makefile, README.md, __pycache__, 
 21 data, example_chain_read_summary.py, example_chain_web_summary.py, 
 22 example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, 
 23 langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, 
 24 short_programs, smolagents_agent_test1.py, smolagents_test.py, 
 25 smolagents_tools.py, templates, tool_anti_hallucination.py, 
 26 tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, 
 27 tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, 
 28 venv]
 29 [Step 0: Duration 3.06 seconds| Input tokens: 1,332 | Output tokens: 19]
 30 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 31 ╭──────────────────────────────────────────────────────────────────────────────╮
 32 │ Calling tool: 'sa_list_directory' with arguments: {'list_dots': True}        │
 33 ╰──────────────────────────────────────────────────────────────────────────────╯
 34 ('Contents of current directory: [Makefile, README.md, __pycache__, data, '
 35  'example_chain_read_summary.py, example_chain_web_summary.py, '
 36  'example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, '
 37  'langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, '
 38  'short_programs, smolagents_agent_test1.py, smolagents_test.py, '
 39  'smolagents_tools.py, templates, tool_anti_hallucination.py, '
 40  'tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, '
 41  'tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, '
 42  'tool_web_search.py, venv]')
 43 Observations: Contents of current directory: [Makefile, README.md, __pycache__, 
 44 data, example_chain_read_summary.py, example_chain_web_summary.py, 
 45 example_judge.py, graph_kuzu_from_text.py, graph_kuzu_property_example.py, 
 46 langgraph_agent_test.py, ollama_tools_examples.py, requirements.txt, 
 47 short_programs, smolagents_agent_test1.py, smolagents_test.py, 
 48 smolagents_tools.py, templates, tool_anti_hallucination.py, 
 49 tool_file_contents.py, tool_file_dir.py, tool_judge_results.py, 
 50 tool_llm_eval.py, tool_sqlite.py, tool_summarize_text.py, tool_web_search.py, 
 51 venv]
 52 [Step 1: Duration 1.58 seconds| Input tokens: 2,900 | Output tokens: 38]
 53 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 54 ╭──────────────────────────────────────────────────────────────────────────────╮
 55 │ Calling tool: 'sa_summarize_directory' with arguments: {}                    │
 56 ╰──────────────────────────────────────────────────────────────────────────────╯
 57 lst='Contents of current directory: [Makefile, README.md, __pycache__, data, example\
 58 _chain_read_summary.py, example_chain_web_summary.py, example_judge.py, graph_kuzu_f
 59 rom_text.py, graph_kuzu_property_example.py, langgraph_agent_test.py, ollama_tools_e
 60 xamples.py, requirements.txt, short_programs, smolagents_agent_test1.py, smolagents_
 61 test.py, smolagents_tools.py, templates, tool_anti_hallucination.py, tool_file_conte
 62 nts.py, tool_file_dir.py, tool_judge_results.py, tool_llm_eval.py, tool_sqlite.py, t
 63 ool_summarize_text.py, tool_web_search.py, venv]'
 64 response.message.content="Based on the provided list of files in the current directo\
 65 ry, here's a summary and an educated guess about the major purposes of each file:\n\
 66 n1. **Makefile**: A build script for automating compilation and other tasks.\n\n2. *
 67 *README.md**: A document providing information about the project, its purpose, usage
 68 , and installation instructions.\n\n3. **__pycache__** (hidden directory): Cache fil
 69 es generated by Python's compiler to speed up importing modules.\n\n4. **data**: Dir
 70 ectory containing data used for testing or training models.\n\n5. **example_chain_re
 71 ad_summary.py**, **example_chain_web_summary.py**: Example scripts demonstrating how
 72  to summarize text from chain-related input, possibly related to natural language pr
 73 ocessing (NLP) or machine learning (ML).\n\n6. **example_judge.py**: An example scri
 74 pt for evaluating the performance of a model or algorithm.\n\n7. **graph_kuzu_from_t
 75 ext.py**, **graph_kuzu_property_example.py**: Scripts that manipulate graphs generat
 76 ed from text data using the Kuzu graph library, possibly used in NLP or ML applicati
 77 ons.\n\n8. **langgraph_agent_test.py**: A test file for a language graph agent, whic
 78 h is likely an AI model designed to process and understand languages.\n\n9. **ollama
 79 _tools_examples.py**: An example script showcasing how to use Ollama, a tool for gen
 80 erating text data.\n\n10. **requirements.txt**: A list of dependencies required to r
 81 un the project, including libraries and tools.\n\n11. **short_programs**: Directory 
 82 containing short programs or scripts that demonstrate specific tasks or algorithms.\
 83 n\n12. **smolagents_agent_test1.py**, **smolagents_test.py**, **smolagents_tools.py*
 84 *: Test files for a small agents framework, possibly an AI model designed to make de
 85 cisions in complex environments.\n\n13. **templates**: A directory containing templa
 86 tes used for generating text or code in certain contexts.\n\n14. **tool_anti_halluci
 87 nation.py**, **tool_file_contents.py**, **tool_file_dir.py**, **tool_judge_results.p
 88 y**, **tool_llm_eval.py**, **tool_sqlite.py**, **tool_summarize_text.py**, **tool_we
 89 b_search.py**: Various tool scripts that provide functionality for tasks like:\n   -
 90  Anti-hallucination (removing fake data from generated text)\n   - Evaluating file c
 91 ontents\n   - File directory manipulation\n   - Judging results\n   - LLM (Large Lan
 92 guage Model) evaluation\n   - SQLite database interactions\n   - Text summarization\
 93 n   - Web search functionality\n\n15. **venv**: A virtual environment script used to
 94  create and manage a separate Python environment for the project.\n\nThese are educa
 95 ted guesses based on common naming conventions and directory structures in software 
 96 development projects, particularly those related to AI, NLP, and machine learning."
 97 Observations: Summary of directory:Based on the provided list of files in the 
 98 current directory, here's a summary and an educated guess about the major 
 99 purposes of each file:
100 
101 1. **Makefile**: A build script for automating compilation and other tasks.
102 
103 2. **README.md**: A document providing information about the project, its 
104 purpose, usage, and installation instructions.
105 
106 3. **__pycache__** (hidden directory): Cache files generated by Python's 
107 compiler to speed up importing modules.
108 
109 4. **data**: Directory containing data used for testing or training models.
110 
111 5. **example_chain_read_summary.py**, **example_chain_web_summary.py**: Example 
112 scripts demonstrating how to summarize text from chain-related input, possibly 
113 related to natural language processing (NLP) or machine learning (ML).
114 
115 6. **example_judge.py**: An example script for evaluating the performance of a 
116 model or algorithm.
117 
118 7. **graph_kuzu_from_text.py**, **graph_kuzu_property_example.py**: Scripts that
119 manipulate graphs generated from text data using the Kuzu graph library, 
120 possibly used in NLP or ML applications.
121 
122 8. **langgraph_agent_test.py**: A test file for a language graph agent, which is
123 likely an AI model designed to process and understand languages.
124 
125 9. **ollama_tools_examples.py**: An example script showcasing how to use Ollama,
126 a tool for generating text data.
127 
128 10. **requirements.txt**: A list of dependencies required to run the project, 
129 including libraries and tools.
130 
131 11. **short_programs**: Directory containing short programs or scripts that 
132 demonstrate specific tasks or algorithms.
133 
134 12. **smolagents_agent_test1.py**, **smolagents_test.py**, 
135 **smolagents_tools.py**: Test files for a small agents framework, possibly an AI
136 model designed to make decisions in complex environments.
137 
138 13. **templates**: A directory containing templates used for generating text or 
139 code in certain contexts.
140 
141 14. **tool_anti_hallucination.py**, **tool_file_contents.py**, 
142 **tool_file_dir.py**, **tool_judge_results.py**, **tool_llm_eval.py**, 
143 **tool_sqlite.py**, **tool_summarize_text.py**, **tool_web_search.py**: Various 
144 tool scripts that provide functionality for tasks like:
145    - Anti-hallucination (removing fake data from generated text)
146    - Evaluating file contents
147    - File directory manipulation
148    - Judging results
149    - LLM (Large Language Model) evaluation
150    - SQLite database interactions
151    - Text summarization
152    - Web search functionality
153 
154 15. **venv**: A virtual environment script used to create and manage a separate 
155 Python environment for the project.
156 
157 These are educated guesses based on common naming conventions and directory 
158 structures in software development projects, particularly those related to AI, 
159 NLP, and machine learning.
160 [Step 2: Duration 13.79 seconds| Input tokens: 4,706 | Output tokens: 54]
161 
162 ...
163 
164 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 5 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
165 ╭──────────────────────────────────────────────────────────────────────────────╮
166 │ Calling tool: 'sa_summarize_directory' with arguments: {}                    │
167 ╰──────────────────────────────────────────────────────────────────────────────╯
168 lst='Contents of current directory: [Makefile, README.md, __pycache__, data, example\
169 _chain_read_summary.py, example_chain_web_summary.py, example_judge.py, graph_kuzu_f
170 rom_text.py, graph_kuzu_property_example.py, langgraph_agent_test.py, ollama_tools_e
171 xamples.py, requirements.txt, short_programs, smolagents_agent_test1.py, smolagents_
172 test.py, smolagents_tools.py, templates, tool_anti_hallucination.py, tool_file_conte
173 nts.py, tool_file_dir.py, tool_judge_results.py, tool_llm_eval.py, tool_sqlite.py, t
174 ool_summarize_text.py, tool_web_search.py, venv]'
175 response.message.content="Based on the names and locations of the files in the curre\
176 nt directory, here's a summary of their contents and a educated guess about their pu
177 rposes:\n\n1. **Makefile**: A build script for automating tasks such as compiling or
178  running code. It likely contains instructions on how to build and install the proje
179 ct.\n\n2. **README.md**: A Markdown document that serves as an introduction or guide
180  for users of the project. It may include information on how to use the tools, depen
181 dencies required, and contributing to the project.\n\n3. **__pycache__**: An empty d
182 irectory that contains compiled Python files (`.cpyc` and `.pyo`) generated by the P
183 yInstaller build process for a Python application.\n\n4. **data**: A directory conta
184 ining data used for testing or training purposes. It might include CSV, JSON, or oth
185 er formats of datasets.\n\n5. **example_chain_read_summary.py** and **example_chain_
186 web_summary.py**: Example scripts demonstrating how to use tools related to text sum
187 marization, possibly for natural language processing (NLP) tasks.\n\n6. **example_ju
188 dge.py**: An example script that likely demonstrates the usage of a judging tool or 
189 an evaluation framework for the project.\n\n7. **graph_kuzu_from_text.py** and **gra
190 ph_kuzu_property_example.py**: Scripts related to graph-based tools, possibly using 
191 Kuzu, a library for graph algorithms. These scripts might illustrate how to work wit
192 h graphs in Python.\n\n8. **langgraph_agent_test.py**: A script that tests the funct
193 ionality of a language graph agent.\n\n9. **ollama_tools_examples.py**: An example s
194 cript demonstrating the usage of OLLAMA tools ( likely Open-source Language Model-ba
195 sed Agent).\n\n10. **requirements.txt**: A text file listing the dependencies requir
196 ed to run the project, such as Python packages or other software libraries.\n\n11. *
197 *short_programs**: A directory containing short programs for demonstration purposes.
198 \n\n12. **smolagents_agent_test1.py** and **smolagents_test.py**: Scripts related to
199  testing SmoLA (Small Model-based Language Agent), a library that allows the use of 
200 small models in language agents.\n\n13. **smolagents_tools.py**: An example script d
201 emonstrating the usage of SmoLA tools.\n\n14. **templates**: A directory containing 
202 template files used for generating documentation or other text output.\n\n15. **tool
203 _anti_hallucination.py**, **tool_file_contents.py**, **tool_file_dir.py**, **tool_ju
204 dge_results.py**, and **tool_llm_eval.py**: Scripts related to various tools, possib
205 ly used for data analysis, model evaluation, or language understanding tasks.\n\n16.
206  **tool_sqlite.py** and **tool_summarize_text.py**: Scripts that interface with SQLi
207 te databases or are used for text summarization.\n\n17. **tool_web_search.py** and *
208 *tool_web_search.py** ( likely a duplicate): These scripts might be related to web s
209 earch-related tools, possibly interfacing with APIs or web scraping techniques.\n\n1
210 8. **venv**: A directory containing the virtual environment configuration files crea
211 ted by `python -m venv` command, allowing the project to isolate its dependencies fr
212 om the system Python environment.\n\nNote: The exact purposes of some files may vary
213  depending on the specific implementation and requirements of the project. This summ
214 ary is based on common file name conventions and their typical associations with cer
215 tain tasks or tools in the field of NLP and agent-based systems."
216 Observations: Summary of directory:Based on the names and locations of the files
217 in the current directory, here's a summary of their contents and a educated 
218 guess about their purposes:
219 
220 1. **Makefile**: A build script for automating tasks such as compiling or 
221 running code. It likely contains instructions on how to build and install the 
222 project.
223 
224 2. **README.md**: A Markdown document that serves as an introduction or guide 
225 for users of the project. It may include information on how to use the tools, 
226 dependencies required, and contributing to the project.
227 
228 3. **__pycache__**: An empty directory that contains compiled Python files 
229 (`.cpyc` and `.pyo`) generated by the PyInstaller build process for a Python 
230 application.
231 
232 4. **data**: A directory containing data used for testing or training purposes. 
233 It might include CSV, JSON, or other formats of datasets.
234 
235 5. **example_chain_read_summary.py** and **example_chain_web_summary.py**: 
236 Example scripts demonstrating how to use tools related to text summarization, 
237 possibly for natural language processing (NLP) tasks.
238 
239 6. **example_judge.py**: An example script that likely demonstrates the usage of
240 a judging tool or an evaluation framework for the project.
241 
242 7. **graph_kuzu_from_text.py** and **graph_kuzu_property_example.py**: Scripts 
243 related to graph-based tools, possibly using Kuzu, a library for graph 
244 algorithms. These scripts might illustrate how to work with graphs in Python.
245 
246 8. **langgraph_agent_test.py**: A script that tests the functionality of a 
247 language graph agent.
248 
249 9. **ollama_tools_examples.py**: An example script demonstrating the usage of 
250 OLLAMA tools ( likely Open-source Language Model-based Agent).
251 
252 10. **requirements.txt**: A text file listing the dependencies required to run 
253 the project, such as Python packages or other software libraries.
254 
255 11. **short_programs**: A directory containing short programs for demonstration 
256 purposes.
257 
258 12. **smolagents_agent_test1.py** and **smolagents_test.py**: Scripts related to
259 testing SmoLA (Small Model-based Language Agent), a library that allows the use 
260 of small models in language agents.
261 
262 13. **smolagents_tools.py**: An example script demonstrating the usage of SmoLA 
263 tools.
264 
265 14. **templates**: A directory containing template files used for generating 
266 documentation or other text output.
267 
268 15. **tool_anti_hallucination.py**, **tool_file_contents.py**, 
269 **tool_file_dir.py**, **tool_judge_results.py**, and **tool_llm_eval.py**: 
270 Scripts related to various tools, possibly used for data analysis, model 
271 evaluation, or language understanding tasks.
272 
273 16. **tool_sqlite.py** and **tool_summarize_text.py**: Scripts that interface 
274 with SQLite databases or are used for text summarization.
275 
276 17. **tool_web_search.py** and **tool_web_search.py** ( likely a duplicate): 
277 These scripts might be related to web search-related tools, possibly interfacing
278 with APIs or web scraping techniques.
279 
280 18. **venv**: A directory containing the virtual environment configuration files
281 created by `python -m venv` command, allowing the project to isolate its 
282 dependencies from the system Python environment.
283 
284 Note: The exact purposes of some files may vary depending on the specific 
285 implementation and requirements of the project. This summary is based on common 
286 file name conventions and their typical associations with certain tasks or tools
287 in the field of NLP and agent-based systems.
288 [Step 5: Duration 18.89 seconds| Input tokens: 10,726 | Output tokens: 102]
289 Reached max iterations.
290 Final answer: The current directory appears to be a repository for various 
291 projects related to natural language processing (NLP) and agent-based systems. 
292 The directory contains a mix of files and subdirectories that suggest it is used
293 for building, testing, and demonstrating NLP-related tools and applications.
294 
295 Here are some key observations about the files in the current directory:
296 
297 1. **Makefile**: A build script for automating tasks such as compiling or 
298 running code.
299 2. **README.md**: A Markdown document serving as an introduction to the project.
300 3. **__pycache__**: An empty directory containing compiled Python files 
301 generated by the PyInstaller build process.
302 
303 4. **data**: A directory containing data used for testing or training purposes, 
304 including CSV, JSON, and other formats of datasets.
305 
306 5. **Example scripts**: Various example scripts demonstrating how to use tools 
307 related to text summarization, judging, graph-based algorithms, language graph 
308 agents, and more.
309 
310 6. **Requirements file**: A text file listing the dependencies required to run 
311 the project.
312 
313 7. **Short programs**: A directory containing short programs for demonstration 
314 purposes.
315 
316 8. **Testing scripts**: Scripts related to testing various NLP-related tools and
317 libraries, including SmoLA and OLLAMA.
318 
319 9. **Tool scripts**: Various scripts related to data analysis, model evaluation,
320 language understanding, web search, and other tasks.
321 
322 10. **Virtual environment**: A directory containing the virtual environment 
323 configuration files created by `python -m venv` command.
324 
325 The current directory seems to be a collection of example projects, tools, and 
326 scripts for working with NLP-related technologies. It is likely used as a 
327 demonstration or testing environment for exploring new ideas and techniques.
328 [Step 6: Duration 0.00 seconds| Input tokens: 12,332 | Output tokens: 442]
329 The current directory appears to be a repository for various projects related to nat\
330 ural language processing (NLP) and agent-based systems. The directory contains a mix
331  of files and subdirectories that suggest it is used for building, testing, and demo
332 nstrating NLP-related tools and applications.
333 
334 Here are some key observations about the files in the current directory:
335 
336 1. **Makefile**: A build script for automating tasks such as compiling or running co\
337 de.
338 2. **README.md**: A Markdown document serving as an introduction to the project.
339 3. **__pycache__**: An empty directory containing compiled Python files generated by\
340  the PyInstaller build process.
341 
342 4. **data**: A directory containing data used for testing or training purposes, incl\
343 uding CSV, JSON, and other formats of datasets.
344 
345 5. **Example scripts**: Various example scripts demonstrating how to use tools relat\
346 ed to text summarization, judging, graph-based algorithms, language graph agents, an
347 d more.
348 
349 6. **Requirements file**: A text file listing the dependencies required to run the p\
350 roject.
351 
352 7. **Short programs**: A directory containing short programs for demonstration purpo\
353 ses.
354 
355 8. **Testing scripts**: Scripts related to testing various NLP-related tools and lib\
356 raries, including SmoLA and OLLAMA.
357 
358 9. **Tool scripts**: Various scripts related to data analysis, model evaluation, lan\
359 guage understanding, web search, and other tasks.
360 
361 10. **Virtual environment**: A directory containing the virtual environment configur\
362 ation files created by `python -m venv` command.
363 
364 The current directory seems to be a collection of example projects, tools, and scrip\
365 ts for working with NLP-related technologies. It is likely used as a demonstration o
366 r testing environment for exploring new ideas and techniques.