Tool Use / Function Calling

The key to effective tool use with local models that aren’t specifically fine-tuned for tool use or function calling is to guide them with a very structured prompt that contains information for available functions and their arguments. Dear reader, we work around not having built-in supported tool use by creating a “template” that contains information about available tools we write ourselves and include in the template the names and arguments of each tool.

Note that many models and inferencing platforms directly support tool use for some combinations of models and client API libraries.

The advantage of “building it ourselves” is the flexibility of being able to use most models, libraries and inferencing platforms.

Before we look at more complex tools we will first look at a simple common example: a tool to use a stubbed out weather API.

An Initial Example: a Tool That is a Simple Python Function

The first example in this chapter can be found in the file LM_Studio_BOOK/src/tool_use/weather_tool.py and uses these steps:

  • Define the tools: First, I need to define the functions that the AI can “call.” These are standard Python functions. For this example, I’ll create a simple get_weather function.
  • Create the prompt template: This is the most critical step for local models that may not directly support tool use. We need to design a prompt that clearly lists the available tools with their descriptions and parameters, and provides a format for the model to use when it wants to call a tool. The prompt should instruct the model to output a specific, parseable format, like JSON.
  • Set up a client for using the LM Studio service APIs: In this example we use the OpenAI Python library, configured to point to the local LM Studio server. This allows for a familiar and standardized way to interact with the local model.
  • Process user input: The user’s query is inserted into the prompt template.
  • Send a prompt to a model and get a response: In this example a complete prompt is sent to the Gemma model running in LM Studio.
  • Parse the JSON response and execute a local Pyhton function in the client script: Check the model’s response. If it contains the special JSON format for a tool call, then we parse it, execute the corresponding Python function with the provided arguments, and then feed the result back to the model for a final, natural language response. If the initial response from the model doesn’t contain a tool call, then we just use the response.
  1 from openai import OpenAI
  2 import json
  3 import re
  4 
  5 def get_weather(city: str, unit: str = "celsius"):
  6     """
  7     Get the current weather for a given city.
  8     
  9     Args:
 10         city (str): The name of the city.
 11         unit (str): The temperature unit, 'celsius' or 'fahrenheit'.
 12     """
 13     # In a real application, you would call a weather API here.
 14     # For this example, we'll just return some mock data.
 15     if "chicago" in city.lower():
 16         return json.dumps({"city": "Chicago", "temperature": "12", "unit": unit})
 17     elif "tokyo" in city.lower():
 18         return json.dumps({"city": "Tokyo", "temperature": "25", "unit": unit})
 19     else:
 20         return json.dumps({"error": "City not found"})
 21 
 22 
 23 # Point to the local server
 24 client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")
 25 
 26 # A dictionary to map tool names to actual functions
 27 available_tools = {
 28     "get_weather": get_weather,
 29 }
 30 
 31 def run_conversation(user_prompt: str):
 32     # System prompt that defines the rules and tools for the model
 33     system_prompt = """
 34     You are a helpful assistant with access to the following tools.
 35     To use a tool, you must respond with a JSON object with two keys: "tool_name" and "parameters".
 36     
 37     Here are the available tools:
 38     {
 39         "tool_name": "get_weather",
 40         "description": "Get the current weather for a given city.",
 41         "parameters": [
 42             {"name": "city", "type": "string", "description": "The city name."},
 43             {"name": "unit", "type": "string", "description": "The unit for temperature, either 'celsius' or 'fahrenheit'."}
 44         ]
 45     }
 46     
 47     If you decide to use a tool, your response MUST be only the JSON object.
 48     If you don't need a tool, answer the user's question directly.
 49     """
 50     
 51     messages = [
 52         {"role": "system", "content": system_prompt},
 53         {"role": "user", "content": user_prompt}
 54     ]
 55 
 56     print("--- User Question ---")
 57     print(user_prompt)
 58 
 59     completion = client.chat.completions.create(
 60         model="local-model", # This will be ignored by LM Studio
 61         messages=messages,
 62         temperature=0.1, # Lower temperature for more predictable, structured output
 63     )
 64 
 65     response_message = completion.choices[0].message.content
 66 
 67     # More robustly find and extract the JSON from the model's response
 68     json_str = None
 69     tool_call = {}
 70     
 71     # Use regex to find JSON within ```json ... ``` or ``` ... ```
 72     match = re.search(r'```(?:json)?\s*(\{.*?\})\s*```', response_message, re.DOTALL)
 73     if match:
 74         json_str = match.group(1)
 75     else:
 76         # If no markdown block, maybe the whole message is the JSON
 77         if response_message.startswith('{'):
 78             json_str = response_message
 79 
 80     if json_str:
 81         try:
 82             tool_call = json.loads(json_str)
 83         except json.JSONDecodeError as e:
 84             print(e)
 85              
 86     # Check if the model wants to call a tool
 87     try:
 88         tool_name = tool_call.get("tool_name")
 89         
 90         if tool_name in available_tools:
 91             print("\n--- Tool Call Detected ---")
 92             print(f"Tool: {tool_name}")
 93             print(f"Parameters: {tool_call.get('parameters')}")
 94             
 95             # Execute the function
 96             function_to_call = available_tools[tool_name]
 97             tool_params = tool_call.get("parameters", {})
 98             function_response = function_to_call(**tool_params)
 99             
100             print("\n--- Tool Response ---")
101             print(function_response)
102             
103             # (Optional) Send the result back to the model for a final summary
104             messages.append({"role": "assistant", "content": response_message})
105             messages.append({"role": "tool", "content": function_response})
106             
107             print("\n--- Final Response from Model ---")
108             final_completion = client.chat.completions.create(
109                 model="local-model",
110                 messages=messages,
111                 temperature=0.7,
112             )
113             print(final_completion.choices[0].message.content)
114             
115         else:
116             # The JSON doesn't match our tool schema
117             print("\n--- Assistant Response (No Tool) ---")
118             print(response_message)
119 
120     except json.JSONDecodeError:
121         # The response was not JSON, so it's a direct answer
122         print("\n--- Assistant Response (No Tool) ---")
123         print(response_message)
124 
125 
126 # --- Run Examples ---
127 run_conversation("What's the weather like in Tokyo in celsius?")
128 print("\n" + "="*50 + "\n")
129 run_conversation("What is the capital of France?")

The output using LM Studio with the model google/gemma-3n-e4b looks like:

 1 $ uv run weather_tool.py
 2 --- User Question ---
 3 What's the weather like in Tokyo in celsius?
 4 
 5 --- Tool Call Detected ---
 6 Tool: get_weather
 7 Parameters: {'city': 'Tokyo', 'unit': 'celsius'}
 8 
 9 --- Tool Response ---
10 {"city": "Tokyo", "temperature": "25", "unit": "celsius"}
11 
12 --- Final Response from Model ---
13 The weather in Tokyo is 25 degrees Celsius.
14 
15 ==================================================
16 
17 --- User Question ---
18 What is the capital of France?
19 
20 --- Assistant Response (No Tool) ---
21 Paris is the capital of France.

Here we tested two prompts: the first uses a tool and the second prompt does not use a tool. We started with a simple example so you understand the low-level process of supporting tool use/function calling. In the next section we will generalize this example into two parts: a separate library and examples that uses this separate library.

Creating a General Purpose Tools/Function Calling Library

This Python code in the file tool_use/function_calling_library.py provides a lightweight and flexible framework for integrating external functions as “tools” with a large language model (LLM). (Note: we will use tools hosted in the LM Studio application in the next chapter.) Our library defines two primary classes: ToolManager, which handles the registration and schema generation for available tools, and ConversationHandler, which orchestrates the multi-step interaction between the user, the LLM, and the tools. This approach allows the LLM to decide when to call a function, execute it within the Python environment, and then use the result to formulate a more informed and human readable response.

  1 import json
  2 import re
  3 import inspect
  4 from openai import OpenAI
  5 
  6 class ToolManager:
  7     """
  8     Manages the registration and formatting of tools for the LLM.
  9     """
 10     def __init__(self):
 11         """Initializes the ToolManager with empty dictionaries for tools."""
 12         self.tools_schema = {}
 13         self.available_tools = {}
 14 
 15     def register_tool(self, func):
 16         """
 17         Registers a function as a tool, extracting its schema from the
 18         docstring and signature.
 19         
 20         Args:
 21             func (function): The function to be registered as a tool.
 22         """
 23         tool_name = func.__name__
 24         self.available_tools[tool_name] = func
 25 
 26         # Extract description from docstring
 27         description = "No description found."
 28         docstring = inspect.getdoc(func)
 29         if docstring:
 30             description = docstring.strip().split('\n\n')[0]
 31 
 32         # Extract parameters from function signature
 33         sig = inspect.signature(func)
 34         parameters = []
 35         for name, param in sig.parameters.items():
 36             param_type = "string" # Default type
 37             if param.annotation is not inspect.Parameter.empty:
 38                 # A simple way to map Python types to JSON schema types
 39                 if param.annotation == int:
 40                     param_type = "integer"
 41                 elif param.annotation == float:
 42                     param_type = "number"
 43                 elif param.annotation == bool:
 44                     param_type = "boolean"
 45             
 46             # Simple docstring parsing for parameter descriptions (assumes "Args:" section)
 47             param_description = ""
 48             if docstring:
 49                 arg_section = re.search(r'Args:(.*)', docstring, re.DOTALL)
 50                 if arg_section:
 51                     param_line = re.search(rf'^\s*{name}\s*\(.*?\):\s*(.*)',
 52                                            arg_section.group(1), re.MULTILINE)
 53                     if param_line:
 54                         param_description = param_line.group(1).strip()
 55 
 56             parameters.append({
 57                 "name": name,
 58                 "type": param_type,
 59                 "description": param_description
 60             })
 61 
 62         self.tools_schema[tool_name] = {
 63             "tool_name": tool_name,
 64             "description": description,
 65             "parameters": parameters
 66         }
 67 
 68     def get_tools_for_prompt(self):
 69         """
 70         Formats the registered tools' schemas into a JSON string for the system prompt.
 71 
 72         Returns:
 73             str: A JSON string representing the list of available tools.
 74         """
 75         if not self.tools_schema:
 76             return "No tools available."
 77         return json.dumps(list(self.tools_schema.values()), indent=4)
 78 
 79 class ConversationHandler:
 80     """
 81     Handles the conversation flow, including making API calls and executing tools.
 82     """
 83     def __init__(self, client: OpenAI, tool_manager: ToolManager,
 84                  model: str = "local-model", temperature: float = 0.1):
 85         """
 86         Initializes the ConversationHandler.
 87 
 88         Args:
 89             client (OpenAI): The OpenAI client instance.
 90             tool_manager (ToolManager): The ToolManager instance with registered tools.
 91             model (str): The model name to use (ignored by LM Studio).
 92             temperature (float): The sampling temperature for the model.
 93         """
 94         self.client = client
 95         self.tool_manager = tool_manager
 96         self.model = model
 97         self.temperature = temperature
 98 
 99     def _create_system_prompt(self):
100         """Creates the system prompt with tool definitions."""
101         return f"""
102 You are a helpful assistant with access to the following tools.
103 To use a tool, you must respond with a JSON object with two keys: "tool_name" and "parameters".
104 
105 Here are the available tools:
106 {self.tool_manager.get_tools_for_prompt()}
107 
108 If you decide to use a tool, your response MUST be only the JSON object.
109 If you don't need a tool, answer the user's question directly.
110 """
111 
112     def run(self, user_prompt: str, verbose: bool = True):
113         """
114         Runs the full conversation loop for a single user prompt.
115 
116         Args:
117             user_prompt (str): The user's question or command.
118             verbose (bool): If True, prints detailed steps of the conversation.
119         """
120         system_prompt = self._create_system_prompt()
121         messages = [
122             {"role": "system", "content": system_prompt},
123             {"role": "user", "content": user_prompt}
124         ]
125 
126         if verbose:
127             print("--- User Question ---")
128             print(user_prompt)
129 
130         # --- First API Call: Check for tool use ---
131         completion = self.client.chat.completions.create(
132             model=self.model,
133             messages=messages,
134             temperature=self.temperature,
135         )
136         response_message = completion.choices[0].message.content
137 
138         # --- Parse for Tool Call ---
139         tool_call = self._parse_tool_call(response_message)
140 
141         if tool_call and tool_call.get("tool_name") in self.tool_manager.available_tools:
142             tool_name = tool_call["tool_name"]
143             tool_params = tool_call.get("parameters", {})
144             
145             if verbose:
146                 print("\n--- Tool Call Detected ---")
147                 print(f"Tool: {tool_name}")
148                 print(f"Parameters: {tool_params}")
149 
150             # --- Execute the Tool ---
151             function_to_call = self.tool_manager.available_tools[tool_name]
152             try:
153                 function_response = function_to_call(**tool_params)
154             except Exception as e:
155                 function_response = f"Error executing tool: {e}"
156 
157             if verbose:
158                 print("\n--- Tool Response ---")
159                 print(function_response)
160 
161             # --- Second API Call: Summarize the result ---
162             messages.append({"role": "assistant", "content": json.dumps(tool_call, indent=4)})
163             messages.append({"role": "tool", "content": str(function_response)})
164             
165             messages.append({
166                 "role": "user", 
167                 "content": "Based on the result from the tool, please formulate a final answer to the original user question."
168             })
169 
170             if verbose:
171                 print("\n--- Final Response from Model ---")
172             
173             final_completion = self.client.chat.completions.create(
174                 model=self.model,
175                 messages=messages,
176                 temperature=0.7, # Higher temp for more natural language
177             )
178             final_response = final_completion.choices[0].message.content
179             print(final_response)
180         else:
181             # --- No Tool Call Detected ---
182             if verbose:
183                 print("\n--- Assistant Response (No Tool) ---")
184             print(response_message)
185 
186     def _parse_tool_call(self, response_message: str) -> dict | None:
187         """
188         Parses the model's response to find and decode a JSON tool call.
189 
190         Args:
191             response_message (str): The raw response content from the model.
192 
193         Returns:
194             dict or None: A dictionary representing the tool call, or None if not found.
195         """
196         json_str = None
197         # Use regex to find JSON within ```json ... ``` or ``` ... ```
198         match = re.search(r'```(?:json)?\s*(\{.*?\})\s*```', response_message, re.DOTALL)
199         if match:
200             json_str = match.group(1)
201         # If no markdown block, check if the whole message is a JSON object
202         elif response_message.strip().startswith('{'):
203             json_str = response_message
204 
205         if json_str:
206             try:
207                 # Clean up the JSON string before parsing
208                 cleaned_json_str = json_str.strip()
209                 return json.loads(cleaned_json_str)
210             except json.JSONDecodeError as e:
211                 print(f"JSON Decode Error: {e} in string '{cleaned_json_str}'")
212                 return None
213         return None

The first class, ToolManager, serves as a registry for the functions you want to expose to the LLM. Its core method, register_tool, uses Python’s inspect module to dynamically analyze a function’s signature and docstring. It extracts the function’s name, its parameters (including their type hints), and their descriptions from the “Args” section of the docstring. This information is then compiled into a JSON schema that describes the tool in a machine-readable format. This automated process is powerful because it allows a developer to make a standard Python function available to the LLM simply by adding it to the manager, without manually writing complex JSON schemas.

The second class, ConversationHandler, is the engine that drives the interaction. When its run method is called, it first constructs a detailed system prompt. This special prompt instructs the LLM on how to behave and includes the JSON schemas for all registered tools, informing the model of its capabilities. The user’s question is then sent to the LLM. The model’s first task is to decide whether to answer directly or to use one of the provided tools. If it determines a tool is necessary, it is instructed to respond only with a JSON object specifying the tool_name and the parameters needed to run it.

The process concludes with a crucial two-step execution logic. If the ConversationHandler receives a valid JSON tool call from the LLM, it executes the corresponding Python function with the provided parameters. The return value from that function is then packaged into a new message with the role “tool” and sent back to the LLM in a second API call. This second call prompts the model to synthesize the tool’s output into a final, natural-language answer for the user. If the model’s initial response was not a tool call, the system assumes no tool was needed and simply presents that response directly to the user. This conditional, multi-step approach enables the LLM to leverage external code to answer questions it otherwise couldn’t.

First example Using Function Calling Library: Generate Python and Execute to Answer User Questions

This Python script in the file tool_use/test1.py demonstrates a practical implementation of the function_calling_library by creating a specific tool designed to solve mathematical problems. It defines a function, solve_math_problem, that can execute arbitrary Python code in an insecure (you must trust the Python tool functions that the model writes for you to help solve a user prompt or query), isolated process. The main part of the script then initializes the ToolManager and ConversationHandler from the library developed in the previous section, registers the new math tool, and runs two example conversations: one that requires complex calculation, thereby triggering the tool, and another that is a general knowledge question, which the LLM answers directly.

 1 import os
 2 import subprocess
 3 import json
 4 from openai import OpenAI
 5 from function_calling_library import ToolManager, ConversationHandler
 6 
 7 # --- Define the Custom Tool ---
 8 
 9 def solve_math_problem(python_code: str):
10     """
11     Executes a given string of Python code to solve a math problem and returns the output.
12     The code should be a complete, runnable script that prints the final result to standard output.
13 
14     Args:
15         python_code (str): A string containing the Python code to execute.
16     """
17     temp_filename = "temp.py"
18     
19     # Ensure any previous file is removed
20     if os.path.exists(temp_filename):
21         os.remove(temp_filename)
22 
23     try:
24         # Write the code to a temporary file
25         with open(temp_filename, "w") as f:
26             f.write(python_code)
27         
28         # Execute the python script as a separate process
29         result = subprocess.run(
30             ['python', temp_filename], 
31             capture_output=True, 
32             text=True, 
33             check=True, # This will raise CalledProcessError if the script fails
34             timeout=10 # Add a timeout for safety
35         )
36         
37         # The output from the script's print() statements
38         return result.stdout.strip()
39 
40     except subprocess.CalledProcessError as e:
41         # If the script has a runtime error, return the error message
42         error_message = f"Error executing the Python code:\nSTDOUT:\n{e.stdout}\nSTDERR:\n{e.stderr}"
43         return error_message
44     except Exception as e:
45         return f"An unexpected error occurred: {e}"
46     finally:
47         # Clean up the temporary file
48         if os.path.exists(temp_filename):
49             os.remove(temp_filename)
50 
51 
52 # --- Main Execution Logic ---
53 
54 if __name__ == "__main__":
55     # Point to the local LM Studio server
56     client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")
57     
58     # 1. Initialize the ToolManager
59     tool_manager = ToolManager()
60     
61     # 2. Register the custom tool
62     tool_manager.register_tool(solve_math_problem)
63     
64     # 3. Initialize the ConversationHandler with the client and tools
65     handler = ConversationHandler(client, tool_manager)
66     
67     # 4. Define a user prompt that requires the tool
68     user_prompt = "Can you please calculate the area of a circle with a radius of 7.5 and also find the 20th number in the Fibonacci sequence? Please provide the Python code to do this."
69     
70     # 5. Run the conversation
71     handler.run(user_prompt)
72 
73     print("\n" + "="*50 + "\n")
74 
75     # Example of a question that should NOT use the tool
76     non_tool_prompt = "What is the most popular programming language?"
77     handler.run(non_tool_prompt)

The core of this script is the solve_math_problem function, which serves as the custom tool. It’s designed to somewhat safely execute a string of Python code passed to it by the LLM. To avoid security risks associated with eval() or exec(), it writes the code to a temporary file (temp.py). It then uses Python’s subprocess module to run this file as an entirely separate process. This sandboxes the execution, and the capture_output=True argument ensures that any output printed by the script (e.g., the result of a calculation) is captured. The function includes robust error handling, returning any standard error from the script if it fails, and a finally block to guarantee the temporary file is deleted, maintaining a clean state.

Please note that code generated my the LLM is not fully sandboxed and this approach is tailored to personal development environments. For production consider running the generated code in a container that limits network and file access activity, as appropriate.

The main execution block, guarded by if __name__ == "__main__", orchestrates the entire demonstration. It begins by configuring the OpenAI client to connect to a local server, such as LM Studio. It then instantiates the ToolManager and registers the solve_math_problem function as an available tool. With the tool ready, it creates a ConversationHandler to manage the flow. The script then showcases the system’s decision-making ability by running two different prompts. The first asks for two distinct mathematical calculations, a task that perfectly matches the solve_math_problem tool’s purpose. The second prompt is a general knowledge question that requires no calculation, demonstrating the LLM’s ability to differentiate between tasks and answer directly when a tool is not needed.

Sample output:

 1 $ uv run test1.py              
 2 --- User Question ---
 3 Can you please calculate the area of a circle with a radius of 7.5 and also find the 20th number in the Fibonacci sequence? Please provide the Python code to do this.
 4 
 5 --- Tool Call Detected ---
 6 Tool: solve_math_problem
 7 Parameters: {'python_code': 'import math\nradius = 7.5\narea = math.pi * radius**2\nprint(area)\n\ndef fibonacci(n):\n  if n <= 0:\n    return 0\n  elif n == 1:\n    return 1\n  else:\n    a, b = 0, 1\n    for _ in range(2, n + 1):\n      a, b = b, a + b\n    return b\n\nprint(fibonacci(20))'}
 8 
 9 --- Tool Response ---
10 176.71458676442586
11 6765
12 
13 --- Final Response from Model ---
14 The area of a circle with a radius of 7.5 is approximately 176.7146, and the 20th number in the Fibonacci sequence is 6765.
15 
16 
17 ==================================================
18 
19 --- User Question ---
20 What is the most popular programming language?
21 
22 --- Assistant Response (No Tool) ---
23 Python is generally considered the most popular programming language.

Having a model generate Python code to solve problems is a powerful technique so please, dear reader, take some time experimenting with this last example and adapting it to your own use cases.

Second example Using Function Calling Library: Stub of Weather API

This is our original example, modified to use the library developed earlier in this chapter.

This script in the file tool_use/test2.py provides another clear example of how the function_calling_library can be used to extend an LLM’s capabilities, this time by simulating an external data fetch from a weather API. It defines a simple get_weather function that returns mock data for specific cities. The main execution logic then sets up the ConversationHandler, registers this new tool, and processes two distinct user prompts to demonstrate the LLM’s ability to intelligently decide when to call the function and when to rely on its own knowledge base.

 1 import json
 2 from openai import OpenAI
 3 from function_calling_library import ToolManager, ConversationHandler
 4 
 5 # --- Define the Custom Tool ---
 6 
 7 def get_weather(city: str, unit: str = "celsius"):
 8     """
 9     Get the current weather for a given city.
10     
11     Args:
12         city (str): The name of the city.
13         unit (str): The temperature unit, 'celsius' or 'fahrenheit'.
14     """
15     # In a real application, you would call a weather API here.
16     # For this example, we'll just return some mock data.
17     if "chicago" in city.lower():
18         return json.dumps({"city": "Chicago", "temperature": "12", "unit": unit})
19     elif "tokyo" in city.lower():
20         return json.dumps({"city": "Tokyo", "temperature": "25", "unit": unit})
21     else:
22         return json.dumps({"error": "City not found"})
23 
24 # --- Main Execution Logic ---
25 
26 if __name__ == "__main__":
27     # Point to the local LM Studio server
28     client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")
29     
30     # 1. Initialize the ToolManager
31     tool_manager = ToolManager()
32     
33     # 2. Register the custom tool
34     tool_manager.register_tool(get_weather)
35     
36     # 3. Initialize the ConversationHandler with the client and tools
37     handler = ConversationHandler(client, tool_manager)
38     
39     # 4. Define a user prompt that requires the tool
40     user_prompt = "What's the weather like in Tokyo in celsius?"
41     
42     # 5. Run the conversation
43     handler.run(user_prompt)
44 
45     print("\n" + "="*50 + "\n")
46 
47     # Example of a question not using tool calling:
48     another_prompt = "What do Chicago and Tokyo have in common? Provide a fun answer."
49     handler.run(another_prompt)

The custom tool in this example is the get_weather function. It is defined with clear parameters, city and temperature unit, and includes type hints and a docstring that the ToolManager will use to automatically generate its schema. Instead of making a live call to a weather service, this function contains simple conditional logic to return hardcoded JSON strings for “Chicago” and “Tokyo,” or an error if another city is requested. This mock implementation is a common and effective development practice, as it allows you to build and test the entire function-calling logic without depending on external network requests or API keys. The function’s return value is a JSON string, which is a standard data interchange format easily understood by both the Python environment and the LLM.

The main execution block follows the same clear, step-by-step pattern as the previous example. It configures the client, initializes the ToolManager, and registers the get_weather function. After setting up the ConversationHandler, it runs two test cases that highlight the system’s contextual awareness. The first prompt, “What’s the weather like in Tokyo in celsius?”, directly maps to the functionality of the get_weather tool, and the LLM correctly identifies this and generates the appropriate JSON tool call. The second prompt, which asks for commonalities between the two cities, is a conceptual question outside the tool’s scope. In this case, the LLM correctly bypasses the tool-calling mechanism and provides a direct, creative answer from its own training data, demonstrating the robustness of the overall approach.

Here is sample output from the test2.py script:

 1 $ uv run test2.py
 2 --- User Question ---
 3 What's the weather like in Tokyo in celsius?
 4 
 5 --- Tool Call Detected ---
 6 Tool: get_weather
 7 Parameters: {'city': 'Tokyo', 'unit': 'celsius'}
 8 
 9 --- Tool Response ---
10 {"city": "Tokyo", "temperature": "25", "unit": "celsius"}
11 
12 --- Final Response from Model ---
13 The weather in Tokyo is 25 degrees Celsius.
14 
15 
16 ==================================================
17 
18 --- User Question ---
19 What do Chicago and Tokyo have in common? Provide a fun answer.
20 
21 --- Assistant Response (No Tool) ---
22 Chicago and Tokyo both have amazing food scenes! Chicago is famous for deep-dish pizza, while Tokyo is known for its incredible sushi and ramen. Both cities are culinary adventures! 🍕🍜

Dear reader, you probably use many APIs in developing applications. Choose one or two of these APIs that you are familiar with and modify this last example to call real APIs, making sure that you use type metadata and a descriptive comment in each Python tool function you write. Then you can experiment with “chatting” with live application data from APIs that you use in your own work or research.