Using Local LLMs With Ollama
Running local models with Ollama offers several practical advantages for Common Lisp developers, especially those of us building exploratory or long-lived AI systems:
- Local inference eliminates network latency and external API dependencies, which simplifies debugging, improves reproducibility, and enables fully offline workflows—important when iterating on symbolic/LLM hybrids or REPL-driven experiments.
- Data never leaves the machine, providing strong privacy guarantees and avoiding compliance issues that can arise when sending prompts or intermediate representations to third-party services.
- Cost and rate-limit concerns disappear: once a model is downloaded, usage is bounded only by local compute, making it ideal for background agents, continuous evaluation, or batch reasoning tasks initiated from Lisp.
- Ollama’s simple HTTP interface fits naturally with Common Lisp’s strengths—process control, incremental development, and meta-programming, allowing developers to treat local language models as just another deterministic(ish) subsystem under their control.
The ollama package developed here provides generative AI code and tool use/function calling generative AI code in the directory loving-common-lisp/src/ollama.
Note: I added an example for using built in web search tooling with Ollama Cloud on March 15, 2026.
Design Notes (Optional Material)
Here we describe the design and architecture of the Ollama Common Lisp library, which provides an interface to the Ollama API for running local LLMs.
1. Common Utilities
The shared utilities are defined in ollama-helper.lisp and provide foundational functionality used by both basic completions and tool-calling APIs.
Configuration
- model-host — The Ollama API endpoint URL, defaulting to
http://localhost:11434/api/chat
JSON Handling
- lisp-to-json-string — Converts Lisp data structures (alists) to JSON strings using
cl-json - substitute-subseq — String substitution utility used to work around
cl-json’s encoding ofnilasnull(the Ollama API requiresfalsefor the stream parameter)
HTTP Communication
-
ollama-helper — Core request handler that:
- Executes curl commands via
uiop:run-program - Parses JSON responses
- Extracts message content and tool calls from the response structure
- Returns multiple values:
(content function-calls)
Package Definition
The ollama package (defined in package.lisp) exports:
completions,completions-with-tools— Main API functionssummarize,answer-question— Convenience wrappers*model-name*,*tool-model-name*,*model-host*— Configuration variables
2. Generative AI
Basic generative AI functionality is provided in ollama.lisp for simple text completions without tool calling.
Configuration
- model-name — Model identifier, defaults to
"mistral:v0.3"
Core Functions
-
completions — Sends a user prompt to the LLM and returns the text response
- Constructs a message with role “user” and the provided content
- Builds the request payload with model, stream (false), and messages
- Uses
ollama-helperto execute the request and extract content
Convenience Wrappers
- summarize — Prepends “Summarize: “ to input text and calls
completions - answer-question — Formats input as a Q&A prompt and calls
completions
Request Flow
1 User Text → Message Construction → JSON Encoding → curl Command →
2 Ollama API → JSON Response → Content Extraction → Return String
3. Generative AI with Tools
Tool-calling (function calling) support is implemented in ollama-tools.lisp, enabling the LLM to invoke registered functions.
Configuration
- tool-model-name — Model for tool calling, defaults to
"mistral:v0.3" - available-functions — Hash table storing registered tool functions
Data Structures
-
ollama-function — Struct containing:
name— Function identifier stringdescription— Human-readable description for the LLMparameters— JSON Schema defining expected argumentshandler— Common Lisp function to invoke when called
Function Registration
-
register-tool-function — Registers a tool with the system
- Parameters:
name,description,parameters(JSON Schema),handler(Lisp function) - Stores an
ollama-functionstruct in*available-functions*
Tool Execution
-
handle-tool-function-call — Processes an LLM tool call
- Extracts function name and arguments from the response
- Falls back to
infer-function-name-from-argsif model returns empty name - Looks up the registered handler and invokes it with the arguments
-
infer-function-name-from-args — Workaround for models that return empty function names
- Inspects argument keys to determine which function was intended
Main API
-
completions-with-tools — Enhanced completion with tool support
- Accepts prompt text and optional list of function names to enable
- Builds tool definitions from registered functions
- Sends request to Ollama with tools specification
- Automatically invokes handlers when LLM returns tool calls
Built-in Tools
Two sample tools are pre-registered:
-
get_weather — Returns mock weather data for a location
- Parameters:
location(string) — The city name - Returns: Formatted weather string
-
calculate — Evaluates mathematical expressions
- Parameters:
expression(string) — Math expression like “2 + 2” - Uses Common Lisp’s
evalto compute results
Tool Call Flow
1 User Prompt + Tool Names → Build Tool Definitions → JSON Request →
2 Ollama API → Response with Tool Calls → Parse Function Call →
3 Lookup Handler → Invoke with Arguments → Return Result
Example Usage
1 (ollama::completions-with-tools
2 "What's the weather like in New York?"
3 '("get_weather" "calculate"))
4 ;; => "Weather in New York: Sunny, 72°F"
System Definition
The ASDF system (ollama.asd) loads components in dependency order:
package— Package definitionollama-helper— Shared utilitiesollama-tools— Tool-calling supportollama— Basic completions
Dependencies: uiop, cl-json
Implementation of Common Helper Code
The defpackage form for the #:ollama library establishes an isolated namespace for interacting with local Large Language Models. By inheriting functionality from #:cl, #:uiop, and #:cl-json, the package handles core logic, system-level file operations, and the JSON-heavy communication required by the Ollama REST API. The exported symbols define a public interface, ranging from high-level text processing functions like summarize and answer-question.
Listing of package.lisp:
1 ;;;; package.lisp
2
3 (defpackage #:ollama
4 (:use #:cl #:uiop #:cl-json)
5 (:export #:completions #:completions-with-tools
6 #:summarize #:answer-question
7 *model-name* *tool-model-name* *model-host*))
Listing of ollama.asd that defines a defsystem for this package:
1 ;;;; ollama.asd
2
3 (asdf:defsystem #:ollama
4 :description "Library for using the ollama APIs"
5 :author "Mark Watson"
6 :license "Apache 2"
7 :depends-on (#:uiop #:cl-json)
8 :components ((:file "package")
9 (:file "ollama-helper")
10 (:file "ollama-tools")
11 (:file "ollama")))
The following implementation establishes a bridge between Common Lisp and the Ollama local API, providing the infrastructure necessary for handling structured LLM interactions. By defining a dedicated ollama package and setting a default local Ollama server host variable, the code creates a controlled environment for external communication. The utility functions included here address two primary technical hurdles: the conversion of Lisp data structures into JSON-compliant strings for API consumption and a manual string substitution routine for fine-tuning command payloads. At the heart of this listing is a robust helper function that orchestrates a system-level curl call, capturing the resulting output and parsing the returned JSON. This process involves a traversal of the response object to isolate the model’s textual content and any prospective tool calls, ensuring that the final output is returned in a format that Lisp can easily manipulate for downstream logic.
Listing of ollama-helper.lisp:
1 (in-package #:ollama)
2
3 (defvar *model-host* "http://localhost:11434/api/chat")
4
5 (defun lisp-to-json-string (data)
6 (with-output-to-string (s)
7 (json:encode-json data s)))
8
9 (defun substitute-subseq (string old new &key (test #'eql))
10 (let ((pos (search old string :test test)))
11 (if pos
12 (concatenate 'string
13 (subseq string 0 pos)
14 new
15 (subseq string (+ pos (length old))))
16 string)))
17
18 (defun ollama-helper (curl-command)
19 (princ curl-command)
20 (terpri)
21 (handler-case
22 (let ((response
23 (uiop:run-program
24 curl-command
25 :output :string
26 :error-output :string)))
27 (princ "Raw response: ")
28 (princ response)
29 (terpri)
30 (with-input-from-string
31 (s response)
32 (let* ((json-as-list (json:decode-json s))
33 (message (cdr (assoc :message json-as-list)))
34 (content (cdr (assoc :content message)))
35 ;; Extract function details from each tool_call
36 (function-calls (mapcar (lambda (tc)
37 (cdr (assoc :function tc)))
38 tool-calls)))
39 (values content function-calls))))
40 (error (e)
41 (format t "Error executing curl command: ~a~%" e)
42 nil)))
The code begins by setting up the environment with a global variable for the Ollama endpoint and helper functions for data transformation. The function lisp-to-json-string leverages the cl-json library to serialize data, while substitute-subseq provides a specialized way to replace substrings within the command strings. These utilities ensure that the data sent to the model is formatted correctly and that the commands remain flexible.
The core logic resides in ollama-helper, which uses uiop:run-program to execute a shell command and capture its output. The function is designed with error handling to manage potential connectivity or execution failures gracefully. Once a response is received, it decodes the JSON and performs an association list lookup to extract both the natural language message and any structured function calls, returning them as multiple values for the caller to process.
Implementation of Generative AI Functionality
In this section, we examine a practical implementation of a Common Lisp client designed to interface with the Ollama local LLM inference service. The code defines a workflow for sending synchronous requests to a Large Language Model (LLM) by wrapping the system’s curl utility to communicate with the Ollama API. By utilizing the mistral:v0.3 model as a default, the program demonstrates how to structure Lisp data, specifically association lists, into the JSON format required by the endpoint. It includes a specific handling mechanism for boolean conversion, ensuring that Lisp’s nil is correctly interpreted as a JSON false to disable streaming. Beyond the core transport logic, the listing provides high-level abstractions for common natural language processing tasks, such as summarization and question answering, illustrating how simple string concatenation can be used to format prompts that guide the model toward specific generative behaviors.
Listing of ollama.lisp:
1 (in-package #:ollama)
2
3 ;;; Basic Ollama completions without tool calling support
4 ;;; For tool calling, see ollama-tools.lisp
5
6 (defvar *model-name* "mistral:v0.3")
7
8 (defun completions (starter-text)
9 "Simple completion without function/tool calling support."
10 (let* ((message (list (cons :|role| "user")
11 (cons :|content| starter-text)))
12 (data (list (cons :|model| *model-name*)
13 (cons :|stream| nil)
14 (cons :|messages| (list message))))
15 (json-data (lisp-to-json-string data))
16 ;; Hack: cl-json encodes nil as null, but we need false for stream
17 (fixed-json-data (substitute-subseq json-data ":null" ":false" :test #'string=))
18 (curl-command
19 (format nil "curl ~a -d ~s"
20 ollama::*model-host*
21 fixed-json-data)))
22 (multiple-value-bind (content function-call)
23 (ollama-helper curl-command)
24 (declare (ignore function-call))
25 (or content "No response content"))))
26
27 ;;(ollama:completions "Complete the following text: The President went to")
28
29 ;; Helper functions for summarization and question answering
30 (defun summarize (some-text)
31 (completions (concatenate 'string "Summarize: " some-text)))
32
33 (defun answer-question (some-text)
34 (completions (concatenate 'string "
35 Q: " some-text "
36 A:")))
The core of this implementation lies in the completions function, which manages the transformation of Lisp structures into a command-line request. A notable detail is the manual string substitution used on the JSON payload; since many Common Lisp JSON libraries represent nil as null, the code explicitly replaces these occurrences with false to satisfy the Ollama API’s requirement for the stream parameter. This ensures the function waits for a complete response rather than processing a continuous stream of tokens, simplifying the return value for the caller.
The program also showcases the extensibility of the base completion logic through the summarize and answer-question helper functions. These functions act as specialized wrappers that prepend task-specific instructions to the user input, effectively demonstrating “prompt engineering” within a programmatic context. By delegating the heavy lifting to the ollama-helper and the external curl command, the code remains focused on message preparation and providing a clean, functional interface for Lisp-based AI applications.
Sample output:
1
Implementation of Tool Use/Function Calling Generative AI Functionality
The following listing an experimental implementation of tool-calling (also known as function-calling) within a Common Lisp environment using the Ollama API. By defining a custom ollama-function structure and a global registry via a hash table, the code allows developers to map Large Language Model (LLM) tool requests directly to native Lisp handlers. The primary entry point, completions-with-tools, handles the complex task of serializing Lisp data structures into the specific JSON format required by Ollama’s /api/chat endpoint, including a necessary workaround for JSON boolean representation. Furthermore, the implementation includes a defensive “inference” mechanism to recover function names from argument keys if the model returns an incomplete response, ensuring that calls to registered tools like get_weather or calculate are dispatched correctly even when the model’s output is slightly malformed.
Listing of ollama-tools.lisp:
1 (in-package #:ollama)
2
3 ;;; Ollama completions with tool/function calling support
4 ;;; Uses shared utilities from ollama-helper.lisp
5
6 (defvar *tool-model-name* "qwen3:1.7b")
7
8 (defvar *available-functions* (make-hash-table :test 'equal))
9
10 (defstruct ollama-function
11 name
12 description
13 parameters
14 handler) ;; Common Lisp function to handle the call
15
16 (defun register-tool-function (name description parameters handler)
17 "Register a function that can be called by the LLM via tool calling.
18 HANDLER is a Common Lisp function that takes a plist of arguments."
19 (setf (gethash name *available-functions*)
20 (make-ollama-function
21 :name name
22 :description description
23 :parameters parameters
24 :handler handler)))
25
26 (defun infer-function-name-from-args (args)
27 "Infer the function name based on argument keys
28 (workaround for models that return empty name)."
29 (let ((arg-keys (mapcar #'car args)))
30 (cond
31 ((member :location arg-keys) "get_weather")
32 ((member :expression arg-keys) "calculate")
33 (t nil))))
34
35 (defun handle-tool-function-call (function-call)
36 "Handle a function call returned from the LLM
37 by invoking the registered handler."
38 (format t "~%DEBUG handle-tool-function-call: ~a~%" function-call)
39 (let* ((raw-name (cdr (assoc :name function-call)))
40 (args (cdr (assoc :arguments function-call)))
41 ;; If name is empty, try to infer from arguments
42 (name (if (or (null raw-name) (string= raw-name ""))
43 (infer-function-name-from-args args)
44 raw-name))
45 (func (gethash name *available-functions*)))
46 (format t "DEBUG raw-name=~a inferred-name=~a args=~a func=~a~%"
47 raw-name name args func)
48 (if func
49 (let ((handler (ollama-function-handler func)))
50 (if handler
51 (funcall handler args)
52 (format nil
53 "No handler for function ~a, args: ~a" name args)))
54 (error "Unknown function: ~a" name))))
55
56 (defun completions-with-tools (starter-text &optional functions)
57 "Completion with function/tool calling support.
58 STARTER-TEXT is the prompt to send to the LLM.
59 FUNCTIONS is an optional list of registered function names
60 to make available."
61 (let* ((function-defs
62 (when functions
63 (mapcar
64 (lambda (f)
65 (let ((func (gethash f *available-functions*)))
66 (list
67 (cons :|name| (ollama-function-name func))
68 (cons :|description|
69 (ollama-function-description func))
70 (cons :|parameters|
71 (ollama-function-parameters func)))))
72 functions)))
73 (message (list (cons :|role| "user")
74 (cons :|content| starter-text)))
75 (base-data (list (cons :|model| *tool-model-name*)
76 (cons :|stream| nil)
77 (cons :|messages| (list message))))
78 (data (if function-defs
79 (append base-data
80 (list (cons :|tools| function-defs)))
81 base-data))
82 (json-data (lisp-to-json-string data))
83 ;; Hack: cl-json encodes nil as null, but we need false
84 (fixed-json-data
85 (substitute-subseq json-data ":null" ":false"
86 :test #'string=))
87 (curl-command
88 (format nil "curl ~a -d ~s"
89 ollama::*model-host*
90 fixed-json-data)))
91 (multiple-value-bind (content function-call)
92 (ollama-helper curl-command)
93 (if function-call
94 (handle-tool-function-call (car function-call))
95 (or content "No response content")))))
96
97 ;; Define handler functions
98
99 (defun get_weather (args)
100 "Handler for get_weather tool. ARGS is an alist with :location key."
101 (format t "get_weather called with args: ~a~%" args)
102 (let ((location (cdr (assoc :location args))))
103 (format nil "Weather in ~a: Sunny, 72°F" (or location "Unknown"))))
104
105 (defun calculate (args)
106 "Handler for calculate tool. ARGS is an alist with :expression key."
107 (let ((expression (cdr (assoc :expression args))))
108 (if expression
109 (handler-case
110 (format nil "Result: ~a"
111 (eval (read-from-string expression)))
112 (error (e) (format nil "Error calculating: ~a" e)))
113 "No expression provided")))
114
115 ;; Register sample functions with handlers
116 (register-tool-function
117 "get_weather"
118 "Get current weather for a location"
119 (list (cons :|type| "object")
120 (cons :|properties|
121 (list (cons :|location|
122 (list (cons :|type| "string")
123 (cons :|description|
124 "The city name")))))
125 (cons :|required| '("location")))
126 #'get_weather)
127
128 (register-tool-function
129 "calculate"
130 "Perform a mathematical calculation"
131 (list (cons :|type| "object")
132 (cons :|properties|
133 (list (cons :|expression|
134 (list (cons :|type| "string")
135 (cons :|description|
136 "Math expression like 2 + 2")))))
137 (cons :|required| '("expression")))
138 #'calculate)
The core of this system lies in the decoupling of tool definitions from their execution logic. By using the register-tool-function routine, you can define the JSON schema for a tool, specifying required parameters and types, while simultaneously binding it to a specific Lisp function. This allows the handle-tool-function-call dispatcher to act as a bridge, looking up the appropriate handler in the available-functions hash table and executing it with the arguments extracted from the LLM’s response.
One particularly noteworthy aspect of this implementation is its handling of Lisp’s unique syntax and data types during the JSON conversion process. Because standard Lisp libraries often encode nil as null, the code performs a string substitution to ensure the API receives false, which is mandatory for the :stream parameter in the Ollama schema. Additionally, the calculate tool demonstrates the power of this integration by using read-from-string and eval, allowing the LLM to effectively execute dynamic mathematical expressions directly within a Common Lisp REPL or program.
Sample output (I include a lot of debug printout):
1 * (ql:quickload :ollama)
2 To load "ollama":
3 Load 1 ASDF system:
4 ollama
5 ; Loading "ollama"
6 [package ollama].
7
8 use:
9
10 (in-package :ollama)
11 nil
12 * (ollama::completions-with-tools "Use the get_weather tool for: What's the weather like in New York?" '("get_weather" "calculate"))
13 curl http://localhost:11434/api/chat -d "{\"model\":\"qwen3:1.7b\",\"stream\":false,\"messages\":[{\"role\":\"user\",\"content\":\"Use the get_weather tool for: What's the weather like in New York?\"}],\"tools\":[{\"name\":\"get_weather\",\"description\":\"Get current weather for a location\",\"parameters\":{\"type\":\"object\",\"properties\":{\"location\":{\"type\":\"string\",\"description\":\"The city name\"}},\"required\":[\"location\"]}},{\"name\":\"calculate\",\"description\":\"Perform a mathematical calculation\",\"parameters\":{\"type\":\"object\",\"properties\":{\"expression\":{\"type\":\"string\",\"description\":\"Math expression like 2 + 2\"}},\"required\":[\"expression\"]}}]}"
14 Raw response: {"model":"qwen3:1.7b","created_at":"2025-12-28T18:01:28.789187Z","message":{"role":"assistant","content":"","thinking":"Okay, the user is asking about the weather in New York. Let me check the available tools. There's a get_weather tool, which I think is meant to fetch weather information. The function name is probably \"get_weather\" and it takes a parameter, maybe the location. The user specified \"New York,\" so I need to call the get_weather function with \"New York\" as the argument. Let me make sure the parameters are correct. The tool's parameters are described as having a type \"properties\" but no specific details. Since the user provided the location, I'll pass that directly. I should structure the tool call with the name and arguments as a JSON object. Alright, that's it. Just call get_weather with \"New York\" as the argument.\n","tool_calls":[{"id":"call_fadz9if9","function":{"index":0,"name":"","arguments":{"location":"New York"}}}]},"done":true,"done_reason":"stop","total_duration":2964277542,"load_duration":79982833,"prompt_eval_count":150,"prompt_eval_duration":111485791,"eval_count":181,"eval_duration":2745854365}
15
16 DEBUG handle-tool-function-call: ((index . 0) (name . )
17 (arguments (location . New York)))
18 DEBUG raw-name= inferred-name=get_weather args=((location . New York)) func=#S(ollama-function
19 :name get_weather
20 :description Get current weather for a location
21 :parameters ((type
22 . object)
23 (properties
24 (location
25 (type
26 . string)
27 (description
28 . The city name)))
29 (required
30 location))
31 :handler #<function ollama::get_weather>)
32 get_weather called with args: ((location . New York))
33 "Weather in New York: Sunny, 72°F"
34 *
Using Built In Web Search Tool on Ollama Cloud
The file ollama-cloud-search.lisp demonstrates how to integrate Common Lisp with the Ollama Cloud API to create an autonomous agent capable of performing real-time web searches and content retrieval. By defining explicit tool schemas for the web_search and web_fetch, this example code instructs a model running on the Ollama Cloud service to identify when it requires external data to fulfill a user request. This implementation leverages uiop:run-program to execute curl commands for network communication and utilizes the cl-json library to handle the translation between Lisp association lists and the JSON format required by the API. This architectural pattern transforms a static LLM into a dynamic agent that can bridge the gap between its training data and the live web, specifically handling the iterative loop of requesting tools, executing local functions, and feeding results back to the model until a final answer is synthesized.
1 ;; ollama-cloud-search.lisp
2 (in-package #:ollama)
3
4 ;;; Ollama Cloud agent with web_search and web_fetch tool calling.
5 ;;; Requires OLLAMA_API_KEY to be set in the environment.
6
7 (defvar *cloud-model-name* "gpt-oss:120b-cloud")
8 (defvar *cloud-host* "https://ollama.com/api/chat")
9
10 ;;; Tool schemas sent to the model
11
12 (defvar *web-search-tool-schema*
13 (list (cons :|type| "function")
14 (cons :|function|
15 (list (cons :|name| "web_search")
16 (cons :|description| "Search the web for current information")
17 (cons :|parameters|
18 (list (cons :|type| "object")
19 (cons :|properties|
20 (list (cons :|query|
21 (list (cons :|type| "string")
22 (cons
23 :|description|
24 "The search query string")))))
25 (cons :|required| '("query"))))))))
26
27 (defvar *web-fetch-tool-schema*
28 (list (cons :|type| "function")
29 (cons :|function|
30 (list (cons :|name| "web_fetch")
31 (cons :|description| "Fetch the content of a web page by URL")
32 (cons :|parameters|
33 (list (cons :|type| "object")
34 (cons :|properties|
35 (list (cons :|url|
36 (list (cons :|type| "string")
37 (cons :|description|
38 "The URL to fetch")))))
39 (cons :|required| '("url"))))))))
40
41 ;;; API key helper
42
43 (defun get-api-key ()
44 "Read OLLAMA_API_KEY from the environment. Signals an error if not set."
45 (or (uiop:getenv "OLLAMA_API_KEY")
46 (error "OLLAMA_API_KEY environment variable is not set")))
47
48 ;;; Tool execution
49
50 (defun execute-web-search (args)
51 "Search the web via DuckDuckGo. ARGS is an alist with :query key."
52 (let* ((query (or (cdr (assoc :query args)) ""))
53 (encoded (substitute #\+ #\Space query))
54 (url
55 (format
56 nil
57 "https://api.duckduckgo.com/?q=~a&format=json&no_html=1&skip_disambig=1"
58 encoded))
59 (curl-cmd (format nil "curl -s --max-time 10 ~s" url)))
60 (format t " [web_search] query: ~a~%" query)
61 (handler-case
62 (let ((result
63 (uiop:run-program
64 curl-cmd :output :string :error-output :string)))
65 (format t " [web_search] got ~a chars~%" (length result))
66 result)
67 (error (e) (format nil "web_search error: ~a" e)))))
68
69 (defun execute-web-fetch (args)
70 "Fetch the content of a URL. ARGS is an alist with :url key."
71 (let* ((url (or (cdr (assoc :url args)) ""))
72 (curl-cmd (format nil "curl -s -L --max-time 15 ~s" url)))
73 (format t " [web_fetch] url: ~a~%" url)
74 (handler-case
75 (let ((result
76 (uiop:run-program curl-cmd :output :string
77 :error-output :string)))
78 (format t " [web_fetch] got ~a chars~%" (length result))
79 ;; Limit size to avoid overwhelming the model context
80 (subseq result 0 (min 4000 (length result))))
81 (error (e) (format nil "web_fetch error: ~a" e)))))
82
83 ;;; Single API call to Ollama Cloud
84
85 (defun cloud-ollama-call (messages)
86 "Make one chat request to Ollama Cloud with web_search/web_fetch tools.
87 Returns (values content tool-calls raw-message-alist)."
88 (let* ((api-key (get-api-key))
89 (tools (list *web-search-tool-schema* *web-fetch-tool-schema*))
90 (data (list (cons :|model| *cloud-model-name*)
91 (cons :|stream| nil)
92 (cons :|messages| messages)
93 (cons :|tools| tools)))
94 (json-data (lisp-to-json-string data))
95 ;; Hack: cl-json encodes nil as null, but stream needs false
96 (fixed-json (substitute-subseq json-data ":null" ":false" :test #'string=))
97 (auth-header (format nil "Authorization: Bearer ~a" api-key))
98 (curl-cmd
99 (format nil "curl -s -H ~s -H \"Content-Type: application/json\" ~a -d ~s"
100 auth-header
101 *cloud-host*
102 fixed-json)))
103 (format t "~%Calling Ollama Cloud (~a)...~%" *cloud-model-name*)
104 (handler-case
105 (let ((response
106 (uiop:run-program curl-cmd :output :string
107 :error-output :string)))
108 (format t "Raw response: ~a~%" response)
109 (with-input-from-string (s response)
110 (let* ((parsed (json:decode-json s))
111 ;; raw-message is an alist; re-encoding it preserves tool_calls
112 ;; because cl-json round-trips :TOOL--CALLS <-> "tool_calls"
113 (raw-message (cdr (assoc :message parsed)))
114 (content (cdr (assoc :content raw-message)))
115 (tool-calls (cdr (assoc :tool--calls raw-message))))
116 (values content tool-calls raw-message))))
117 (error (e)
118 (format t "Error calling Ollama Cloud: ~a~%" e)
119 (values nil nil nil)))))
120
121 ;;; Agent loop
122
123 (defun cloud-search-agent (prompt)
124 "Agent loop: calls Ollama Cloud with web_search and web_fetch tools,
125 executing any tool calls and feeding results back until the model
126 returns a final answer. Returns the final answer string."
127 (let ((messages (list (list (cons :|role| "user")
128 (cons :|content| prompt)))))
129 (loop
130 (multiple-value-bind (content tool-calls raw-message)
131 (cloud-ollama-call messages)
132 ;; Append the model's response (including any tool_calls) to history.
133 ;; raw-message is the cl-json decoded alist; re-encoding it is safe because
134 ;; cl-json round-trips :ROLE -> "role", :TOOL--CALLS -> "tool_calls", etc.
135 (when raw-message
136 (setf messages (append messages (list raw-message))))
137
138 (cond
139 ;; Model requested one or more tool calls
140 (tool-calls
141 (format t "~%Model requested ~a tool call(s).~%" (length tool-calls))
142 (dolist (tc tool-calls)
143 (let* ((func (cdr (assoc :function tc)))
144 (name (cdr (assoc :name func)))
145 (args (cdr (assoc :arguments func)))
146 (result
147 (cond
148 ((string= name "web_search") (execute-web-search args))
149 ((string= name "web_fetch") (execute-web-fetch args))
150 (t (format nil "Unknown tool: ~a" name)))))
151 (format t " Tool ~a completed.~%" name)
152 ;; Append tool result to history (role "tool" per Ollama Cloud spec)
153 (setf messages
154 (append messages
155 (list (list (cons :|role| "tool")
156 (cons :|content| (format nil "~a" result))
157 (cons :|tool--name| name)))))))
158 ;; Loop back so the model can process the tool results
159 )
160
161 ;; No tool calls - this is the final answer
162 (t
163 (format t "~%Final Answer: ~a~%" content)
164 (return (or content "No response"))))))))
165
166 ;; Usage:
167 ;; (setf (uiop:getenv "OLLAMA_API_KEY") "your-key-here") ; or export in shell
168 ;; (ollama::cloud-search-agent
169 ;; "What is the current price of Bitcoin and who is the CEO of Nvidia?")
The core of the implementation lies in the cloud-search-agent loop, which manages the stateful conversation history between the user and the assistant. When the model determines that a query requires current information such as the price of a cryptocurrency or recent corporate news, and it returns a tool_calls object instead of a text response. The Lisp code parses these calls, dispatches the appropriate local functions and appends the results to the message list with the specific tool role. This enables the model to “see” the results of its requested actions (i.e., either calls to local Common Lisp functions you write or built in functions like web_search) in the next iteration.
Dear reader, please note the technical detail in the handling of the JSON boolean conversion; since cl-json typically encodes nil as null, the code includes a hack of string substitution to ensure the stream parameter is explicitly sent as false, satisfying the API’s strict type requirements. Additionally, the execute-web-fetch function includes a character limit on the returned content to prevent overwhelming the model’s context window. This conservative approach to tool calling provides a blueprint for building sophisticated Lisp applications that interact with modern, hosted large language models.
Here is an example search tool use (note that I left the debug printout in the example code - you might want to remove these debug print statements after tracing through a few tool calls):
1 $ sbcl
2 * (ql:quickload :ollama)
3 To load "ollama":
4 Load 1 ASDF system:
5 ollama
6 ; Loading "ollama"
7 * (ollama::cloud-search-agent "What is the current price of Bitcoin?")
8
9 Calling Ollama Cloud (gpt-oss:120b-cloud)...
10 Raw response: {"model":"gpt-oss:120b-cloud","created_at":"2026-03-15T16:06:58.193679768Z","message":{"role":"assistant","content":"","thinking":"User asks \"What is the current price of Bitcoin?\" Need to fetch up-to-date price. Use web search.","tool_calls":[{"id":"call_h2hs6qtc","function":{"index":0,"name":"web_search","arguments":{"query":"current price of Bitcoin USD"}}}]},"done":true,"done_reason":"stop","total_duration":957583450,"prompt_eval_count":169,"eval_count":55}
11
12 Model requested 1 tool call(s).
13 [web_search] query: current price of Bitcoin USD
14 [web_search] got 1252 chars
15 Tool web_search completed.
16
17 Calling Ollama Cloud (gpt-oss:120b-cloud)...
18 Raw response: {"model":"gpt-oss:120b-cloud","created_at":"2026-03-15T16:06:59.7867423Z","message":{"role":"assistant","content":"","thinking":"The web search didn't return relevant info; maybe need a better source like CoinDesk. Search again.","tool_calls":[{"id":"call_ay1ikwv5","function":{"index":0,"name":"web_search","arguments":{"query":"Bitcoin price USD site:coindesk.com"}}}]},"done":true,"done_reason":"stop","total_duration":794687005,"prompt_eval_count":542,"eval_count":52}
19
20 Model requested 1 tool call(s).
21 [web_search] query: Bitcoin price USD site:coindesk.com
22 [web_search] got 1252 chars
23 Tool web_search completed.
24
25 Calling Ollama Cloud (gpt-oss:120b-cloud)...
26 Raw response: {"model":"gpt-oss:120b-cloud","created_at":"2026-03-15T16:07:02.297827159Z","message":{"role":"assistant","content":"","thinking":"Seems the search function is not returning real results maybe due to restrictions. Alternative: use known API like CoinGecko. Could fetch https://api.coingecko.com/api/v3/simple/price?ids=bitcoin\u0026vs_currencies=usd . Use web_fetch.","tool_calls":[{"id":"call_72fzg0b0","function":{"index":0,"name":"web_fetch","arguments":{"url":"https://api.coingecko.com/api/v3/simple/price?ids=bitcoin\u0026vs_currencies=usd"}}}]},"done":true,"done_reason":"stop","total_duration":1592044680,"prompt_eval_count":916,"eval_count":100}
27
28 Model requested 1 tool call(s).
29 [web_fetch] url: https://api.coingecko.com/api/v3/simple/price?ids=bitcoin&vs_currencies=usd
30 [web_fetch] got 25 chars
31 Tool web_fetch completed.
32
33 Calling Ollama Cloud (gpt-oss:120b-cloud)...
34 Raw response: {"model":"gpt-oss:120b-cloud","created_at":"2026-03-15T16:07:04.353934655Z","message":{"role":"assistant","content":"**Current Bitcoin Price (USD)** – ≈ **$71,560** \n\n*Source:* CoinGecko API (simple price endpoint) – data fetched just now (2024‑06‑15). \n\n\u003e Prices can fluctuate rapidly across exchanges, so the exact rate may differ by a few dollars at any moment. For the most up‑to‑date figure, you can query the same endpoint or check a live market ticker (e.g., CoinDesk, Binance, Kraken).","thinking":"The fetched price is $71,560. Need to present answer with timestamp. Provide approximate current price. Also note markets vary. Provide source."},"done":true,"done_reason":"stop","total_duration":1341640736,"prompt_eval_count":1037,"eval_count":138}
35
36 Final Answer: **Current Bitcoin Price (USD)** – ≈ **$71,560**
37
38 *Source:* CoinGecko API (simple price endpoint) – data fetched just now (2024‑06‑15).
39
40 > Prices can fluctuate rapidly across exchanges, so the exact rate may differ by a few dollars at any moment. For the most up‑to‑date figure, you can query the same endpoint or check a live market ticker (e.g., CoinDesk, Binance, Kraken).
41 "**Current Bitcoin Price (USD)** – ≈ **$71,560**
42
43 *Source:* CoinGecko API (simple price endpoint) – data fetched just now (2024‑06‑15).
44
45 > Prices can fluctuate rapidly across exchanges, so the exact rate may differ by a few dollars at any moment. For the most up-to-date figure, you can query the same endpoint or check a live market ticker (e.g., CoinDesk, Binance, Kraken)."
46 *
Ollama Chapter Wrap Up
Dear reader, this chapter demonstrates that the marriage of Common Lisp’s symbolic strengths and Ollama’s local inference creates a powerful environment for building autonomous, privacy-respecting AI systems. By bridging Lisp with a local REST API via Common Lisp libraries uiop and cl-json, we have moved beyond simple text generation into the realm of structured tool use and function calling. The architecture we developed—centered around a central dispatcher and a robust registration system—allows the LLM to behave as a high-level controller that can orchestrate native Lisp code to perform calculations, fetch weather data, or even query the live web.
Looking ahead, the shift from local execution to hybrid cloud agents illustrates the evolving landscape of AI development. As seen in the Ollama Cloud integration with web search and web fetch tools, the transition from a deterministic subsystem to a dynamic agent loop requires careful state management and prompt engineering to handle iterative tool requests. Whether you are leveraging the low latency of a local Mistral model instance or the broad capabilities of a cloud-based search agent, the patterns established here of JSON serialization hacks, error handling for external processes, and recursive agent loops provide a flexible foundation for any modern Lisp-based AI application that can be improved by using LLMs.