Using Local LLMs With Ollama

Running local models with Ollama offers several practical advantages for Common Lisp developers, especially those of us building exploratory or long-lived AI systems:

Local inference eliminates network latency and external API dependencies, which simplifies debugging, improves reproducibility, and enables fully offline workflows – important when iterating on symbolic/LLM hybrids or REPL-driven experiments.
Data never leaves the machine, providing strong privacy guarantees and avoiding compliance issues that can arise when sending prompts or intermediate representations to third-party services.
Cost and rate-limit concerns disappear: once a model is downloaded, usage is bounded only by local compute, making it ideal for background agents, continuous evaluation, or batch reasoning tasks initiated from Lisp.
Ollama’s OpenAI-compatible HTTP interface fits naturally with Common Lisp’s strengths – process control, incremental development, and meta-programming, allowing developers to treat local language models as just another deterministic(ish) subsystem under their control.

The ollama package developed here provides generative AI code and tool use/function calling generative AI code in the directory loving-common-lisp/src/ollama. Under the hood, it leverages the litelm library (located in loving-common-lisp/src/litelm), which handles clean message serialization, OpenAI-compatible HTTP routing, streaming, and tool schemas using native Common Lisp data structures.

Design Notes (Optional Material)

Here we describe the design and architecture of the Ollama Common Lisp library, which provides an interface to the Ollama API for running local LLMs using the litelm routing and transport engine.

1. The litelm Foundation

Instead of invoking shell utilities like curl and manipulating raw JSON string substitutions, the ollama library builds on litelm. The litelm library provides:

Clean Lisp S-expression data structures: Messages and tool specifications are written using native Common Lisp symbols, keywords, strings, and numbers rather than ad-hoc JSON dictionaries.
Unified model routing: Models are addressed using a "provider/model-name" string prefix (e.g. "ollama/qwen3-vl:2b" for local Ollama or "ollama-cloud/gpt-oss:120b-cloud" for Ollama Cloud).
HTTP client management: Built on dexador, requests are dispatched directly to OpenAI-compatible REST endpoints (http://localhost:11434/v1 for local Ollama) without spawning external subprocesses.
Robust error handling: HTTP errors map directly to a structured Common Lisp condition hierarchy (litelm:api-error, litelm:rate-limit-error, litelm:authentication-error, etc.).

2. Common Utilities and Configuration

The shared utilities are defined in ollama-helper.lisp and provide configuration used across basic completions, tool-calling, and cloud agent interfaces.

Configuration

model-host – The Ollama API endpoint base URL, defaulting to http://localhost:11434/v1

Routing Helper

ensure-model-name – Ensures that a model identifier string includes the "ollama/" provider prefix required by litelm when routing to the local server.

Package Definition

The ollama package (defined in package.lisp) exports:

completions, completions-with-tools – Main API functions
summarize, answer-question – Convenience wrappers
register-tool-function – Tool registry function
cloud-search-agent – Multi-turn autonomous agent for Ollama Cloud
*model-name*, *tool-model-name*, *model-host*, *cloud-model-name*, *cloud-host* – Configuration variables

3. Generative AI

Basic generative AI functionality is provided in ollama.lisp for text completions without tool calling.

Configuration

model-name – Model identifier, defaults to "qwen3-vl:2b"

Core Functions

completions – Sends a user prompt to the LLM and returns the text response string
- Prepares the model name via ensure-model-name
- Invokes litelm:completion with the prompt and :api-base *model-host*
- Inspects the returned litelm:response struct and extracts content via litelm:response-content

Convenience Wrappers

summarize – Prepends “Summarize: “ to input text and calls completions
answer-question – Formats input as a Q&A prompt and calls completions

Request Flow

1 User Text -> ensure-model-name -> litelm:completion -> HTTP POST ->
2 Ollama /v1/chat/completions -> litelm:response -> Extract Content -> Return String

4. Generative AI with Tools

Tool-calling (function calling) support is implemented in ollama-tools.lisp, enabling the LLM to invoke registered Common Lisp functions.

Configuration

tool-model-name – Model for tool calling, defaults to "qwen3-vl:2b"
available-functions – Hash table storing registered tool functions

Data Structures

ollama-function – Struct containing:
- name – Function identifier string
- description – Human-readable description for the LLM
- parameters – Parameter specification in litelm format
- handler – Common Lisp function to invoke when called

Function Registration

register-tool-function – Registers a tool with the system
- Parameters: name, description, parameters, handler (Lisp function)
- Accepts parameters directly as clean Lisp lists: ((param-name type desc &key required enum) ...)
- Stores an ollama-function struct in *available-functions*

Tool Execution

handle-tool-function-call – Processes an LLM tool call
- Extracts function name and arguments from the litelm:response-tool-calls plist
- Falls back to infer-function-name-from-args if the model returns an empty name
- Looks up the registered handler and invokes it with the decoded argument alist
infer-function-name-from-args – Workaround for models that return empty function names
- Inspects argument keys to determine which function was intended

Main API

completions-with-tools – Enhanced completion with tool support
- Accepts prompt text and an optional list of function name strings to enable
- Translates registered functions into litelm tool definitions
- Sends request to Ollama via litelm:completion
- Runs the full litelm tool-use loop: dispatches every tool call the model returns, appends each result as a :tool message, and calls the model again until it stops requesting tools and answers in natural language

Built-in Tools

Two sample tools are pre-registered:

get_weather – Returns mock weather data for a location
- Parameters: location (string) – The city name
- Returns: Formatted weather string
calculate – Evaluates mathematical expressions
- Parameters: expression (string) – Infix math expression like “2 + 2”
- Tokenizes and parses the infix text into a prefix Lisp form, then computes it with %evaluate-math, which handles only numbers and the operators + - * / – no eval and no symbol resolution, so LLM-supplied text cannot execute arbitrary Lisp

Tool Call Flow

1 User Prompt + Tool Names -> Build Tool Definitions -> litelm:completion ->
2 Ollama API -> Response with Tool Calls -> Parse Function Calls ->
3 For Each Call: Lookup Handler -> Invoke with Arguments -> Append :tool Result ->
4 litelm:completion (repeat until no tool calls) -> Final Answer

Example Usage

1 (ollama:completions-with-tools 
2   "Use the get_weather tool for: What's the weather like in New York?" 
3   '("get_weather" "calculate"))
4 ;; => "The weather in New York is currently Sunny with a temperature of 72°F."

5. Multi-Turn Autonomous Agent with Ollama Cloud

The file ollama-cloud-search.lisp demonstrates how to integrate Common Lisp with the Ollama Cloud service to create an autonomous agent that searches the web and fetches web pages to answer current questions.

Defines explicit tool schemas for web_search and web_fetch in litelm’s declarative tool format.
Registers the :ollama-cloud provider with litelm:define-provider, binding it to "https://ollama.com/v1".
Runs an iterative agent loop that maintains conversation history with litelm message lists:
- Appends user prompts: (:user prompt)
- Records model responses and tool requests: (:assistant content :tool-calls tool-calls)
- Executes requested local functions (e.g. querying DuckDuckGo or fetching URLs)
- Feeds results back into the conversation: (:tool result :tool-call-id id)
- Loops until the model determines it has enough information to synthesize a final answer.

System Definition

The ASDF system (ollama.asd) loads components in dependency order:

package – Package definition
ollama-helper – Shared utilities and routing configuration
ollama-tools – Tool-calling support and tool registry
ollama – Basic completions and convenience wrappers
ollama-cloud-search – Ollama Cloud autonomous search agent

Dependencies: litelm, uiop

Implementation of Common Helper Code

The defpackage form for the #:ollama library establishes an isolated namespace for interacting with local Large Language Models. By relying on #:cl, the package provides a clean public interface for high-level text processing functions like summarize and answer-question, as well as tool-calling routines and cloud agents.

Listing of package.lisp:

 1 ;;;; package.lisp
 2 
 3 (defpackage #:ollama
 4   (:use #:cl)
 5   (:export #:completions
 6            #:completions-with-tools
 7            #:summarize
 8            #:answer-question
 9            #:register-tool-function
10            #:cloud-search-agent
11            #:*model-name*
12            #:*tool-model-name*
13            #:*model-host*
14            #:*cloud-model-name*
15            #:*cloud-host*))

Listing of ollama.asd that defines a defsystem for this package:

 1 ;;;; ollama.asd
 2 
 3 (asdf:defsystem #:ollama
 4   :description "Library for using the ollama APIs via litelm"
 5   :author "Mark Watson"
 6   :license "Apache 2"
 7   :depends-on (#:litelm #:uiop)
 8   :components ((:file "package")
 9                (:file "ollama-helper")
10                (:file "ollama-tools") 
11                (:file "ollama")
12                (:file "ollama-cloud-search")))

The following helper code sets up the environment with a default endpoint and model routing utility. In contrast to earlier designs that performed manual JSON encoding and spawned shell processes, ollama-helper.lisp simply manages the local host endpoint configuration and ensures that model identifiers include the appropriate routing prefix for litelm.

Listing of ollama-helper.lisp:

1 (in-package #:ollama)
2 
3 (defvar *model-host* "http://localhost:11434/v1")
4 
5 (defun ensure-model-name (model)
6   "Ensure MODEL has a provider prefix for litelm routing (defaults to ollama/)."
7   (if (find #\/ model)
8       model
9       (concatenate 'string "ollama/" model)))

Implementation of Generative AI Functionality

In this section, we examine the implementation of basic synchronous text completions. The completions function routes requests through litelm:completion, which handles HTTP communication to the local Ollama server. Beyond the core completion call, the listing provides high-level abstractions for common natural language processing tasks, such as summarization and question answering, illustrating how simple string concatenation formats prompts that guide the model toward specific generative behaviors.

Listing of ollama.lisp:

 1 (in-package #:ollama)
 2 
 3 ;;; Basic Ollama completions without tool calling support
 4 ;;; For tool calling, see ollama-tools.lisp
 5 
 6 (defvar *model-name* "qwen3-vl:2b")
 7 
 8 (defun completions (starter-text &key (model *model-name*))
 9   "Simple completion without function/tool calling support."
10   (let* ((full-model (ensure-model-name model))
11          (resp (litelm:completion full-model
12                                   :messages starter-text
13                                   :api-base *model-host*)))
14     (format t "Raw response: ~s~%" (litelm:response-raw resp))
15     (or (litelm:response-content resp) "No response content")))
16 
17 ;;(ollama:completions "Complete the following text: The President went to")
18 
19 ;; Helper functions for summarization and question answering
20 (defun summarize (some-text)
21   (completions (concatenate 'string "Summarize: " some-text)))
22 
23 (defun answer-question (some-text)
24   (completions (concatenate 'string "
25 Q: " some-text "
26 A:")))

The core of this implementation lies in the completions function. By delegating transport, HTTP connections, and JSON parsing to litelm, the code remains concise and free of low-level shell calls. The raw response is printed for debugging transparency, and the textual answer is extracted directly with litelm:response-content.

The program also showcases the extensibility of the base completion logic through the summarize and answer-question helper functions. These functions act as specialized wrappers that prepend task-specific instructions to the user input, effectively demonstrating basic prompt engineering within a programmatic Common Lisp context.

Implementation of Tool Use/Function Calling Generative AI Functionality

The following listing implements tool-calling (also known as function-calling) within a Common Lisp environment using Ollama and litelm. By defining a custom ollama-function structure and a global registry via a hash table, the code allows developers to map Large Language Model (LLM) tool requests directly to native Lisp handlers. The primary entry point, completions-with-tools, translates the registered tools into litelm’s native Lisp tool format and submits them to Ollama. When the model requests tool calls, the dispatcher unpacks the arguments and invokes the corresponding Common Lisp functions. Because a single model turn may request several tools — and because the model has not yet seen any tool output when it makes that request — completions-with-tools runs the complete litelm tool-use loop: it executes every requested call, appends each result to the conversation as a :tool message, and calls the model again, repeating until the model stops asking for tools and returns a synthesized final answer.

Listing of ollama-tools.lisp:

  1 (in-package #:ollama)
  2 
  3 ;;; Ollama completions with tool/function calling support
  4 ;;; Uses litelm for model routing, message handling, and tool schemas.
  5 
  6 (defvar *tool-model-name* "qwen3-vl:2b")
  7 
  8 (defvar *available-functions* (make-hash-table :test 'equal))
  9 
 10 (defstruct ollama-function
 11   name
 12   description
 13   parameters
 14   handler)  ;; Common Lisp function to handle the call
 15 
 16 (defun register-tool-function (name description parameters handler)
 17   "Register a function that can be called by the LLM via tool calling.
 18    NAME is a string or symbol.
 19    PARAMETERS is a list of parameters in litelm format:
 20      ((param-name param-type param-description &key required enum) ...)
 21    HANDLER is a Common Lisp function that takes an alist of arguments."
 22   (let ((tool-name (string-downcase (string name))))
 23     (setf (gethash tool-name *available-functions*)
 24           (make-ollama-function
 25            :name tool-name
 26            :description description
 27            :parameters parameters
 28            :handler handler))))
 29 
 30 (defun infer-function-name-from-args (args)
 31   "Infer the function name based on argument keys
 32    (workaround for models that return empty name)."
 33   (let ((arg-keys (mapcar (lambda (pair)
 34                             (string-downcase (string (car pair))))
 35                           args)))
 36     (cond
 37       ((member "location" arg-keys :test #'string=) "get_weather")
 38       ((member "expression" arg-keys :test #'string=) "calculate")
 39       (t nil))))
 40 
 41 (defun handle-tool-function-call (function-call)
 42   "Handle a function call returned from the LLM
 43    by invoking the registered handler."
 44   (format t "~%DEBUG handle-tool-function-call: ~a~%" function-call)
 45   (let* ((raw-name (or (getf function-call :name)
 46                        (cdr (assoc :name function-call))))
 47          (args (or (getf function-call :arguments)
 48                    (cdr (assoc :arguments function-call))))
 49          ;; If name is empty, try to infer from arguments
 50          (name (if (or (null raw-name) (string= raw-name ""))
 51                    (infer-function-name-from-args args)
 52                    raw-name))
 53          (func (gethash (string-downcase (string name)) *available-functions*)))
 54     (format t "DEBUG raw-name=~a inferred-name=~a args=~a func=~a~%"
 55             raw-name name args func)
 56     (if func
 57         (let ((handler (ollama-function-handler func)))
 58           (if handler
 59               (funcall handler args)
 60               (format nil
 61                       "No handler for function ~a, args: ~a" name args)))
 62         (error "Unknown function: ~a" name))))
 63 
 64 (defun %convert-to-litelm-tools (functions)
 65   "Convert registered tool names into litelm tool definitions:
 66    ((name description ((param-name param-type param-desc ...) ...)) ...)"
 67   (mapcar (lambda (f)
 68             (let* ((name-str (string-downcase (string f)))
 69                    (func (gethash name-str *available-functions*)))
 70               (unless func
 71                 (error "Unknown tool function: ~a" f))
 72               (list (ollama-function-name func)
 73                     (ollama-function-description func)
 74                     (ollama-function-parameters func))))
 75           functions))
 76 
 77 (defun completions-with-tools (starter-text &optional functions)
 78   "Completion with function/tool calling support.
 79    STARTER-TEXT is the prompt to send to the LLM.
 80    FUNCTIONS is an optional list of registered function names to make available.
 81    Runs the complete litelm tool-use loop: every tool call the model requests is
 82    dispatched, each result is appended to the conversation as a :tool message,
 83    and the model is called again until it returns a plain final answer.
 84    Returns the final answer string."
 85   (let* ((full-model (ensure-model-name *tool-model-name*))
 86          (tool-defs (when functions
 87                       (%convert-to-litelm-tools functions)))
 88          (messages (list (list :user starter-text))))
 89     (loop
 90       (let* ((resp (litelm:completion full-model
 91                                       :messages messages
 92                                       :tools tool-defs
 93                                       :api-base *model-host*))
 94              (content (litelm:response-content resp))
 95              (tool-calls (litelm:response-tool-calls resp)))
 96         (format t "Raw response: ~s~%" (litelm:response-raw resp))
 97         (if tool-calls
 98             (progn
 99               (format t "~%Model requested ~a tool call(s).~%" (length tool-calls))
100               ;; Record the assistant turn that asked for the tool calls.
101               (setf messages
102                     (append messages
103                             (list (list :assistant content :tool-calls tool-calls))))
104               ;; Execute every requested call and feed each result back.
105               (dolist (tc tool-calls)
106                 (let ((result (handle-tool-function-call tc)))
107                   (format t "  Tool ~a completed.~%" (getf tc :name))
108                   (setf messages
109                         (append messages
110                                 (list (list :tool (format nil "~a" result)
111                                             :tool-call-id (getf tc :id))))))))
112             ;; No tool calls - the model produced its final answer.
113             (return (or content "No response content")))))))
114 
115 ;; Define handler functions
116 
117 (defun get_weather (args)
118   "Handler for get_weather tool. ARGS is an alist with :location key."
119   (format t "get_weather called with args: ~a~%" args)
120   (let ((location (or (cdr (assoc :location args :test #'string-equal))
121                       (cdr (assoc "location" args :test #'string-equal)))))
122     (format nil "Weather in ~a: Sunny, 72°F" (or location "Unknown"))))
123 
124 ;;; The calculate tool evaluates ordinary infix arithmetic ("2 + 2",
125 ;;; "(3 + 4) * 2") written by the LLM. That text is untrusted, so the path from
126 ;;; string to number is deliberately narrow:
127 ;;;   1. Tokens are restricted to ASCII digits, "." and the operators + - * / ( ).
128 ;;;   2. Number literals are read with *READ-EVAL* bound to NIL, which disables
129 ;;;      #. read-time evaluation.
130 ;;;   3. The parsed form is walked by %EVALUATE-MATH, which understands only
131 ;;;      numbers and the four operators. No symbol is ever resolved and EVAL is
132 ;;;      never called, so LLM-supplied text cannot execute arbitrary Lisp.
133 ;;; Expression length is capped so a hostile or runaway prompt cannot consume
134 ;;; unbounded time or memory building huge bignums and rationals.
135 
136 (defvar *calculate-max-expression-length* 1000
137   "Maximum number of characters accepted by the calculate tool's :expression.")
138 
139 (defun %ascii-digit-char-p (ch)
140   "True if CH is one of the ASCII digits 0-9."
141   (char<= #\0 ch #\9))
142 
143 (defun %tokenize-math (string)
144   "Split the infix math STRING into a vector of numbers and operator characters."
145   (let ((*read-eval* nil)
146         (tokens '())
147         (number (make-string-output-stream)))
148     (labels ((flush-number ()
149                (let ((text (get-output-stream-string number)))
150                  (when (plusp (length text))
151                    (push (read-from-string text) tokens)))))
152       (loop for ch across string do
153         (cond ((or (%ascii-digit-char-p ch) (char= ch #\.))
154                (write-char ch number))
155               ((find ch "+-*/()") (flush-number) (push ch tokens))
156               (t (flush-number))))
157       (flush-number))
158     (coerce (nreverse tokens) 'vector)))
159 
160 (defun %parse-math (tokens)
161   "Parse the infix TOKENS into a prefix Lisp form. Grammar:
162      expression := term (('+' | '-') term)*
163      term       := factor (('*' | '/') factor)*
164      factor     := number | '(' expression ')' | '-' factor"
165   (let ((pos 0))
166     (labels ((peek () (when (< pos (length tokens)) (aref tokens pos)))
167              (take () (prog1 (peek) (incf pos)))
168              (expression ()
169                (let ((left (term)))
170                  (loop for op = (peek)
171                        while (member op '(#\+ #\-))
172                        do (take)
173                           (setf left (list (if (char= op #\+) '+ '-)
174                                            left (term)))
175                        finally (return left))))
176              (term ()
177                (let ((left (factor)))
178                  (loop for op = (peek)
179                        while (member op '(#\* #\/))
180                        do (take)
181                           (setf left (list (if (char= op #\*) '* '/)
182                                            left (factor)))
183                        finally (return left))))
184              (factor ()
185                (let ((token (take)))
186                  (cond ((null token) (error "Unexpected end of expression"))
187                        ((numberp token) token)
188                        ((char= token #\() (prog1 (expression)
189                                             (unless (eql (take) #\))
190                                               (error "Missing close parenthesis"))))
191                        ((char= token #\-) (list '- (factor)))
192                        (t (error "Unexpected token ~s" token))))))
193       (let ((form (expression)))
194         (when (peek)
195           (error "Unexpected trailing input"))
196         form))))
197 
198 (defun %evaluate-math (form)
199   "Evaluate a parsed arithmetic FORM of numbers and the operators + - * /.
200 Returns a number. Signals an error for any other shape, so a malformed tree can
201 never reach the Lisp evaluator."
202   (cond
203     ((numberp form) form)
204     ;; Unary minus, produced by FACTOR for input such as "-3 + 10".
205     ((and (consp form) (eq (car form) '-) (null (cddr form)))
206      (- (%evaluate-math (second form))))
207     ;; Binary operation: exactly three elements, operator restricted to + - * /.
208     ((and (consp form) (member (car form) '(+ - * /)) (null (cdddr form)))
209      (let ((left (%evaluate-math (second form)))
210            (right (%evaluate-math (third form))))
211        (ecase (car form)
212          (+ (+ left right))
213          (- (- left right))
214          (* (* left right))
215          (/ (/ left right)))))
216     (t (error "Refusing to evaluate unsafe expression form: ~s" form))))
217 
218 (defun calculate (args)
219   "Handler for calculate tool. ARGS is an alist with :expression key."
220   (format t "calculate called with args: ~a~%" args)
221   (let ((expression (or (cdr (assoc :expression args :test #'string-equal))
222                         (cdr (assoc "expression" args :test #'string-equal)))))
223     (if expression
224         (handler-case
225             (progn
226               (when (> (length expression) *calculate-max-expression-length*)
227                 (error "Expression too long (~a characters, limit is ~a)"
228                        (length expression) *calculate-max-expression-length*))
229               (format nil "Result: ~a"
230                       (%evaluate-math (%parse-math (%tokenize-math expression)))))
231           (error (e) (format nil "Error calculating: ~a" e)))
232         "No expression provided")))
233 
234 ;; Register sample functions with handlers
235 (register-tool-function
236  "get_weather"
237  "Get current weather for a location"
238  '((location "string" "The city name"))
239  #'get_weather)
240 
241 (register-tool-function
242  "calculate"
243  "Perform a mathematical calculation"
244  '((expression "string" "Math expression like 2 + 2"))
245  #'calculate)

The core of this system lies in decoupling tool definitions from their execution logic. By using the register-tool-function routine, you can define tools using clean Common Lisp parameter lists, specifying parameter names, types, and descriptions, while simultaneously binding them to a specific Lisp function. This allows the handle-tool-function-call dispatcher to act as a bridge, looking up the appropriate handler in the *available-functions* hash table and executing it with the arguments returned by the LLM. The surrounding loop in completions-with-tools then closes the circuit: each handler result is added back to messages as a :tool message carrying the originating call’s :tool-call-id, so the next litelm:completion call lets the model read the tool output and answer the user in natural language.

Additionally, the calculate tool shows what it takes to run a safe evaluator over untrusted model output. Because the model writes human-style infix arithmetic, %tokenize-math splits the text into numbers and operator characters and %parse-math turns that token stream into a prefix Lisp form – but the result is never handed to eval. Instead %evaluate-math walks the form itself, accepting only numbers and the operators + - * /, so no symbol from the LLM is ever resolved and no arbitrary code can run. Three further details matter: the tokenizer accepts only ASCII digits (not digit-char-p, which on SBCL also returns true for non-ASCII digits such as Arabic-Indic U+0663), number literals are read with *read-eval* bound to nil so #. read-time evaluation is disabled, and the expression length is capped so a runaway prompt cannot build enormous bignums or rationals. A naive (eval (read-from-string expression)) would be wrong twice over: read-from-string stops after the first form, so "2 + 2" would evaluate to just 2, and any text reaching eval is a code-execution path. Malformed or rejected input is caught by handler-case and returned as an "Error calculating: ..." string, which the tool loop hands back to the model so it can correct itself on the next turn.

Sample REPL session:

 1 * (ql:quickload :ollama)
 2 To load "ollama":
 3   Load 1 ASDF system:
 4     ollama
 5 ; Loading "ollama"
 6 [package ollama].
 7 
 8 * (ollama:completions-with-tools "Use the get_weather tool for: What's the weather like in New York?" '("get_weather" "calculate"))
 9 Raw response: (("id" . "chatcmpl-336") ("object" . "chat.completion")
10                ("created" . 1789576838) ("model" . "qwen3-vl:2b")
11                ("system_fingerprint" . "fp_ollama")
12                ("choices"
13                 (("index" . 0)
14                  ("message" ("role" . "assistant") ("content" . "")
15                   ("reasoning"
16                    . "Okay, the user is asking about the weather in New York. ...")
17                   ("tool_calls"
18                    (("id" . "call_g0yphv05") ("index" . 0)
19                     ("type" . "function")
20                     ("function" ("name" . "get_weather")
21                      ("arguments" . "{\"location\":\"New York\"}")))))
22                  ("finish_reason" . "tool_calls")))
23                ("usage" ("prompt_tokens" . 238) ("completion_tokens" . 103) ("total_tokens" . 341)))
24 
25 Model requested 1 tool call(s).
26 
27 DEBUG handle-tool-function-call: (ID call_g0yphv05 NAME get_weather ARGUMENTS ((LOCATION . New York)))
28 DEBUG raw-name=get_weather inferred-name=get_weather args=((LOCATION . New York)) func=#S(OLLAMA-FUNCTION :NAME get_weather ...)
29 get_weather called with args: ((LOCATION . New York))
30   Tool get_weather completed.
31 Raw response: (("id" . "chatcmpl-200") ("object" . "chat.completion")
32                ("created" . 1789576839) ("model" . "qwen3-vl:2b")
33                ("system_fingerprint" . "fp_ollama")
34                ("choices"
35                 (("index" . 0)
36                  ("message" ("role" . "assistant")
37                   ("content"
38                    . "The weather in New York is currently **Sunny** with a temperature of **72°F**. Enjoy your day! 😊")
39                   ("reasoning"
40                    . "Okay, the user asked for the weather in New York. ..."))
41                  ("finish_reason" . "stop")))
42                ("usage" ("prompt_tokens" . 283) ("completion_tokens" . 153) ("total_tokens" . 436)))
43 
44 "The weather in New York is currently **Sunny** with a temperature of **72°F**. Enjoy your day! 😊"

Note that the model is called twice. The first turn returns a tool_calls finish reason and no prose, so completions-with-tools executes get_weather and appends both the assistant turn and the tool result to messages. The second turn then has the tool output available and returns the final natural-language answer with a stop finish reason, which ends the loop. When a single turn requests several tools (for example, asking for both the weather and a calculation), the dolist dispatches each one and appends one :tool message per call before the next model call.

Using Built In Web Search Tool on Ollama Cloud

The file ollama-cloud-search.lisp demonstrates how to integrate Common Lisp with the Ollama Cloud API to create an autonomous agent capable of performing real-time web searches and content retrieval. By defining explicit tool specifications for web_search and web_fetch, this example code instructs a hosted model running on Ollama Cloud to identify when it requires external data to fulfill a user request.

By using litelm, tool specifications and messages are represented as standard Lisp lists. The agent loop iteratively calls litelm:completion, executes any requested tool calls, appends the tool results to the conversation history, and continues until the model returns a synthesized natural-language answer.

Listing of ollama-cloud-search.lisp:

  1 (in-package #:ollama)
  2 
  3 ;;; Ollama Cloud agent with web_search and web_fetch tool calling using litelm.
  4 ;;; Requires OLLAMA_API_KEY to be set in the environment.
  5 
  6 (defvar *cloud-model-name* "gpt-oss:120b-cloud")
  7 (defvar *cloud-host* "https://ollama.com/v1")
  8 
  9 ;; Register Ollama Cloud provider with litelm
 10 (eval-when (:load-toplevel :execute)
 11   (litelm:define-provider :ollama-cloud "https://ollama.com/v1"
 12     :env-keys '("OLLAMA_API_KEY")))
 13 
 14 ;;; Tool definitions in litelm format
 15 
 16 (defvar *cloud-search-tools*
 17   '((web_search "Search the web for current information"
 18       ((query "string" "The search query string")))
 19     (web_fetch "Fetch the content of a web page by URL"
 20       ((url "string" "The URL to fetch")))))
 21 
 22 ;;; API key helper
 23 
 24 (defun get-api-key ()
 25   "Read OLLAMA_API_KEY from the environment. Signals an error if not set."
 26   (or (uiop:getenv "OLLAMA_API_KEY")
 27       (error "OLLAMA_API_KEY environment variable is not set")))
 28 
 29 ;;; Tool execution
 30 
 31 (defun execute-web-search (args)
 32   "Search the web via DuckDuckGo. ARGS is an alist with :query key."
 33   (let* ((query (or (cdr (assoc :query args :test #'string-equal))
 34                     (cdr (assoc "query" args :test #'string-equal))
 35                     ""))
 36          (encoded (substitute #\+ #\Space query))
 37          (url (format nil
 38                       "https://api.duckduckgo.com/?q=~a&format=json&no_html=1&skip_disambig=1"
 39                       encoded))
 40          (curl-cmd (format nil "curl -s --max-time 10 ~s" url)))
 41     (format t "  [web_search] query: ~a~%" query)
 42     (handler-case
 43         (let ((result (uiop:run-program curl-cmd :output :string :error-output :string)))
 44           (format t "  [web_search] got ~a chars~%" (length result))
 45           result)
 46       (error (e) (format nil "web_search error: ~a" e)))))
 47 
 48 (defun execute-web-fetch (args)
 49   "Fetch the content of a URL. ARGS is an alist with :url key."
 50   (let* ((url (or (cdr (assoc :url args :test #'string-equal))
 51                   (cdr (assoc "url" args :test #'string-equal))
 52                   ""))
 53          (curl-cmd (format nil "curl -s -L --max-time 15 ~s" url)))
 54     (format t "  [web_fetch] url: ~a~%" url)
 55     (handler-case
 56         (let ((result (uiop:run-program curl-cmd :output :string :error-output :string)))
 57           (format t "  [web_fetch] got ~a chars~%" (length result))
 58           ;; Limit size to avoid overwhelming the model context
 59           (subseq result 0 (min 4000 (length result))))
 60       (error (e) (format nil "web_fetch error: ~a" e)))))
 61 
 62 ;;; Agent loop using litelm
 63 
 64 (defun cloud-search-agent (prompt &key (model *cloud-model-name*))
 65   "Agent loop: calls Ollama Cloud with web_search and web_fetch tools,
 66    executing any tool calls and feeding results back until the model
 67    returns a final answer. Returns the final answer string."
 68   (let ((messages (list (list :user prompt)))
 69         (full-model (if (find #\/ model)
 70                         model
 71                         (concatenate 'string "ollama-cloud/" model))))
 72     (loop
 73       (format t "~%Calling Ollama Cloud (~a)...~%" model)
 74       (let* ((resp (litelm:completion full-model
 75                                       :messages messages
 76                                       :tools *cloud-search-tools*
 77                                       :api-base *cloud-host*
 78                                       :api-key (get-api-key)))
 79              (content (litelm:response-content resp))
 80              (tool-calls (litelm:response-tool-calls resp)))
 81         (format t "Raw response: ~s~%" (litelm:response-raw resp))
 82         (cond
 83           ;; Model requested one or more tool calls
 84           (tool-calls
 85            (format t "~%Model requested ~a tool call(s).~%" (length tool-calls))
 86            ;; Append assistant message with tool calls to history
 87            (setf messages
 88                  (append messages
 89                          (list (list :assistant content :tool-calls tool-calls))))
 90            (dolist (tc tool-calls)
 91              (let* ((name (getf tc :name))
 92                     (args (getf tc :arguments))
 93                     (result
 94                       (cond
 95                         ((string-equal name "web_search") (execute-web-search args))
 96                         ((string-equal name "web_fetch")  (execute-web-fetch args))
 97                         (t (format nil "Unknown tool: ~a" name)))))
 98                (format t "  Tool ~a completed.~%" name)
 99                ;; Append tool result to history in litelm message format
100                (setf messages
101                      (append messages
102                              (list (list :tool (format nil "~a" result)
103                                          :tool-call-id (getf tc :id))))))))
104           ;; No tool calls - this is the final answer
105           (t
106            (format t "~%Final Answer: ~a~%" content)
107            (return (or content "No response"))))))))

The core of the implementation lies in the cloud-search-agent loop, which manages the stateful conversation history between the user and the assistant. When the model determines that a query requires current information (such as the price of a cryptocurrency or recent corporate news), it returns tool calls instead of a final text response. The Lisp code parses these calls, dispatches the appropriate local functions (execute-web-search or execute-web-fetch), and appends the results to the message list with the :tool role. This enables the model to see the results of its requested actions in the subsequent turn.

Additionally, the execute-web-fetch function enforces a character limit on the returned HTML/text content to prevent overwhelming the model’s context window.

Here is an example search tool session:

 1 $ sbcl
 2 * (ql:quickload :ollama)
 3 To load "ollama":
 4   Load 1 ASDF system:
 5     ollama
 6 ; Loading "ollama"
 7 * (ollama:cloud-search-agent "What is the current price of Bitcoin?")
 8 
 9 Calling Ollama Cloud (gpt-oss:120b-cloud)...
10 Raw response: ...
11 Model requested 1 tool call(s).
12   [web_search] query: current price of Bitcoin USD
13   [web_search] got 1252 chars
14   Tool web_search completed.
15 
16 Calling Ollama Cloud (gpt-oss:120b-cloud)...
17 Raw response: ...
18 Model requested 1 tool call(s).
19   [web_fetch] url: https://api.coingecko.com/api/v3/simple/price?ids=bitcoin&vs_currencies=usd
20   [web_fetch] got 25 chars
21   Tool web_fetch completed.
22 
23 Calling Ollama Cloud (gpt-oss:120b-cloud)...
24 Raw response: ...
25 
26 Final Answer: **Current Bitcoin Price (USD)**: approx. **$71,560**  
27 
28 *Source:* CoinGecko API (simple price endpoint)

Ollama Chapter Wrap Up

Dear reader, this chapter demonstrates that the marriage of Common Lisp’s symbolic strengths and Ollama’s local inference creates a powerful environment for building autonomous, privacy-respecting AI systems. By utilizing the litelm routing engine, we have eliminated brittle subprocess calls and string replacement hacks, replacing them with idiomatic Lisp S-expressions for messages, tools, and response handling.

The architecture we developed – centered around a central dispatcher, a declarative tool registry, and an autonomous agent loop – allows the LLM to behave as a high-level controller that can orchestrate native Lisp code to perform calculations, fetch weather data, or query the live web.

Looking ahead, the shift from single-turn local execution to multi-turn cloud and local agents illustrates the evolving landscape of AI development. Whether you are leveraging the low latency and zero cost of a local Qwen model instance or the broad capabilities of a cloud-based search agent, the patterns established here provide a solid foundation for any modern Lisp-based AI application.

Optional Practice Problems

Custom Temperature and Parameter Configuration: Extend the completions function in ollama.lisp to support optional configuration parameters such as :temperature, :max-tokens, or :top-p. Pass these parameters directly to litelm:completion via keyword arguments and verify that the local model adjusts its creativity or output length accordingly.
System Prompt Support: Modify the message construction inside completions in ollama.lisp to accept a :system-prompt keyword argument. If provided, format the messages list as ((:system system-prompt) (:user starter-text)) when invoking litelm:completion. Test this extension by instructing the local LLM to restrict its answers to a specific persona or output format.
Streaming Responses to the REPL: The litelm library supports streaming via the :stream keyword argument to litelm:completion. Modify completions in ollama.lisp to accept an optional :stream argument (defaulting to nil). When t, pass a callback to litelm:completion that immediately writes each token delta to *standard-output* and flushes with finish-output.
Bounding the Tool Loop: The loop in completions-with-tools in ollama-tools.lisp repeats until the model stops requesting tools, with no upper bound. Add a :max-iterations keyword argument (for example, defaulting to 8) and, when the limit is reached, stop and return the most recent assistant content together with a diagnostic noting that the limit was hit. Verify that a prompt engineered to keep requesting tools terminates instead of looping indefinitely.
Tool Error Handling and Recovery: An uncaught error anywhere in the dispatch aborts the entire tool loop. handle-tool-function-call signals an error for an unregistered tool name, and a handler may throw for any other reason. Wrap the handler dispatch in ollama-tools.lisp in handler-case and return a descriptive message string as the tool result instead. Then exercise the path — for example, register a tool whose handler deliberately signals an error — and observe how the LLM recovers in the multi-turn loop. (Note that calculate already catches its own arithmetic errors such as division by zero, so pick a different failure for this exercise.)
Web Search Fallback for Local Models: Combine the tool-calling mechanism of completions-with-tools from ollama-tools.lisp and the DuckDuckGo query mechanism of execute-web-search/execute-web-fetch from ollama-cloud-search.lisp to run entirely on a local model (such as qwen3-vl:2b or qwen3.5:2b). Register the web tools locally and test how effectively small local models can perform multi-turn search queries.

Advanced Reasoning with World Models and LLMs: The llm-reasoning Library

Up next

Image Processing With Local Ollama Models