A Racket Coding Agent
The source code for this example is in the directory coding-agent-harness.
The Agentic Loop
Modern large language models are not limited to answering questions in a single turn. When given access to tools (i.e., callable functions that can read files, run commands, or search the web) an LLM can operate as an autonomous agent: it reasons about what it needs to know, calls a tool to gather information, receives the result, and continues reasoning until the task is complete. This pattern is called an agentic loop.
For a coding assistant, the loop typically looks like this:
- The user describes a change or a bug to fix.
- The LLM decides it needs to read a file and calls
read_file. - After seeing the file contents, the LLM proposes an edit via
propose_edit. - The user reviews the colored diff and approves or rejects it.
- On approval, the agent writes the file and runs
make check. - The LLM reads the check result and either continues or summarizes what changed.
The key architectural insight is that the LLM is stateless between API calls. It only knows what is in the message history. The agent accumulates tool results into that history turn by turn, giving the model the context it needs to decide what to do next.
This chapter builds a complete Racket implementation of such a coding agent. The agent classifies each user request as a coding task, a general question, or a hybrid, and routes it accordingly. It supports live web search via Brave or Exa AI, renders colored unified diffs before any file is written, and lets the user interrupt a running task at any time with a single ESC keypress. It runs against either a cloud provider (Fireworks AI) or a local model server (Ollama) through a single provider-agnostic loop.
Module Architecture
The project is organized into eight source files, each with a single clear responsibility:
1 chat-loop.rkt Provider-agnostic agentic tool loop shared by both backends
2 fireworks-ai.rkt Fireworks API client (SSE streaming), session stats, chat helpers
3 ollama-ai.rkt Local Ollama API client, session stats, chat helpers
4 tools.rkt Tool registry, five coding tools, the propose_edit approval gate
5 approval.rkt Colored diff printer, ESC-aware y/n/s prompt
6 search.rkt Brave Search and Exa AI search backends
7 interrupt.rkt Shared thread-safe task-interrupt flag
8 agent.rkt REPL, intent classifier, provider dispatch, top-level run
The static dependency graph is nearly linear, but there is a subtle circularity: chat-loop.rkt needs to dispatch tool calls defined in tools.rkt, while tools.rkt needs the interrupt flag and the approval prompt. The interrupt flag is the key to resolving this cleanly. It lives in its own tiny module, interrupt.rkt, with no dependencies, so every other module can require it statically. The approval prompt, which needs the flag, lives in approval.rkt; the tools, which need both the flag and the prompt, live in tools.rkt. By factoring the shared flag into its own leaf module, no module needs a dynamic-require and the graph stays acyclic.
The Shared Interrupt Flag
The ESC-to-interrupt feature needs a single flag that many modules read and write: the loop checks it before each iteration, the tool executor checks it before each tool call, and the approval prompt sets it when ESC is pressed. Because it is shared so widely, it lives in its own dependency-free module, interrupt.rkt:
1 #lang racket
2
3 (provide task-interrupted?
4 task-interrupted-set!
5 task-interrupted-clear!)
6
7 (define task-interrupted-box (box #f))
8 (define task-interrupted-sema (make-semaphore 1))
9
10 (define (task-interrupted?)
11 (call-with-semaphore task-interrupted-sema
12 (lambda () (unbox task-interrupted-box))))
13
14 (define (task-interrupted-set!)
15 (call-with-semaphore task-interrupted-sema
16 (lambda () (set-box! task-interrupted-box #t))))
17
18 (define (task-interrupted-clear!)
19 (call-with-semaphore task-interrupted-sema
20 (lambda () (set-box! task-interrupted-box #f))))
The flag uses a plain box (a mutable cell) protected by a semaphore because the main thread writes to it while the worker thread reads from it. call-with-semaphore acquires the semaphore, runs the body, and releases the semaphore even if an exception is raised.
The Provider-Agnostic Agentic Loop
The heart of the agent is a loop that is deliberately independent of any particular model vendor. It lives in chat-loop.rkt and is parameterized by a post-fn argument: a function that takes an OpenAI-style chat-completions payload and returns a normalized response hash. Both Fireworks and Ollama supply their own post-fn, and the loop never talks to a network directly.
Here is the complete file:
1 #lang racket
2
3 (require racket/string
4 "interrupt.rkt"
5 "tools.rkt")
6
7 (provide chat*
8 chat-with-tools*)
9
10 (define (without-dangling msgs)
11 (if (and (not (null? msgs))
12 (hash-has-key? (last msgs) 'tool_calls))
13 (drop-right msgs 1)
14 msgs))
15
16 (define (chat* post-fn messages
17 #:model-id model-id
18 #:max-tokens max-tokens
19 #:temperature temperature)
20 (define payload
21 (hash 'model model-id
22 'max_tokens max-tokens
23 'temperature temperature
24 'messages messages))
25 (define data (post-fn payload))
26 (define content
27 (hash-ref (hash-ref (first (hash-ref data 'choices)) 'message) 'content ""))
28 (if (and (string? content) (not (string=? content "")))
29 content
30 "No response content"))
31
32 (define (chat-with-tools* post-fn messages tools
33 #:model-id model-id
34 #:max-tokens max-tokens
35 #:temperature temperature
36 #:max-iterations max-iterations)
37 (define tools-rendered (render-tools tools))
38 (define current-messages (box (map (lambda (m) m) messages)))
39
40 (define (loop iter)
41 (cond
42 [(task-interrupted?)
43 (values "(task interrupted by user)" (without-dangling (unbox current-messages)))]
44 [(>= iter max-iterations)
45 ;; One final no-tools call to generate a summary
46 (if (task-interrupted?)
47 (values "(task interrupted by user)" (without-dangling (unbox current-messages)))
48 (let ()
49 (define payload
50 (hash 'model model-id
51 'max_tokens max-tokens
52 'temperature temperature
53 'messages (unbox current-messages)))
54 (with-handlers ([exn:fail? (lambda (_) (values "(max tool iterations reached)" (unbox current-messages)))])
55 (define data (post-fn payload))
56 (define msg (hash-ref (first (hash-ref data 'choices)) 'message))
57 (define content (hash-ref msg 'content ""))
58 (values (if (and (string? content) (not (string=? content "")))
59 content
60 "(no summary from model)")
61 (append (unbox current-messages) (list msg))))))]
62 [else
63 (define payload
64 (let ([base (hash 'model model-id
65 'max_tokens max-tokens
66 'temperature temperature
67 'messages (unbox current-messages))])
68 (if (null? tools-rendered)
69 base
70 (hash-set* base 'tools tools-rendered 'tool_choice "auto"))))
71 (define data (post-fn payload))
72 (define msg (hash-ref (first (hash-ref data 'choices)) 'message))
73 (define tool-calls (hash-ref msg 'tool_calls #f))
74 (define content (hash-ref msg 'content ""))
75 (set-box! current-messages (append (unbox current-messages) (list msg)))
76 (cond
77 [(task-interrupted?)
78 (values "(task interrupted by user)" (without-dangling (unbox current-messages)))]
79 [(and content tool-calls (not (string=? (string-trim content) "")))
80 ;; Model narrated its reasoning AND requested tools: show the text, then run.
81 (displayln "")
82 (displayln (string-trim content))
83 (define results (execute-tool-calls tool-calls))
84 (for ([r (in-list results)])
85 (define call-id (first r))
86 (define name (second r))
87 (define result-str (third r))
88 (set-box! current-messages
89 (append (unbox current-messages)
90 (list (hash 'role "tool"
91 'tool_call_id call-id
92 'name name
93 'content result-str)))))
94 (loop (add1 iter))]
95 [(not tool-calls)
96 (values (or content "(empty response from model)") (unbox current-messages))]
97 [else
98 (define results (execute-tool-calls tool-calls))
99 (for ([r (in-list results)])
100 (define call-id (first r))
101 (define name (second r))
102 (define result-str (third r))
103 (set-box! current-messages
104 (append (unbox current-messages)
105 (list (hash 'role "tool"
106 'tool_call_id call-id
107 'name name
108 'content result-str)))))
109 (loop (add1 iter))])]))
110
111 (loop 0))
At each iteration the model either produces text (the loop ends) or requests tool calls. Tool results are appended as messages with role "tool" and the loop recurses. If the interrupt flag is set at any iteration boundary, the loop returns immediately with a graceful status message.
There are two edge cases worth noting. The first is a model that emits both text and tool calls in the same turn: the text is printed to the user immediately (so it can narrate “I’ll read the file first”), then the tools run, then the loop continues. The second is the iteration cap: at max-iterations the loop makes one final call with no tools, asking the model to summarize what it did, rather than returning nothing.
The without-dangling helper strips a trailing assistant message that has tool_calls but no corresponding tool results – a history in that state would be rejected by the API.
The Fireworks AI Client
The API and Pricing
Fireworks AI is a hosted inference platform that serves many open-weight models through an OpenAI-compatible API. This agent defaults to DeepSeek v4 Flash, a fast and cost-effective model well-suited for coding tasks. The pricing at the time of writing is $0.14 per million uncached input tokens, $0.028 per million cached input tokens (an 80 percent cache discount), and $0.28 per million output tokens.
The estimated session cost accumulated over a conversation is:
where 
is the total prompt tokens, 
is the cached portion of those prompt tokens (billed at the discount), and 
is the total completion tokens. The agent tracks all of these and displays the running total on demand.
Streaming with Server-Sent Events
Unlike the plain one-shot chat, the Fireworks client streams its responses using server-sent events (SSE). The API sends a sequence of data: lines, each carrying a small delta of the response, terminated by a data: [DONE] line. Streaming has two benefits: the user sees the reply being written in real time, and there is no wall-clock cap on generation time. The only timeouts are CURL-MAX-TIME (seconds to wait for response headers and the TCP connection) and STREAM-IDLE-TIMEOUT (seconds of silence from the server before giving up). As long as tokens keep flowing, a request may run for minutes.
Here is the complete file:
1 #lang racket
2
3 (require net/http-easy
4 json
5 racket/string
6 racket/port
7 "interrupt.rkt"
8 "tools.rkt"
9 "chat-loop.rkt")
10
11 (provide FIREWORKS-ENDPOINT
12 FIREWORKS-MODEL
13 MAX-TOKENS
14 DEBUG-LOG
15 CURL-MAX-TIME
16 PRICE-PER-M-PROMPT
17 PRICE-PER-M-CACHED-PROMPT
18 PRICE-PER-M-COMPLETION
19 prompt-cost
20 completion-cost
21 cached-cost
22 accumulate-usage
23 reset-session-stats
24 session-cost
25 print-session-stats
26 chat
27 chat-with-tools
28 make-sse-line-reader
29 parse-sse-response)
30
31 (define FIREWORKS-ENDPOINT "https://api.fireworks.ai/inference/v1/chat/completions")
32 (define FIREWORKS-MODEL (make-parameter "accounts/fireworks/models/deepseek-v4-flash-0731"))
33 (define MAX-TOKENS 32768)
34 (define DEBUG-LOG (make-parameter #f))
35 (define CURL-MAX-TIME 600)
36 (define STREAM-IDLE-TIMEOUT 300)
37
38 (define PRICE-PER-M-PROMPT 0.14)
39 (define PRICE-PER-M-CACHED-PROMPT 0.028)
40 (define PRICE-PER-M-COMPLETION 0.28)
41
42 (define stats-sema (make-semaphore 1))
43 (define session-prompt-tokens (box 0))
44 (define session-completion-tokens (box 0))
45 (define session-total-tokens (box 0))
46 (define session-cached-tokens (box 0))
47
48 (define (reset-session-stats)
49 (call-with-semaphore stats-sema
50 (lambda ()
51 (set-box! session-prompt-tokens 0)
52 (set-box! session-completion-tokens 0)
53 (set-box! session-total-tokens 0)
54 (set-box! session-cached-tokens 0))))
55
56 (define (prompt-cost tokens)
57 (* tokens PRICE-PER-M-PROMPT (/ 1 1000000)))
58
59 (define (cached-cost tokens)
60 (* tokens PRICE-PER-M-CACHED-PROMPT (/ 1 1000000)))
61
62 (define (completion-cost tokens)
63 (* tokens PRICE-PER-M-COMPLETION (/ 1 1000000)))
64
65 (define (session-cost)
66 (call-with-semaphore stats-sema
67 (lambda ()
68 (define pt (unbox session-prompt-tokens))
69 (define ca (unbox session-cached-tokens))
70 (+ (prompt-cost (max 0 (- pt ca)))
71 (cached-cost ca)
72 (completion-cost (unbox session-completion-tokens))))))
73
74 (define (print-session-stats)
75 (define-values (pt ct tt ca)
76 (call-with-semaphore stats-sema
77 (lambda ()
78 (values (unbox session-prompt-tokens)
79 (unbox session-completion-tokens)
80 (unbox session-total-tokens)
81 (unbox session-cached-tokens)))))
82 (define cost (session-cost))
83 (displayln "")
84 (displayln "Session token usage:")
85 (displayln (format " Prompt tokens: ~a" pt))
86 (displayln (format " Completion tokens: ~a" ct))
87 (displayln (format " Total tokens: ~a" tt))
88 (when (> ca 0)
89 (define pct (* 100.0 (/ ca (max 1 pt))))
90 (displayln (format " Cached tokens: ~a (~a% of prompt)" ca (~r pct #:precision 1))))
91 (displayln (format " Estimated cost: $~a ($~a/M input, $~a/M cached input, $~a/M output)"
92 (~r cost #:precision 6)
93 (~r PRICE-PER-M-PROMPT #:precision 4)
94 (~r PRICE-PER-M-CACHED-PROMPT #:precision 4)
95 (~r PRICE-PER-M-COMPLETION #:precision 4))))
96
97 (define (accumulate-usage data)
98 (define usage (hash-ref data 'usage (hash)))
99 (when (and (hash? usage) (not (hash-empty? usage)))
100 (call-with-semaphore stats-sema
101 (lambda ()
102 (set-box! session-prompt-tokens
103 (+ (unbox session-prompt-tokens)
104 (hash-ref usage 'prompt_tokens 0)))
105 (set-box! session-completion-tokens
106 (+ (unbox session-completion-tokens)
107 (hash-ref usage 'completion_tokens 0)))
108 (set-box! session-total-tokens
109 (+ (unbox session-total-tokens)
110 (hash-ref usage 'total_tokens 0)))
111 (define details (hash-ref usage 'prompt_tokens_details (hash)))
112 (when (hash? details)
113 (set-box! session-cached-tokens
114 (+ (unbox session-cached-tokens)
115 (hash-ref details 'cached_tokens 0))))))))
116
117 (define (get-api-key)
118 (define key (getenv "FIREWORKS_API_KEY"))
119 (unless (and key (not (string=? key "")))
120 (error 'fireworks-ai "FIREWORKS_API_KEY environment variable not set"))
121 key)
122
123 (define (bytes-index-of bstr b)
124 (let loop ([i 0]
125 [len (bytes-length bstr)])
126 (cond
127 [(= i len) #f]
128 [(= (bytes-ref bstr i) b) i]
129 [else (loop (add1 i) len)])))
130
131 (define (make-sse-line-reader in)
132 (define buf (make-bytes 4096))
133 (define acc (box #""))
134 (define (read-more!)
135 (unless (sync/timeout STREAM-IDLE-TIMEOUT
136 (handle-evt in (lambda (_) #t)))
137 (error 'fireworks-ai
138 "stream idle timeout: no data from Fireworks for ~a seconds"
139 STREAM-IDLE-TIMEOUT))
140 (define n (read-bytes-avail! buf in))
141 (cond
142 [(eof-object? n) #t]
143 [else
144 (when (> n 0)
145 (set-box! acc (bytes-append (unbox acc) (subbytes buf 0 n))))
146 #f]))
147 (lambda ()
148 (let loop ()
149 (define data (unbox acc))
150 (define nl (bytes-index-of data 10)) ; 10 == #\n
151 (cond
152 [nl
153 (set-box! acc (subbytes data (add1 nl)))
154 (subbytes data 0 nl)]
155 [(read-more!)
156 (define rest (unbox acc))
157 (set-box! acc #"")
158 (if (zero? (bytes-length rest)) eof (subbytes rest 0))]
159 [else (loop)]))))
160
161 (define (parse-sse-chunk body)
162 (with-handlers ([exn:fail? (lambda (_) #f)])
163 (string->jsexpr body)))
164
165 (define (parse-sse-response in)
166 (define content-out (open-output-string))
167 (define reasoning-out (open-output-string))
168 (define tool-calls-by-index (make-hash))
169 (define usage #f)
170 (define finish-reason #f)
171 (define message-id (box ""))
172 (define message-model (box ""))
173 (define next-line (make-sse-line-reader in))
174 (let loop ()
175 (define line (next-line))
176 (cond
177 [(eof-object? line) (void)]
178 [else
179 (define trimmed (string-trim (bytes->string/utf-8 line)))
180 (cond
181 [(or (string=? trimmed "")
182 (string-prefix? trimmed ":"))
183 (loop)]
184 [(string-prefix? trimmed "data:")
185 (define body (string-trim (substring trimmed 5)))
186 (cond
187 [(string=? body "[DONE]") (void)]
188 [else
189 (define chunk (parse-sse-chunk body))
190 (when (hash? chunk)
191 (when (hash-has-key? chunk 'error)
192 (define err (hash-ref chunk 'error))
193 (define msg
194 (cond
195 [(hash? err) (hash-ref err 'message (format "~a" err))]
196 [else (format "~a" err)]))
197 (error 'fireworks-ai "Fireworks API error: ~a" msg))
198 (when (hash-has-key? chunk 'id)
199 (set-box! message-id (hash-ref chunk 'id "")))
200 (when (hash-has-key? chunk 'model)
201 (set-box! message-model (hash-ref chunk 'model "")))
202 (define chunk-usage (hash-ref chunk 'usage #f))
203 (when (and chunk-usage (hash? chunk-usage))
204 (set! usage chunk-usage))
205 (for ([c (in-list (hash-ref chunk 'choices '()))])
206 (define delta (hash-ref c 'delta (hash)))
207 (define fr (hash-ref c 'finish_reason #f))
208 (when (and fr (not (equal? fr finish-reason)))
209 (set! finish-reason fr))
210 (define c-delta (hash-ref delta 'content #f))
211 (when (string? c-delta)
212 (display c-delta content-out))
213 (define r-delta (hash-ref delta 'reasoning_content #f))
214 (when (string? r-delta)
215 (display r-delta reasoning-out))
216 (define tc (hash-ref delta 'tool_calls #f))
217 (when (and tc (list? tc))
218 (for ([t (in-list tc)])
219 (define idx (hash-ref t 'index 0))
220 (define entry (hash-ref tool-calls-by-index idx #f))
221 (unless entry
222 (set! entry (make-hash (list (cons 'id "")
223 (cons 'type "function")
224 (cons 'name "")
225 (cons 'arguments (box "")))))
226 (hash-set! tool-calls-by-index idx entry))
227 (define t-id (hash-ref t 'id #f))
228 (when (and (string? t-id) (not (string=? t-id "")))
229 (hash-set! entry 'id t-id))
230 (define t-type (hash-ref t 'type #f))
231 (when (string? t-type)
232 (hash-set! entry 'type t-type))
233 (define f (hash-ref t 'function #f))
234 (when (hash? f)
235 (define f-name (hash-ref f 'name #f))
236 (when (and (string? f-name) (not (string=? f-name "")))
237 (hash-set! entry 'name f-name))
238 (define f-args (hash-ref f 'arguments #f))
239 (when (and (string? f-args) (not (string=? f-args "")))
240 (define b (hash-ref entry 'arguments))
241 (set-box! b (string-append (unbox b) f-args))))))))
242 (loop)])]
243 [else (loop)])]))
244 (define content (get-output-string content-out))
245 (define reasoning (get-output-string reasoning-out))
246 (define idxs (sort (hash-keys tool-calls-by-index) <))
247 (define tool-calls
248 (if (null? idxs)
249 #f
250 (for/list ([idx (in-list idxs)])
251 (define e (hash-ref tool-calls-by-index idx))
252 (hash 'id (hash-ref e 'id)
253 'type (hash-ref e 'type)
254 'function (hash 'name (hash-ref e 'name)
255 'arguments (unbox (hash-ref e 'arguments)))))))
256 (define message
257 (if tool-calls
258 (hash 'role "assistant"
259 'content content
260 'reasoning_content reasoning
261 'tool_calls tool-calls)
262 (hash 'role "assistant"
263 'content content
264 'reasoning_content reasoning)))
265 (hash 'id (unbox message-id)
266 'model (unbox message-model)
267 'choices (list (hash 'message message
268 'finish_reason finish-reason))
269 'usage (or usage (hash))))
270
271 (define (post-fireworks payload)
272 (define api-key (get-api-key))
273 (define headers
274 (hash 'content-type "application/json"
275 'accept "application/json"
276 'authorization (string-append "Bearer " api-key)))
277 (define stream-payload
278 (hash-set* payload
279 'stream #t
280 'stream_options (hash 'include_usage #t)))
281 (when (DEBUG-LOG)
282 (displayln (format "[DEBUG] request: ~a" (jsexpr->string (hash-remove stream-payload 'messages)))))
283 (define data
284 (with-handlers ([exn:fail? (lambda (e) (error 'fireworks-ai "HTTP error: ~a" (exn-message e)))])
285 (define resp
286 (post FIREWORKS-ENDPOINT
287 #:headers headers
288 #:json stream-payload
289 #:stream? #t
290 #:close? #f
291 #:timeouts (make-timeout-config #:request CURL-MAX-TIME
292 #:connect CURL-MAX-TIME)))
293 (define j (parse-sse-response (response-output resp)))
294 (response-close! resp)
295 (when (DEBUG-LOG)
296 (displayln (format "[DEBUG] response: ~a" (jsexpr->string j))))
297 j))
298 (when (hash-has-key? data 'error)
299 (define err (hash-ref data 'error))
300 (define msg
301 (cond
302 [(hash? err) (hash-ref err 'message (format "~a" err))]
303 [else (format "~a" err)]))
304 (error 'fireworks-ai "Fireworks API error: ~a" msg))
305 (accumulate-usage data)
306 (unless (hash-has-key? data 'choices)
307 (error 'fireworks-ai "Fireworks response has no 'choices'. Raw: ~a" (jsexpr->string data)))
308 data)
309
310 (define (chat messages
311 #:model-id [model-id (FIREWORKS-MODEL)]
312 #:max-tokens [max-tokens MAX-TOKENS]
313 #:temperature [temperature 0.6])
314 (chat* post-fireworks messages
315 #:model-id model-id
316 #:max-tokens max-tokens
317 #:temperature temperature))
318
319 (define (chat-with-tools messages tools
320 #:model-id [model-id (FIREWORKS-MODEL)]
321 #:max-tokens [max-tokens MAX-TOKENS]
322 #:temperature [temperature 0.6]
323 #:max-iterations [max-iterations 20])
324 (chat-with-tools* post-fireworks messages tools
325 #:model-id model-id
326 #:max-tokens max-tokens
327 #:temperature temperature
328 #:max-iterations max-iterations))
Reassembling the SSE Stream
The SSE stream is a sequence of lines like:
1 data: {"id":"...","choices":[{"delta":{"content":"The"}}]}
2
3 data: {"id":"...","choices":[{"delta":{"content":" answer"}}]}
4
5 data: [DONE]
make-sse-line-reader returns a stateful function that yields one line at a time, buffering partial lines between calls and applying the idle timeout. parse-sse-response then walks those lines and reassembles the deltas:
contentdeltas are appended to a string output port.reasoning_contentdeltas (for reasoning models) go to a separate port.tool_callsdeltas are the tricky part, because a single tool call’s name and arguments arrive split across many chunks. The code accumulates them in a hash keyed by the call’sindex, appending argument fragments to a boxed string. At the end it sorts the indices and rebuilds the tool-call list.- The final
usagechunk is captured for token accounting.
The output is a single normalized response hash with the same shape the non-streaming Ollama backend produces, so chat-loop.rkt never knows which backend it is talking to.
Token Accounting and Cost
Because Fireworks is a paid service, the module tracks usage. The session counters live in boxes guarded by a semaphore, because the REPL can run tasks in background threads. Cached input tokens are reported by Fireworks in usage.prompt_tokens_details.cached_tokens; they are part of prompt_tokens but are billed at the discounted rate. The session-cost function subtracts them from the uncached pool and bills them separately. The /tokens command prints the breakdown, including the cached-token percentage.
The Ollama Client
The Ollama backend, ollama-ai.rkt, mirrors the Fireworks interface so that the agent loop and the REPL can swap providers with a single parameter. The differences are all about the wire format. Ollama’s /api/chat endpoint is non-streaming, expects tool-call arguments as objects rather than JSON strings, and reports token counts as prompt_eval_count/eval_count rather than prompt_tokens/completion_tokens.
Here is the complete file:
1 #lang racket
2
3 (require net/http-easy
4 json
5 racket/string
6 "fireworks-ai.rkt" ; for DEBUG-LOG (shared /debug toggle)
7 "chat-loop.rkt")
8
9 (provide OLLAMA-ENDPOINT
10 OLLAMA-MODEL
11 OLLAMA-THINK
12 ollama-reset-session-stats
13 ollama-print-session-stats
14 post-ollama
15 ollama-chat
16 ollama-chat-with-tools)
17
18 (define OLLAMA-ENDPOINT "http://localhost:11434/api/chat")
19 (define OLLAMA-MODEL (make-parameter "nemotron-3.5-lightning:30b-mlx"))
20 (define OLLAMA-THINK (make-parameter #f))
21 (define MAX-TOKENS 32768)
22 (define OLLAMA-MAX-TIME 900)
23 (define OLLAMA-CONNECT-TIME 10)
24
25 (define stats-sema (make-semaphore 1))
26 (define session-prompt-tokens (box 0))
27 (define session-completion-tokens (box 0))
28
29 (define (ollama-reset-session-stats)
30 (call-with-semaphore stats-sema
31 (lambda ()
32 (set-box! session-prompt-tokens 0)
33 (set-box! session-completion-tokens 0))))
34
35 (define (ollama-accumulate-usage usage)
36 (when (and (hash? usage) (not (hash-empty? usage)))
37 (call-with-semaphore stats-sema
38 (lambda ()
39 (set-box! session-prompt-tokens
40 (+ (unbox session-prompt-tokens)
41 (hash-ref usage 'prompt_tokens 0)))
42 (set-box! session-completion-tokens
43 (+ (unbox session-completion-tokens)
44 (hash-ref usage 'completion_tokens 0)))))))
45
46 (define (ollama-print-session-stats)
47 (define-values (pt ct)
48 (call-with-semaphore stats-sema
49 (lambda ()
50 (values (unbox session-prompt-tokens)
51 (unbox session-completion-tokens)))))
52 (displayln "")
53 (displayln "Session token usage (local Ollama -- no API cost):")
54 (displayln (format " Prompt tokens: ~a" pt))
55 (displayln (format " Completion tokens: ~a" ct))
56 (displayln (format " Total tokens: ~a" (+ pt ct)))
57 (displayln (format " Estimated cost: $0 (local model ~a)" (OLLAMA-MODEL))))
58
59 (define (sanitize-message msg)
60 (define role (hash-ref msg 'role "user"))
61 (define content (hash-ref msg 'content ""))
62 (define tcs (hash-ref msg 'tool_calls #f))
63 (cond
64 [(equal? role "tool")
65 (hash 'role "tool"
66 'content (if (string? content) content (format "~a" content)))]
67 [(and (equal? role "assistant") (list? tcs))
68 (hash 'role "assistant"
69 'content (if (string? content) content "")
70 'tool_calls
71 (for/list ([tc (in-list tcs)])
72 (define f (hash-ref tc 'function (hash)))
73 (define args (hash-ref f 'arguments "{}"))
74 (define args-obj
75 (cond
76 [(hash? args) args]
77 [(string? args)
78 (with-handlers ([exn:fail? (lambda (_) (hash))])
79 (string->jsexpr args))]
80 [else (hash)]))
81 (hash 'function (hash 'name (hash-ref f 'name "")
82 'arguments args-obj))))]
83 [else
84 (hash 'role role
85 'content (if (string? content) content ""))]))
86
87 (define (normalize-response r)
88 (define msg (hash-ref r 'message (hash)))
89 (define raw-tcs (hash-ref msg 'tool_calls #f))
90 (define tool-calls
91 (and (list? raw-tcs)
92 (not (null? raw-tcs))
93 (for/list ([tc (in-list raw-tcs)] [i (in-naturals 1)])
94 (define f (hash-ref tc 'function (hash)))
95 (define args (hash-ref f 'arguments (hash)))
96 (hash 'id (format "call_~a" i)
97 'type "function"
98 'function (hash 'name (hash-ref f 'name "")
99 'arguments (if (string? args)
100 args
101 (jsexpr->string args)))))))
102 (define content (hash-ref msg 'content ""))
103 (define thinking (hash-ref msg 'thinking ""))
104 (define message
105 (if tool-calls
106 (hash 'role "assistant"
107 'content (if (string? content) content "")
108 'reasoning_content (if (string? thinking) thinking "")
109 'tool_calls tool-calls)
110 (hash 'role "assistant"
111 'content (if (string? content) content "")
112 'reasoning_content (if (string? thinking) thinking ""))))
113 (define prompt-tokens (hash-ref r 'prompt_eval_count 0))
114 (define completion-tokens (hash-ref r 'eval_count 0))
115 (hash 'model (hash-ref r 'model "")
116 'choices (list (hash 'message message
117 'finish_reason (hash-ref r 'done_reason #f)))
118 'usage (hash 'prompt_tokens prompt-tokens
119 'completion_tokens completion-tokens
120 'total_tokens (+ prompt-tokens completion-tokens))))
121
122 (define (post-ollama payload)
123 (define request
124 (hash 'model (hash-ref payload 'model)
125 'messages (map sanitize-message (hash-ref payload 'messages '()))
126 'stream #f
127 'think (OLLAMA-THINK)
128 'options (hash 'num_predict (hash-ref payload 'max_tokens MAX-TOKENS)
129 'temperature (hash-ref payload 'temperature 0.6))))
130 (define request*
131 (if (hash-has-key? payload 'tools)
132 (hash-set request 'tools (hash-ref payload 'tools))
133 request))
134 (when (DEBUG-LOG)
135 (displayln (format "[DEBUG] ollama request: ~a"
136 (jsexpr->string (hash-remove request* 'messages)))))
137 (define data
138 (with-handlers ([exn:fail? (lambda (e) (error 'ollama-ai "HTTP error: ~a" (exn-message e)))])
139 (define resp
140 (post OLLAMA-ENDPOINT
141 #:json request*
142 #:timeouts (make-timeout-config #:request OLLAMA-MAX-TIME
143 #:connect OLLAMA-CONNECT-TIME)))
144 (define j (response-json resp))
145 (when (DEBUG-LOG)
146 (displayln (format "[DEBUG] ollama response: ~a" (jsexpr->string j))))
147 j))
148 (unless (hash-has-key? data 'message)
149 (error 'ollama-ai "Ollama response has no 'message'. Raw: ~a" (jsexpr->string data)))
150 (define normalized (normalize-response data))
151 (ollama-accumulate-usage (hash-ref normalized 'usage))
152 normalized)
153
154 (define (ollama-chat messages
155 #:model-id [model-id (OLLAMA-MODEL)]
156 #:max-tokens [max-tokens MAX-TOKENS]
157 #:temperature [temperature 0.6])
158 (chat* post-ollama messages
159 #:model-id model-id
160 #:max-tokens max-tokens
161 #:temperature temperature))
162
163 (define (ollama-chat-with-tools messages tools
164 #:model-id [model-id (OLLAMA-MODEL)]
165 #:max-tokens [max-tokens MAX-TOKENS]
166 #:temperature [temperature 0.6]
167 #:max-iterations [max-iterations 20])
168 (chat-with-tools* post-ollama messages tools
169 #:model-id model-id
170 #:max-tokens max-tokens
171 #:temperature temperature
172 #:max-iterations max-iterations))
The Two Conversion Functions
The heart of the Ollama module is a pair of conversion functions at the boundary between the two wire formats.
sanitize-message prepares outgoing messages. It drops keys Ollama does not understand (like reasoning_content and tool_call_id), and crucially converts assistant tool-call arguments from JSON strings back into objects, because that is what Ollama’s API expects.
normalize-response does the reverse. It takes Ollama’s response, which has tool-call arguments as objects and no call ids, and rebuilds the OpenAI-style shape: arguments as a JSON string and a synthesized call_1, call_2, … id per call. It also maps prompt_eval_count to prompt_tokens and eval_count to completion_tokens so the shared token-accounting code in the loop works unchanged.
Because local inference is free, the cost display is always $0. The stats are informational only.
The Tool Registry
Defining and Rendering Tools
tools.rkt maintains a central hash table of all registered tools. Here is the complete file:
1 #lang racket
2
3 (require racket/file
4 racket/port
5 racket/string
6 racket/system
7 racket/list
8 json
9 "interrupt.rkt"
10 "approval.rkt")
11
12 (provide define-tool
13 render-tools
14 execute-tool-calls
15 register-all
16 ENABLED-TOOLS)
17
18 (define registry (make-hash))
19
20 (define SHELL-WHITELIST (set "make" "ls" "pwd" "cat" "uv"))
21 (define MAX-CHECK-OUTPUT-CHARS 2000)
22
23 (define (define-tool name params description handler)
24 (hash-set! registry name
25 (hash 'name name
26 'description description
27 'parameters params
28 'handler handler)))
29
30 (define (render-tools names)
31 (for/list ([name (in-list names)])
32 (define tool (hash-ref registry name #f))
33 (unless tool (error 'render-tools "Undefined tool: ~a" name))
34 (define props (make-hash))
35 (define required '())
36 (for ([p (in-list (hash-ref tool 'parameters))])
37 (define pname (first p))
38 (define ptype (second p))
39 (define pdesc (third p))
40 (hash-set! props (string->symbol pname)
41 (hash 'type ptype 'description pdesc))
42 (set! required (cons pname required)))
43 (hash 'type "function"
44 'function (hash 'name (hash-ref tool 'name)
45 'description (hash-ref tool 'description)
46 'parameters (hash 'type "object"
47 'properties props
48 'required (reverse required))))))
Each tool is stored as a hash with its name, description, parameter list, and handler function. render-tools converts the registry entries into OpenAI function-calling schema format: a list of hashes the API understands as callable functions. The model receives these alongside the conversation and decides which, if any, to invoke.
Tool Dispatch
execute-tool-calls receives the list of tool call objects from the API response and dispatches each one:
1 (define (execute-tool-calls tool-calls)
2 (define results '())
3 (for ([call (in-list tool-calls)])
4 #:break (task-interrupted?)
5 (define call-id (hash-ref call 'id ""))
6 (define func (hash-ref call 'function (hash)))
7 (define name (hash-ref func 'name ""))
8 (define args-json (hash-ref func 'arguments "{}"))
9 (define short
10 (if (<= (string-length args-json) 120)
11 args-json
12 (string-append (substring args-json 0 117) "...")))
13 (displayln (format "* ~a ~a" name short))
14 (define args
15 (with-handlers ([exn:fail? (lambda (_) (hash))])
16 (let ([j (string->jsexpr args-json)])
17 (if (hash? j) j (hash)))))
18 (define result (call-tool name args))
19 (set! results (append results (list (list call-id name result)))))
20 results)
The #:break (task-interrupted?) clause in the for loop is idiomatic Racket: it terminates the loop early if the interrupt flag is set between tool calls. Each tool call prints its name and (truncated) arguments so the user can see what the model is doing in real time.
The Five Coding Tools
The agent registers five tools at startup:
| Tool | Purpose |
|---|---|
read_file |
Return the full text of a file |
list_dir |
List files and subdirectories in a directory |
grep |
Recursively search for an extended regex pattern |
run_shell |
Run a whitelisted shell command and return its output |
propose_edit |
Show a colored diff and ask the user to approve the change |
run_shell enforces a strict command whitelist to prevent the model from running arbitrary shell commands:
1 (define (tool-run-shell command)
2 (define tokens (string-split (string-trim command)))
3 (cond
4 [(null? tokens) "empty command"]
5 [else
6 (define cmd (first tokens))
7 (if (not (set-member? SHELL-WHITELIST cmd))
8 (format "Command '~a' not whitelisted. Allowed: ~a"
9 cmd (string-join (sort (set->list SHELL-WHITELIST) string<?) ", "))
10 (with-handlers ([exn:fail? (lambda (e) (format "Error running command: ~a" (exn-message e)))])
11 (define args (rest tokens))
12 (define-values (out code) (run-external cmd args))
13 (string-append out (format "(exit ~a)" code))))]))
If the model attempts to run a disallowed command, it receives an error string describing what is allowed. It can then adapt its approach rather than causing the agent to crash. The whitelist currently allows make, ls, pwd, cat, and uv (the Python package runner). Everything else is refused.
The propose_edit Approval Gate
propose_edit is the most critical tool. Before writing any file it checks for several error conditions, shows the user a diff, waits for approval, writes the file, and then runs make check:
1 (define (tool-propose-edit path old new)
2 (define exists? (file-exists? path))
3 (define current
4 (if exists?
5 (with-handlers ([exn:fail? (lambda (e) (format "Error reading ~a: ~a" path (exn-message e)))])
6 (file->string path))
7 ""))
8 (when (and exists? (string-prefix? current "Error reading"))
9 current)
10 (cond
11 [(and exists? (not (string=? current old)))
12 (format "stale base: on-disk contents of ~a do not match the 'old' you provided. Read the file again and retry." path)]
13 [(and exists? (string=? current new))
14 "no changes (proposed content matches current file)"]
15 [(and (not exists?) (string=? new ""))
16 "refused: cannot create an empty file"]
17 [else
18 (define diff-text (unified-diff current new (string-append "a/" path) (string-append "b/" path)))
19 (displayln "")
20 (unless exists? (displayln (format "(new file: ~a)" path)))
21 (print-colored-diff diff-text)
22 (define answer (prompt-yes-no-skip))
23 (cond
24 [(eq? answer 'interrupted) "change not applied (task interrupted by user)"]
25 [(eq? answer 'no) "user rejected the change"]
26 [(eq? answer 'skip)
27 (define reason (prompt-reason))
28 (format "user skipped: ~a" reason)]
29 [else ; 'yes
30 (make-parent-directory* path)
31 (call-with-output-file path #:exists 'truncate
32 (lambda (out) (display new out)))
33 (define-values (out status) (run-make-check))
34 (if (= status 0)
35 "applied; make check passed"
36 (format "applied; make check FAILED (exit ~a):\n~a"
37 status (truncate-string out MAX-CHECK-OUTPUT-CHARS)))]))])
The stale-base guard is worth understanding carefully. The model reads a file, then constructs a proposed edit based on that content. If the user edits the file externally in between, a naive tool would overwrite those changes silently. By requiring old to exactly match what is on disk, the tool forces the model to re-read the file before retrying – the mismatch is reported as a tool result the model can read and respond to.
The make check gate closes another important feedback loop. If the edit compiles cleanly, "applied; make check passed" goes back into the conversation history and the model can proceed. If make check fails, the output goes back as well, giving the model the compiler errors it needs to self-correct on the next turn. The output is truncated to MAX-CHECK-OUTPUT-CHARS characters so a huge build log does not blow up the context window.
The Approval and Diff System
Generating a Unified Diff
approval.rkt generates diffs by writing the two file versions to temporary files and calling the system diff -u utility. Here is the full source file:
1 #lang racket
2
3 (require racket/file
4 racket/port
5 racket/string
6 racket/system
7 "interrupt.rkt")
8
9 (provide unified-diff
10 print-colored-diff
11 prompt-yes-no-skip
12 prompt-reason
13 save-stty-state
14 restore-stty-state
15 enter-cbreak-mode)
16
17 (define ANSI-RED "\033[31m")
18 (define ANSI-GREEN "\033[32m")
19 (define ANSI-CYAN "\033[36m")
20 (define ANSI-RESET "\033[0m")
21
22 (define (shell-quote s)
23 (string-append "'"
24 (string-replace s "'" "'\\''")
25 "'"))
26
27 (define (unified-diff old-content new-content old-label new-label)
28 (define old-path (make-temporary-file "rk-diff-old~a"))
29 (define new-path (make-temporary-file "rk-diff-new~a"))
30 (define out-path (make-temporary-file "rk-diff-out~a"))
31 (dynamic-wind
32 void
33 (lambda ()
34 (call-with-output-file old-path #:exists 'truncate
35 (lambda (out) (display old-content out)))
36 (call-with-output-file new-path #:exists 'truncate
37 (lambda (out) (display new-content out)))
38 (define cmd
39 (format "diff -u -L ~a -L ~a ~a ~a > ~a 2>&1"
40 (shell-quote old-label)
41 (shell-quote new-label)
42 (shell-quote (path->string old-path))
43 (shell-quote (path->string new-path))
44 (shell-quote (path->string out-path))))
45 (system cmd)
46 (with-handlers ([exn:fail? (lambda (_) "")])
47 (file->string out-path)))
48 (lambda ()
49 (when (file-exists? old-path) (delete-file old-path))
50 (when (file-exists? new-path) (delete-file new-path))
51 (when (file-exists? out-path) (delete-file out-path)))))
dynamic-wind takes three thunks: a before-thunk (here void), a body-thunk, and an after-thunk. The after-thunk runs whether the body completes normally or raises an exception – analogous to Python’s try/finally. This guarantees the three temporary files are cleaned up regardless of what goes wrong.
Colorizing the Diff
print-colored-diff walks each line of the unified diff output and applies ANSI terminal color codes:
1 (define (print-colored-diff diff-text)
2 (for ([line (in-list (string-split diff-text "\n"))])
3 (cond
4 [(or (string-prefix? line "+++")
5 (string-prefix? line "---")
6 (string-prefix? line "@@"))
7 (displayln (string-append ANSI-CYAN line ANSI-RESET))]
8 [(string-prefix? line "+")
9 (displayln (string-append ANSI-GREEN line ANSI-RESET))]
10 [(string-prefix? line "-")
11 (displayln (string-append ANSI-RED line ANSI-RESET))]
12 [else (displayln line)])))
Lines beginning with + are added lines and appear green; lines beginning with - are removed and appear red; diff headers (+++, ---, @@) appear cyan. This makes it straightforward to review a proposed change without reading both full file versions.
ESC-Aware Prompts
The approval prompt puts the terminal in raw mode so that a bare ESC keypress can be detected immediately, without waiting for the user to press Enter. The trickiest part is distinguishing a bare ESC from the beginning of an ANSI escape sequence (which is how arrow keys and function keys are encoded):
1 (define (read-line-raw)
2 (define esc? #f)
3 (define chars '())
4 (define done? #f)
5 (let loop ()
6 (unless done?
7 (define ready? (char-ready? (current-input-port)))
8 (if ready?
9 (let ([ch (read-char (current-input-port))])
10 (cond
11 [(eof-object? ch) (set! done? #t)]
12 [(char=? ch #\u001b)
13 (sleep 0.05)
14 (if (char-ready? (current-input-port))
15 (let drain ()
16 (when (char-ready? (current-input-port))
17 (read-char (current-input-port))
18 (sleep 0.02)
19 (drain)))
20 (begin (set! esc? #t) (set! done? #t)))
21 (unless done? (loop))]
22 [(or (char=? ch #\newline) (char=? ch #\return))
23 (display "\r")
24 (flush-output)
25 (set! done? #t)]
26 [(or (char=? ch #\backspace) (char=? ch #\rubout))
27 (when (not (null? chars))
28 (set! chars (rest chars))
29 (display "\b \b")
30 (flush-output))
31 (loop)]
32 [(char=? ch (integer->char 3)) ; Ctrl-C -- treat like ESC
33 (set! esc? #t)
34 (set! done? #t)]
35 [else
36 (display ch)
37 (flush-output)
38 (set! chars (cons ch chars))
39 (loop)]))
40 (begin (sleep 0.02) (loop)))))
41 (values (list->string (reverse chars)) esc?))
After reading an ESC character the code waits 50 ms. If the input port has more bytes ready within that window, they belong to an ANSI sequence and are drained. If no bytes arrive, it was a standalone ESC and esc? is set to #t. The function returns two values: the accumulated character string and the ESC flag.
Because raw mode has echo off, read-line-raw echoes printable characters itself and handles backspace by erasing the last buffered character. Ctrl-C is treated like ESC. On newline it emits a bare carriage return so the caller can add the final newline.
The module also provides save-stty-state, restore-stty-state, and enter-cbreak-mode, which capture and restore the terminal state. enter-cbreak-mode runs stty -icanon -echo, deliberately not full stty raw, because full raw mode disables output post-processing and would cause multi-line output to “stair-step” on screen.
prompt-yes-no-skip wraps read-line-raw in dynamic-wind to ensure the terminal is always restored, then interprets the result:
1 (define (prompt-yes-no-skip)
2 (define saved (save-stty-state))
3 (define raw? (try-raw-mode))
4 (dynamic-wind
5 void
6 (lambda ()
7 (let loop ()
8 (when (task-interrupted?) (values 'interrupted))
9 (display "\nApply this change? [y]es / [n]o / [s]kip and tell the model why: ")
10 (flush-output)
11 (define-values (line esc?)
12 (if raw?
13 (read-line-raw)
14 (let ([l (read-line (current-input-port))])
15 (values (if (eof-object? l) "" (string-trim l)) #f))))
16 (when raw? (displayln ""))
17 (cond
18 [esc?
19 (task-interrupted-set!)
20 (displayln "[interrupted]")
21 'interrupted]
22 [(task-interrupted?) 'interrupted]
23 [else
24 (define norm (string-downcase (string-trim line)))
25 (cond
26 [(member norm '("y" "yes")) 'yes]
27 [(member norm '("n" "no")) 'no]
28 [(member norm '("s" "skip")) 'skip]
29 [else
30 (displayln "Please answer y, n, or s.")
31 (loop)])])))
32 (lambda () (when raw? (restore-mode saved)))))
When stty is not available (for example, when running in a non-TTY context), the code falls back to a standard read-line call and ESC detection is disabled.
Web Search Integration
search.rkt provides two search backends with identical return shapes, making them interchangeable at the call site. Here is the complete file:
1 #lang racket
2
3 (require net/http-easy
4 net/uri-codec
5 json
6 racket/string)
7
8 (provide brave-search
9 exa-search)
10
11 (define EXA-ENDPOINT "https://api.exa.ai/search")
12
13 (define (brave-search query [num-results 5])
14 (define api-key (getenv "BRAVE_SEARCH_API_KEY"))
15 (unless (and api-key (not (string=? api-key "")))
16 (error 'brave-search "BRAVE_SEARCH_API_KEY environment variable not set"))
17 (define encoded (uri-encode query))
18 (define url (format "https://api.search.brave.com/res/v1/web/search?q=~a&count=~a"
19 encoded num-results))
20 (define headers
21 (hash 'X-Subscription-Token api-key
22 'content-type "application/json"
23 'accept "application/json"))
24 (define resp (get url #:headers headers))
25 (define data (response-json resp))
26 (define web (hash-ref data 'web (hash)))
27 (define results (hash-ref web 'results '()))
28 (for/list ([r (in-list results)])
29 (list (hash-ref r 'url "")
30 (hash-ref r 'title "")
31 (hash-ref r 'description ""))))
32
33 (define (exa-search query [num-results 5])
34 (define api-key (getenv "EXA_SEARCH_API_KEY"))
35 (unless (and api-key (not (string=? api-key "")))
36 (error 'exa-search "EXA_SEARCH_API_KEY environment variable not set"))
37 (define payload
38 (hash 'query query
39 'type "auto"
40 'numResults num-results
41 'contents (hash 'highlights #t)))
42 (define headers
43 (hash 'content-type "application/json"
44 'authorization (string-append "Bearer " api-key)))
45 (define resp
46 (post EXA-ENDPOINT
47 #:headers headers
48 #:json payload))
49 (define data (response-json resp))
50 (define results (hash-ref data 'results '()))
51 (for/list ([r (in-list results)])
52 (list (hash-ref r 'url "")
53 (hash-ref r 'title "")
54 (let ([hl (hash-ref r 'highlights '())])
55 (if (and (list? hl) (not (null? hl))) (first hl) "")))))
Both functions return a list of (url title description) triples. Brave uses a GET request with an API key header and returns web search results with title and description snippets. Exa uses a POST with a JSON body and returns neural search results with highlighted excerpts.
The net/http-easy package, installable via raco pkg install http-easy, provides the get, post, and response-json procedures used here.
The Main REPL
Provider Dispatch
agent.rkt is the entry point that ties everything together. It holds a PROVIDER parameter selecting 'fireworks (cloud) or 'ollama (local), defaulting to Fireworks unless the AGENT_PROVIDER environment variable is set to ollama. All model calls go through two small dispatch functions that pick the right backend:
1 (define PROVIDER
2 (make-parameter
3 (let ([p (getenv "AGENT_PROVIDER")])
4 (if (and p (string=? (string-downcase p) "ollama")) 'ollama 'fireworks))))
5
6 (define (using-ollama?) (eq? (PROVIDER) 'ollama))
7
8 (define (current-model-id)
9 (if (using-ollama?) (OLLAMA-MODEL) (FIREWORKS-MODEL)))
10
11 (define (set-current-model! m)
12 (if (using-ollama?) (OLLAMA-MODEL m) (FIREWORKS-MODEL m)))
13
14 (define (llm-chat msgs
15 #:max-tokens [max-tokens MAX-TOKENS]
16 #:temperature [temperature 0.6])
17 (if (using-ollama?)
18 (ollama-chat msgs
19 #:model-id (OLLAMA-MODEL)
20 #:max-tokens max-tokens
21 #:temperature temperature)
22 (chat msgs
23 #:model-id (FIREWORKS-MODEL)
24 #:max-tokens max-tokens
25 #:temperature temperature)))
26
27 (define (llm-chat-with-tools msgs tools)
28 (if (using-ollama?)
29 (ollama-chat-with-tools msgs tools #:model-id (OLLAMA-MODEL))
30 (chat-with-tools msgs tools #:model-id (FIREWORKS-MODEL))))
The /provider slash command switches the parameter at runtime, and the /model command changes the model for the current provider.
Intent Classification
Before sending any message to the model, agent.rkt classifies the user’s intent as one of three categories: "general", "coding", or "hybrid". The classification uses a two-stage approach.
Stage one is a keyword heuristic – free and instant:
1 (define GENERAL-KEYWORDS
2 (list "movie" "film" "cinema" "theater" "theatre" "showing" "playing" "showtime"
3 "weather" "forecast" "rain" "snow" "temperature outside"
4 "restaurant" "recipe" "menu" "where to eat"
5 "news" "sports" "score" "standings"
6 "near me" "nearby" "directions to"
7 "hotel" "flight" "travel" "vacation"
8 "population of" "history of" "capital of"
9 "who is " "who was " "where is " "when is " "when does "
10 "price of" "cost of" "how much does"))
11
12 (define CODING-KEYWORDS
13 (list ".lisp" ".py" ".js" ".ts" ".java" ".cpp" ".go" ".rb" ".rs" ".c "
14 "def " "class " "function " "refactor" "implement " "compile" "makefile"
15 "stacktrace" "segfault" "git commit" "git push" "git pull"
16 "unit test" "pull request" "fix the bug" "add a function" "write a function"))
17
18 (define (heuristic-classify lower)
19 (cond
20 [(for/or ([kw (in-list GENERAL-KEYWORDS)])
21 (string-contains? lower kw))
22 "general"]
23 [(for/or ([kw (in-list CODING-KEYWORDS)])
24 (string-contains? lower kw))
25 "coding"]
26 [else #f]))
for/or is the Racket comprehension form that returns the first “truthy” value or #f if none is found.
If the heuristic returns #f (the query is ambiguous), stage two calls the LLM with a minimal two-message conversation and requests a single-word answer:
1 (define (llm-classify user-line)
2 (with-handlers ([exn:fail? (lambda (e)
3 (displayln (format "[Classifier LLM error: ~a — defaulting to coding]" (exn-message e)))
4 "coding")])
5 (define msgs
6 (list (hash 'role "system"
7 'content "You are a one-word query classifier. Reply with exactly one word and nothing else.")
8 (hash 'role "user"
9 'content
10 (string-append
11 "Classify this query as exactly one word — GENERAL, CODING, or HYBRID:\n"
12 "GENERAL = factual or informational; nothing to do with writing, editing, or debugging code.\n"
13 "CODING = writing, editing, refactoring, or debugging code or files.\n"
14 "HYBRID = coding question that benefits from web docs or library references.\n"
15 (format "Query: ~a\n" user-line)
16 "One-word answer:"))))
17 (define raw (llm-chat msgs #:max-tokens 10 #:temperature 0.0))
18 (define up (string-upcase (string-trim raw)))
19 (cond
20 [(string-contains? up "GENERAL") "general"]
21 [(string-contains? up "HYBRID") "hybrid"]
22 [else "coding"])))
23
24 (define (classify-intent user-line)
25 (or (heuristic-classify (string-downcase user-line))
26 (llm-classify user-line)))
max-tokens 10 and temperature 0.0 keep the classifier call cheap and deterministic. If the classifier itself fails, the handler defaults to "coding" – a conservative choice that enables the full tool set.
Routing to the Model
send-to-model uses the classification to choose the right system prompt and call path:
1 (define (send-to-model user-line)
2 (define intent (classify-intent user-line))
3 (define label
4 (hash-ref (hash "general" "web search, no coding tools"
5 "coding" "coding tools, no search"
6 "hybrid" "coding tools + web search if /search is on")
7 intent))
8 (displayln (format "[intent: ~a → ~a]" intent label))
9 (cond
10 [(string=? intent "general")
11 (define content (or (maybe-search user-line #t) user-line))
12 (define msgs
13 (list (hash 'role "system" 'content GENERAL-SYSTEM-PROMPT)
14 (hash 'role "user" 'content content)))
15 (define reply (llm-chat msgs))
16 (displayln (format "\n~a" (clean reply)))]
17 [(string=? intent "coding")
18 (define updated (append (unbox messages-box) (list (hash 'role "user" 'content user-line))))
19 (define-values (reply new-messages)
20 (llm-chat-with-tools updated ENABLED-TOOLS))
21 (set-box! messages-box new-messages)
22 (displayln (format "\n~a" (clean reply)))]
23 [else ; hybrid
24 (define content (or (maybe-search user-line #f) user-line))
25 (define updated (append (unbox messages-box) (list (hash 'role "user" 'content content))))
26 (define-values (reply new-messages)
27 (llm-chat-with-tools updated ENABLED-TOOLS))
28 (set-box! messages-box new-messages)
29 (displayln (format "\n~a" (clean reply)))]))
General questions use a lightweight one-shot call and a simple system prompt. Coding requests go through the full agentic tool loop using a system prompt that describes the five tools and the rules for using them. Hybrid requests get both the tool loop and web search results prepended to the message (if /search is enabled).
The System Prompt
The coding system prompt is set once per session and injected as the first message with role "system". It tells the model which tools are available and how to use them correctly:
1 (define SYSTEM-PROMPT-TEMPLATE
2 "You are an interactive coding assistant working in the directory {cwd}.
3
4 Rules:
5 - Use read_file, list_dir, and grep to understand the code BEFORE proposing edits.
6 - To EDIT an existing file: read_file it first, then pass its exact current contents
7 as `old` to propose_edit.
8 - To CREATE a new file: call propose_edit with the empty string \"\" as `old` and
9 the full desired contents as `new`. Do not call read_file first for a file that
10 does not exist yet.
11 - One file per propose_edit call. Keep diffs small and focused.
12 - If the user rejects an edit or `make check` fails, ask for clarification instead
13 of retrying blindly.
14 - run_shell only accepts whitelisted commands: make, ls, pwd, cat, uv.
15 - When you are done, reply with a short natural-language summary of what changed.")
The {cwd} placeholder is replaced with the actual working directory at session start. Telling the model the working directory helps it construct relative paths for read_file and list_dir calls.
Context Management
As an agentic conversation grows, every tool result is appended to the message list, and the context window fills up. agent.rkt provides two commands to manage this. /context shows a formatted table of messages with estimated character and token counts:
1 (define (show-context)
2 (define msgs (unbox messages-box))
3 (define total (for/sum ([m (in-list msgs)]) (message-char-size m)))
4 (displayln "")
5 (displayln (format "Context: ~a message~a, ~a chars, ~a tokens (est.)"
6 (length msgs)
7 (if (= (length msgs) 1) "" "s")
8 total
9 (quotient total 4)))
10 ...)
/compact sends the whole transcript to the model with a “compactor” system prompt, gets back a dense summary, and replaces everything except the original system prompt with that summary:
1 (define COMPACT-SYSTEM-PROMPT
2 "You are a context compactor for a coding assistant. Summarize the conversation transcript into a compact brief that will replace it. Preserve: the user's goals and instructions, decisions made, files created or modified (with paths), important code and tool-output details, and outstanding tasks. Write dense bullets, no preamble.")
This trades a little fidelity for a lot of context budget, keeping the model inside its window on long sessions.
Skills
The agent supports loading “skills” from ~/.agents/skills/<name>/SKILL.md. Each skill file is a Markdown document that is injected into the conversation as a system message, telling the model to treat it as authoritative guidance. /skills lists available skills (parsing a description: field from each file’s YAML frontmatter), and /<skill-name> loads one. This lets you package reusable instructions that the model will follow for the rest of the session.
The ESC Interrupt
The most novel aspect of the REPL is the ESC-to-interrupt mechanism. The model call runs in a background thread while the main thread polls for ESC in raw terminal mode:
1 (define (run-model-with-escape thunk)
2 (task-interrupted-clear!)
3 (define worker (thread thunk))
4 (define has-stty?
5 (with-handlers ([exn:fail? (lambda (_) #f)])
6 (and (terminal-port? (current-input-port))
7 (system "stty -g >/dev/null 2>&1"))))
8 (define saved-stty (and has-stty? (save-stty-state)))
9 (define escaped? #f)
10 (when has-stty? (enter-cbreak-mode))
11 (dynamic-wind
12 void
13 (lambda ()
14 (let loop ()
15 (when (thread-running? worker)
16 (when (and (not escaped?) (escape-pressed?))
17 (set! escaped? #t)
18 (task-interrupted-set!)
19 (display "\n[ESC — stopping after the current step…]\n")
20 (flush-output))
21 (sleep 0.05)
22 (loop))))
23 (lambda ()
24 (when has-stty?
25 (if saved-stty
26 (restore-stty-state saved-stty)
27 (system "stty sane 2>/dev/null")))))
28 (define done? (sync/timeout INTERRUPT-WAIT-TIMEOUT worker))
29 (unless done?
30 (displayln (format "[Task did not stop within ~as; worker may still be running in background]"
31 INTERRUPT-WAIT-TIMEOUT)))
32 escaped?)
escape-pressed? performs a non-blocking char-ready? check so the poll loop spends nearly all its time sleeping 50 ms between checks. When ESC is detected, task-interrupted-set! flips the shared flag. The worker thread’s chat-with-tools* loop checks that flag at each iteration boundary and returns early without being killed. This cooperative shutdown approach is cleaner than killing the thread forcefully because it allows the thread to restore any state it owns before exiting.
sync/timeout waits up to INTERRUPT-WAIT-TIMEOUT (180) seconds for the worker to finish; if it does not stop within that window the user sees a warning but the REPL continues normally. The dynamic-wind guarantees the terminal state is restored even if the task throws.
The REPL Loop
The main loop in agent.rkt is a straightforward tail-recursive function:
1 (define (run)
2 (register-all)
3 (reset-conversation)
4 (print-banner)
5 (let loop ()
6 (display "\n> ")
7 (flush-output)
8 (define line
9 (with-handlers ([exn:fail? (lambda (_) eof)])
10 (read-line (current-input-port) 'any)))
11 (cond
12 [(eof-object? line)
13 (displayln "")
14 (void)]
15 [else
16 (define trimmed (string-trim line))
17 (cond
18 [(string=? trimmed "") (loop)]
19 [else
20 (define cmd (handle-slash-command trimmed))
21 (cond
22 [(eq? cmd 'quit) (void)]
23 [(eq? cmd 'continue) (loop)]
24 [else
25 (define (task)
26 (with-handlers ([exn:fail? (lambda (e)
27 (unless (task-interrupted?)
28 (displayln (format "\nError talking to model: ~a" (exn-message e)))
29 (flush-output)))])
30 (send-to-model trimmed)))
31 (define interrupted? (run-model-with-escape task))
32 (when interrupted?
33 (displayln "[Interrupted. Type your next request or /reset.]")
34 (flush-output))
35 (loop)])])])))
36
37 (module+ main
38 (run))
handle-slash-command recognizes /reset, /history, /context, /compact, /model, /provider, /debug, /search, /tokens, /help, /skills, and skill names before any model call is made. The module+ main form lets agent.rkt be both loaded as a library (for testing) and run directly from the command line.
Running the Agent
Installation
Install Racket 8.11 or later from racket-lang.org. Then install the http-easy HTTP client package:
1 raco pkg install --auto http-easy
Export your API keys. Only FIREWORKS_API_KEY is required for the cloud provider; the search keys are optional:
1 export FIREWORKS_API_KEY=fw_...
2 export BRAVE_SEARCH_API_KEY=BSA... # optional
3 export EXA_SEARCH_API_KEY=... # optional
For the local provider, make sure an Ollama server is running with a model pulled:
1 ollama pull nemotron-3.5-lightning:30b-mlx
2 export AGENT_PROVIDER=ollama
Starting the REPL
1 make run
or directly:
1 racket agent.rkt
The banner shows the working directory, the active provider, and the active model:
1 Coding Agent REPL. /help for commands, /quit to exit.
2 cwd: /Users/mark/myproject
3 provider: fireworks
4 model: accounts/fireworks/models/deepseek-v4-flash-0731
5
6 >
Sample Session
The following session asks the agent to add a helper function to an existing file. Lines beginning with > are user input; everything else is agent output.
1 > add a function called word-count that takes a string and returns the number of words
2
3 [intent: coding → coding tools, no search]
4 * read_file utils.rkt
5 * propose_edit utils.rkt
6
7 --- a/utils.rkt
8 +++ b/utils.rkt
9 @@ -14,3 +14,7 @@
10 (define (trim-lines text)
11 (string-join (map string-trim (string-split text "\n")) "\n"))
12 +
13 +(define (word-count str)
14 + (length (string-split str)))
15 +
16 +(provide word-count)
17
18 Apply this change? [y]es / [n]o / [s]kip and tell the model why: y
19 applied; make check passed
20
21 Added `word-count` to utils.rkt. It splits the string on whitespace using
22 `string-split` (which treats consecutive spaces as one separator) and returns
23 the length of the resulting list.
24
25 > /tokens
26
27 Session token usage:
28 Prompt tokens: 1842
29 Completion tokens: 87
30 Total tokens: 1929
31 Estimated cost: $0.000482 ($0.1400/M input, $0.0280/M cached input, $0.2800/M output)
32
33 > /quit
Enabling Web Search
Toggle search on with /search. Switch between engines with /search brave or /search exa:
1 > /search brave
2 Web search ON (engine: brave)
3
4 > what is the current version of Racket?
5
6 [intent: general → web search, no coding tools]
7 [Web search results for: what is the current version of Racket?]
8 1. Racket -- A programmable programming language
9 https://racket-lang.org
10 Racket 8.14 was released on ...
11 ...
12
13 As of mid-2026, the current stable release of Racket is version 8.14.
Switching Providers
Switch between Fireworks and Ollama at runtime with /provider:
1 > /provider ollama
2 Provider set to ollama (model: nemotron-3.5-lightning:30b-mlx)
3
4 > /tokens
5
6 Session token usage (local Ollama -- no API cost):
7 Prompt tokens: 0
8 Completion tokens: 0
9 Total tokens: 0
10 Estimated cost: $0 (local model nemotron-3.5-lightning:30b-mlx)
Interrupting a Long Task
Press ESC during any multi-step operation to stop the agent before the next tool call:
1 > refactor all error handling in the project to use a unified log-error helper
2
3 [intent: coding → coding tools, no search]
4 * list_dir .
5 * grep "error" .
6 * read_file src/main.rkt
7 ^[
8 [ESC — stopping after the current step…]
9 [Interrupted. Type your next request or /reset.]
10
11 >
The terminal is restored immediately and the REPL is ready for the next input.
Interpreting the Output
When the agent prints * tool-name arguments it is showing a tool call in progress. The tool name and a truncated version of the arguments help you follow the model’s reasoning. read_file utils.rkt means the model decided it needs to see the file before editing it – a sign it is following the system prompt rules. propose_edit always appears after a read_file for the same path.
make check passed tells you both that the model’s proposed syntax was valid Racket and that your project’s own compile step accepted it. If you see make check FAILED, the failure output follows immediately and appears in the agent’s next prompt – giving the model a second chance to correct the error autonomously.
The /tokens output shows prompt tokens growing much faster than completion tokens. That is expected in an agentic loop: the conversation history (including tool results, which can be long) is re-sent to the model on every iteration, while the model’s replies are comparatively short. The Fireworks cost display also separates cached input tokens, which are billed at an 80 percent discount, from uncached input tokens.
The [intent: ... → ...] line tells you how the agent routed your request. A general question is answered without touching any tools; a coding task gets the full tool loop. If the routing looks wrong, you can inspect the keyword lists and adjust them.
Wrap Up
This chapter built a complete Racket coding agent in roughly 800 lines across eight focused modules. The main ideas were:
Separation of concerns. The shared agentic loop, the two provider clients, the tool registry, the approval UI, and the search backends each live in their own file. The interrupt flag lives in its own dependency-free module, which resolves the circular dependency cleanly without dynamic-require.
Provider abstraction. The agentic loop in chat-loop.rkt is parameterized by a post-fn adapter, so Fireworks (cloud, SSE streaming) and Ollama (local, non-streaming) both run through the identical loop. The only differences are in the wire-format conversion at the boundary.
The stale-base guard in propose_edit. Requiring the model to supply the exact current contents of a file before any edit is accepted prevents silent overwrites when the file changes between the read and edit steps. The mismatch is reported as a tool result the model can read and respond to.
Two-stage intent classification. A free keyword heuristic handles the common cases and falls back to a cheap LLM call only for ambiguous queries. Defaulting to "coding" on classifier failure keeps the full tool set available.
The make check feedback loop. Every accepted edit is immediately verified by the project’s own build target. Failures go back into the conversation history, giving the model the information it needs to self-correct on the next iteration.
Cooperative ESC interruption. Racket’s native threads make it straightforward to run the model call in the background while the main thread polls for user input. The interrupt flag coordinates graceful shutdown without forcefully killing the worker thread, so terminal state is always restored cleanly.
These patterns – tool registries, approval gates with stale-base guards, intent routing, quality gates, provider adapters, and graceful interruption – apply broadly across languages and LLM providers. The Racket implementation here serves as a concrete reference for how each piece fits together at the system level.
Optional Practice Problems
Problem 1: Add a write_file tool
The agent currently has no way for the model to create a file without going through propose_edit. Add a write_file tool to tools.rkt that accepts a path and content parameter, writes the content directly (without a diff prompt), and returns a confirmation string. Register it in register-all and add it to ENABLED-TOOLS. Consider what safety constraints, if any, should prevent the model from overwriting files outside the working directory.
Problem 2: Extend the shell whitelist dynamically
SHELL-WHITELIST is currently a compile-time constant. Add a /allow-cmd slash command to agent.rkt that lets the user append a command to the whitelist at runtime – for example, /allow-cmd git would let the model run git status and git diff. Update handle-slash-command to recognize the new command and update the set stored in tools.rkt. Think about where the mutable whitelist state should live and how tools.rkt should expose it.
Problem 3: Persistent session history
At present, /reset discards the conversation history and there is no way to resume a previous session. Add two slash commands: /save <filename> that writes the current messages-box contents to a JSON file using jsexpr->string, and /load <filename> that reads that file and restores the conversation. Use string->jsexpr for loading. Handle file-not-found and malformed JSON gracefully by printing an error and leaving the existing history unchanged.
Problem 4: Token-budget guard
chat-with-tools* will keep iterating until the model stops calling tools or max-iterations is reached. Add a token-budget guard that checks session-total-tokens after each iteration and returns early with a warning message if the running total exceeds a configurable threshold. Expose the threshold as a /budget <n> slash command that sets it, and a /budget command with no argument that prints the current setting and remaining budget.
Problem 5: Second search backend – DuckDuckGo
Add a ddg-search function to search.rkt using the DuckDuckGo Instant Answer API at https://api.duckduckgo.com/?q=QUERY&format=json. The response contains a RelatedTopics array of objects with Text and FirstURL fields. Return results in the same (url title description) triple format as brave-search and exa-search. Update agent.rkt to accept /search ddg as a valid engine selection.
Problem 6: Colored intent label in the REPL prompt
The line [intent: coding → ...] is printed in plain text. Use ANSI codes (already defined in approval.rkt) to color the label: green for "coding", cyan for "general", and yellow ("\033[33m") for "hybrid". Update send-to-model in agent.rkt to apply the color. Since approval.rkt already defines the ANSI constants, think about whether to require them from there, redefine them locally, or move them to a shared ansi.rkt module.
Problem 7: Retry on make check failure
Currently, when propose_edit runs make check and it fails, the failure output is returned to the model as a tool result – but the model must then propose a new edit from scratch. Modify tool-propose-edit so that on a make check failure it offers the user a [r]etry option at the approval prompt (in addition to the existing y/n/s choices). On retry, revert the file to its previous contents using call-with-output-file, print a confirmation, and return a result string that tells the model the file was reverted and includes the check output so it can try again with a corrected edit.
Problem 8: Stream the SSE deltas to the terminal
The Fireworks client already reassembles the SSE stream into a single response via parse-sse-response, but the user does not see the reply being written in real time. Modify post-fireworks so that, while parse-sse-response is accumulating the response, each delta.content fragment is also displayed to the terminal as it arrives. Consider how to do this without double-printing the final text (which the REPL also prints after the call returns), and whether the tool-call argument fragments should be hidden.