Agentic RAG Using the Gemini LLM APIs
This chapter implements an Agentic Retrieval-Augmented Generation (RAG) system in Common Lisp, inspired by Google’s June 2026 research blog post Unlocking Dependable Responses with Agentic RAG.
In the previous chapter on document question answering, we built a “vanilla” RAG system: embed documents, embed the query, find similar chunks, and pass them to an LLM for answer generation. That approach works well for simple factual questions, but falls short on complex queries that require information from multiple sources or where the first retrieval pass misses critical details.
Agentic RAG addresses this limitation by introducing multiple specialized agents that plan, rewrite queries, assess context sufficiency, and iteratively search until enough information is gathered to produce a reliable answer. The key insight from the Google research is the Sufficient Context Agent, a quality-control step that evaluates whether the retrieved passages actually contain enough information to answer the question, and if not, generates specific feedback about what’s missing so the system can refine its search.
The source code for this example is in the directory src/RAG of the book’s GitHub repository. It uses the Gemini gemini-embedding-001 model for embeddings (free tier) and gemini-3-flash-preview for all agent LLM calls (very inexpensive).
Overview of the Agentic RAG Architecture
The system implements a multi-agent pipeline with five phases:
- Query Rewriting: A Gemini-powered agent decomposes complex questions into 1–3 focused sub-queries for retrieval.
- Search Fanout: All sub-queries are embedded in one batched API call, then each is searched across multiple document corpora. Results are deduplicated.
- Sufficient Context Assessment: A specialized agent evaluates whether the retrieved passages contain enough information to fully answer the original question.
- Iterative Refinement: If context is insufficient, the system generates refined search queries based on feedback about what’s missing, then searches again. This loop repeats up to a configurable limit.
- Synthesis: Once context is sufficient (or the iteration limit is reached), a synthesis agent generates a grounded answer citing source documents.
This differs fundamentally from vanilla RAG. In a vanilla system, if the first retrieval doesn’t find the right passages, you get a partial answer or a hallucination. In agentic RAG, the system recognizes the gap and actively searches for the missing information.
Project Structure
The project is organized as an ASDF system with five source files, a test file, and sample data:
| File | Description |
|---|---|
| embeddings.lisp | Gemini embedding integration (batch API, caching, retries) and cosine similarity |
| vector-store.lisp | In-memory vector store with document chunking and corpus persistence |
| agents.lisp | Multi-agent pipeline (rewriter, search, sufficiency, synthesis) |
| rag.lisp | Top-level API, interactive demo, and test code |
| tests.lisp | Offline unit tests that run without an API key |
| data/ | Sample text documents for the demo |
The ASDF system definition in rag.asd defines both the library and its offline test system:
1 (asdf:defsystem #:rag
2 :description "Agentic RAG (Retrieval-Augmented Generation) using Gemini"
3 :author "Mark Watson"
4 :license "Apache 2"
5 :version "1.1.0"
6 :serial t
7 :depends-on (#:llm #:cl-json #:dexador #:usocket #:uiop)
8 :components ((:file "package")
9 (:file "embeddings")
10 (:file "vector-store")
11 (:file "agents")
12 (:file "rag"))
13 :in-order-to ((asdf:test-op (asdf:test-op #:rag/test))))
14
15 (asdf:defsystem #:rag/test
16 :description "Offline unit tests for the rag system (no network access)."
17 :author "Mark Watson"
18 :license "Apache 2"
19 :depends-on (#:rag #:uiop)
20 :serial t
21 :components ((:file "tests"))
22 :perform (asdf:test-op (op c)
23 (declare (ignore op c))
24 (uiop:symbol-call :rag-tests :run-tests)))
The usocket dependency exists so the retry logic can recognize connection-level errors (refused connections, timeouts, DNS failures), which Dexador signals as usocket conditions rather than HTTP errors.
The package exports the main entry points:
1 (defpackage #:rag
2 (:use #:cl)
3 (:export #:make-corpus
4 #:add-document
5 #:save-corpus
6 #:load-corpus
7 #:corpus-chunk-count
8 #:query
9 #:agentic-rag
10 #:interactive-demo
11 #:test
12 #:*rag-verbose*
13 #:*rag-model*
14 #:*embedding-model*
15 #:*embedding-dimension*
16 #:*embedding-batch-limit*
17 #:*embedding-cache-cap*
18 #:clear-embedding-cache
19 #:cosine-similarity
20 #:dot-product
21 #:vector-magnitude
22 #:normalize-vector))
Computing Embeddings With the Gemini API
The file embeddings.lisp provides the foundation for semantic search. We use Google’s gemini-embedding-001 model, which produces 3072-dimensional vectors and is available on the free tier.
All HTTP access goes through llm:post-json from the llm library we built earlier, so there is a single HTTP path shared with the rest of the book’s examples. The API key travels in the x-goog-api-key request header, never in the URL:
1 (defun %api-headers ()
2 "Headers for every embedding API call: the key travels in the
3 x-goog-api-key header, not the URL."
4 (list '("Content-Type" . "application/json")
5 (cons "x-goog-api-key" (get-google-api-key))))
Keys in URL query strings leak into server logs, proxy logs, and shell history; the header keeps the key out of every place the URL is recorded.
The Dexador HTTP library signals a dex:http-request-failed condition when the API returns an error status, and usocket conditions when the connection itself fails. Not every failure deserves a retry, so a small predicate classifies them:
1 (defun %transient-p (condition)
2 "True when CONDITION is worth retrying: HTTP 429/5xx, or a
3 connection-level failure (usocket). Permanent 4xx client errors
4 (bad request, bad API key, wrong model name) signal immediately."
5 (typecase condition
6 (dex:http-request-failed
7 (let ((status (dex:response-status condition)))
8 (or (= status 429) (>= status 500))))
9 (usocket:socket-error t)
10 (usocket:ns-error t)
11 (t nil)))
A 400 (malformed request) or 401 (bad API key) will fail again in exactly the same way, so retrying just burns seconds of backoff before surfacing the real problem. Status 429 (rate limited) and 5xx (server trouble) usually clear up, and connection errors usually mean a transient network problem:
1 (defparameter *retry-sleep-fn* #'sleep
2 "Function called to pause between retries. Rebind to a no-op in
3 tests so retry backoff does not slow the suite down.")
4
5 (defun call-with-retries (thunk &key (attempts 3) (initial-delay 1.0))
6 "Call THUNK, retrying transient failures (HTTP 429/5xx, connection
7 errors) with exponential backoff (1s, 2s, 4s by default). Permanent
8 HTTP 4xx errors signal immediately; after ATTEMPTS transient
9 failures an error is signaled, including the underlying condition."
10 (loop for attempt from 1
11 for delay = initial-delay then (* delay 2)
12 do (handler-case (return (funcall thunk))
13 (error (e)
14 (cond ((not (%transient-p e))
15 (error "Non-transient API error: ~A" e))
16 ((>= attempt attempts)
17 (error "API request failed after ~A attempts: ~A"
18 attempts e))
19 (t
20 (%debug-log "~%DEBUG call-with-retries: attempt ~A/~A failed ~
21 (~A), retrying in ~A seconds~%"
22 attempt attempts e delay)
23 (funcall *retry-sleep-fn* delay)))))))
The *retry-sleep-fn* variable follows the same rebindable-function idiom we use for the API calls themselves: the default waits for real backoff, and the test suite rebinds it to a no-op so retry tests run instantly.
Debug output throughout the system goes through the %debug-log macro, which prints only when *rag-verbose* is true. The verbose tracing is valuable when following the pipeline in this chapter, and setting the variable to NIL turns the system into a quiet library.
Embeddings are memoized in *embedding-cache*, a hash table keyed on the model name and text, so reloading a document or re-running the demo never pays for the same API call twice. Because a long-running process embedding many queries would grow the cache without bound, %cache-put clears it when it reaches *embedding-cache-cap* (50,000 entries by default):
1 (defun %cache-put (text vec)
2 "Store VEC for TEXT, evicting the whole cache when the cap is reached
3 (simple and safe; a full rebuild costs a few batched API calls)."
4 (when (and *embedding-cache-cap*
5 (>= (hash-table-count *embedding-cache*) *embedding-cache-cap*))
6 (%debug-log "~%DEBUG embedding cache reached ~A entries; clearing~%"
7 *embedding-cache-cap*)
8 (clrhash *embedding-cache*))
9 (setf (gethash (embedding-cache-key text) *embedding-cache*) vec))
The model, its output dimension, and the batch size are configurable:
1 (defparameter *embedding-model* "gemini-embedding-001"
2 "Gemini embedding model name. If you change this you must re-embed
3 existing corpora: saved corpus files hold vectors from the old
4 model/dimension and search signals a dimension mismatch.")
5
6 (defparameter *embedding-dimension* nil
7 "Output embedding dimension, or NIL for the model default (3072 for
8 gemini-embedding-001). The API accepts 768, 1536, or 3072 for this
9 model; 768 saves 4x memory and search time with little quality
10 loss. Set before building or loading a corpus.")
11
12 (defparameter *embedding-batch-limit* 100
13 "Maximum texts per batchEmbedContents request; the API rejects more.")
The *embedding-dimension* knob is worth knowing about. The model supports Matryoshka output: you can truncate the 3072-value vector to 768 or 1536 values and keep most of the retrieval quality, at a quarter (or half) of the memory and search cost. One quirk we discovered testing against the live API: gemini-embedding-001 honors the deprecated top-level outputDimensionality field but silently ignores the newer embedContentConfig, while newer models like gemini-embedding-2 accept both. %make-embedding-request sends the dimension both ways so either model works.
The low-level function %fetch-embedding calls the embedContent endpoint for a single string:
1 (defparameter *embedding-fn* #'%fetch-embedding
2 "Function of one argument (a string) returning an embedding vector.
3 Rebind this in tests to run the pipeline without network access.
4 When left at its default, GET-EMBEDDINGS batches all cache misses
5 through *batch-request-fn* instead.")
6
7 (defun %fetch-embedding (text)
8 "Compute an embedding vector for TEXT via the embedContent endpoint.
9 Returns a simple-vector of floats. Retries transient failures."
10 (let* ((api-url (concatenate 'string
11 *embedding-api-url*
12 *embedding-model*
13 ":embedContent"))
14 (payload (make-hash-table :test 'equal)))
15 (setf (gethash "content" payload)
16 (gethash "content" (%make-embedding-request text))
17 (gethash "model" payload)
18 (concatenate 'string "models/" *embedding-model*))
19 (when *embedding-dimension*
20 (setf (gethash "outputDimensionality" payload) *embedding-dimension*
21 (gethash "embedContentConfig" payload)
22 (let ((cfg (make-hash-table :test 'equal)))
23 (setf (gethash "outputDimensionality" cfg)
24 *embedding-dimension*)
25 cfg)))
26 (coerce (%decode-embedding-response
27 (call-with-retries
28 (lambda ()
29 (llm:post-json api-url (%api-headers) payload))))
30 'simple-vector)))
Note the special variable *embedding-fn*: the public entry point get-embedding calls whatever function it holds. Defaulting it to the real HTTP implementation while allowing tests to rebind it to a stub is a simple Common Lisp idiom we will use again for the LLM calls, and it is what makes the offline unit tests possible.
The batch path has its own injection point. %fetch-embeddings-batch splits its input into groups of at most *embedding-batch-limit* texts (the API rejects a 101st request with an INVALID_ARGUMENT error, which we confirmed the hard way) and calls *batch-request-fn* per group:
1 (defparameter *batch-request-fn* #'%post-batch-request
2 "Function of one argument (a list of texts, at most
3 *embedding-batch-limit* long) returning a list of embedding vectors
4 in the same order. Rebind in tests to stub the HTTP layer.")
5
6 (defun %fetch-embeddings-batch (texts)
7 "Compute embeddings for all TEXTS, splitting into batches of at most
8 *embedding-batch-limit* texts per batchEmbedContents request (the
9 API cap). Returns a list of vectors in the same order as TEXTS."
10 (loop for batch on texts by (lambda (l) (nthcdr *embedding-batch-limit* l))
11 nconc (funcall *batch-request-fn*
12 (subseq batch 0
13 (min *embedding-batch-limit* (length batch))))))
Two public functions round out the interface. get-embedding checks the cache before calling the API, and get-embeddings fetches all cache misses for a list of texts with batched batchEmbedContents calls, turning N HTTP requests into a handful when a document is first loaded:
1 (defun get-embedding (text)
2 "Compute (or retrieve from cache) an embedding vector for TEXT.
3 Returns a simple-vector of floats."
4 (let ((key (embedding-cache-key text)))
5 (multiple-value-bind (cached present-p) (gethash key *embedding-cache*)
6 (if present-p
7 (progn
8 (%debug-log "~%DEBUG get-embedding: cache hit for ~S~%"
9 (subseq text 0 (min 60 (length text))))
10 cached)
11 (let ((vec (coerce (funcall *embedding-fn* text) 'simple-vector)))
12 (%cache-put text vec)
13 (%debug-log "~%DEBUG get-embedding: got ~A-dimensional vector for ~S~%"
14 (length vec) (subseq text 0 (min 60 (length text))))
15 vec)))))
We also define cosine-similarity to compare two embedding vectors; this is how we determine which document chunks are most relevant to a query:
1 (defun dot-product (vec-a vec-b)
2 "Compute the dot product of two equal-length vectors of floats.
3 Signals an error on length mismatch: silently truncating would hide
4 a model/dimension change and corrupt similarity scores."
5 (let ((a (coerce vec-a 'simple-vector))
6 (b (coerce vec-b 'simple-vector)))
7 (unless (= (length a) (length b))
8 (error "Embedding dimension mismatch: ~A vs ~A (did *embedding-model* ~
9 or *embedding-dimension* change after the corpus was built?)"
10 (length a) (length b)))
11 (loop for x across a
12 for y across b
13 sum (* x y))))
14
15 (defun vector-magnitude (vec)
16 "Compute the magnitude (L2 norm) of a vector of floats."
17 (let ((v (coerce vec 'simple-vector)))
18 (sqrt (loop for x across v sum (* x x)))))
19
20 (defun cosine-similarity (vec-a vec-b)
21 "Compute cosine similarity between two embedding vectors.
22 Returns a value between -1 and 1."
23 (let ((mag-a (vector-magnitude vec-a))
24 (mag-b (vector-magnitude vec-b)))
25 (if (or (zerop mag-a) (zerop mag-b))
26 0.0
27 (/ (dot-product vec-a vec-b) (* mag-a mag-b)))))
The dimension check in dot-product earns its keep. loop for x across a for y across b stops at the shorter vector, so before this check a 3072-value chunk and a 1536-value query (say, after switching *embedding-dimension*) would silently score against a truncated vector instead of failing. A wrong-but-plausible score is worse than an error because nobody notices it.
The embedding API returns a JSON response containing a list of floating-point values. The cosine similarity between two vectors measures how similar their directions are in the high-dimensional embedding space, regardless of magnitude. A similarity of 1.0 means the texts are semantically identical; 0.0 means they are unrelated.
In-Memory Vector Store
The file vector-store.lisp implements a simple in-memory document store. Production systems would use a dedicated vector database like Pinecone or Chroma, but for a book example, an in-memory list with brute-force cosine similarity is clearer and requires zero setup.
We define two structs: document-chunk holds a piece of text with its source filename, embedding vector, and precomputed norm; and corpus is a named collection of chunks:
1 (defstruct (document-chunk (:print-function %print-document-chunk))
2 "A chunk of text with its source file, embedding vector (a normalized
3 simple-vector), and precomputed L2 norm (1.0 for normalized chunks)."
4 text
5 source
6 embedding
7 (norm 1.0 :type float))
8
9 (defstruct (corpus (:print-function %print-corpus))
10 "A named collection of document chunks for retrieval."
11 name
12 description
13 (chunks nil))
A struct with 3072 floats per chunk is painful to inspect at the REPL: printing a corpus dumps thousands of numbers per chunk and swamps the terminal. Both structs therefore install custom print functions. A chunk prints its text and source in full but only the first 10 embedding values:
1 #<DOCUMENT-CHUNK :SOURCE "renewable-energy.txt" :TEXT "Renewable Energy
2 Sources and Technologies ..." :EMBEDDING #(3.2552084e-4 6.510417e-4
3 9.765625e-4 0.0013020834 0.0016276041 0.001953125 0.0022786458
4 0.0026041667 0.0029296875 0.0032552083 ...) [3072 dimensions]>
and a corpus prints just its name, description, and chunk count:
1 (#<CORPUS :NAME "renewable-energy" :DESCRIPTION "Renewable energy sources
2 and technologies" :CHUNKS 9>
3 #<CORPUS :NAME "electric-vehicles" :DESCRIPTION "Electric vehicle
4 technology and infrastructure" :CHUNKS 7>
5 #<CORPUS :NAME "climate-science" :DESCRIPTION "Climate science and carbon
6 emissions" :CHUNKS 7>)
The printers emit unreadable #<...> forms on purpose. A readable #S(...) form with a truncated embedding could be read back into a program as a chunk whose vector is only 10 values long; the #< prefix makes clear the printed form is for humans. Chunks are still reachable through document-chunk-embedding, and save-corpus remains the way to write them to disk.
The norm slot and the custom printer live in the same struct for related reasons: both are about treating embeddings as opaque bulk data. Normalizing each chunk embedding once, when the document is added, means search never recomputes a chunk norm; cosine similarity against a normalized chunk is just a dot product divided by the query’s norm. make-document-chunk/embedded is the constructor that does the work:
1 (defun make-document-chunk/embedded (text source raw-embedding)
2 "Build a document-chunk with a normalized simple-vector embedding and
3 its precomputed norm. All chunks in the store are normalized, so
4 search is a dot product (see search-corpus)."
5 (let* ((vec (coerce raw-embedding 'simple-vector))
6 (norm (vector-magnitude vec)))
7 (when (zerop norm)
8 (error "Zero-magnitude embedding for chunk from ~A; cannot normalize" source))
9 (make-document-chunk :text text :source source :embedding vec :norm norm)))
Each chunk also gets a stable identity, used for deduplication later:
1 (defun document-chunk-key (chunk)
2 "Stable identity for a chunk across corpora: (source . text). The
3 same text in different source files is a different chunk."
4 (cons (document-chunk-source chunk) (document-chunk-text chunk)))
The function split-into-chunks breaks a long text into overlapping pieces of approximately 500 characters each, trying to break at sentence boundaries (periods or newlines) rather than cutting words in half:
1 (defparameter *default-chunk-size* 500
2 "Default size in characters for splitting documents into chunks.")
3
4 (defparameter *chunk-overlap* 50
5 "Number of characters to overlap between adjacent chunks.")
6
7 (defun split-into-chunks (text &key (chunk-size *default-chunk-size*)
8 (overlap *chunk-overlap*))
9 "Split TEXT into overlapping chunks of approximately CHUNK-SIZE characters.
10 Tries to break at sentence boundaries when possible."
11 (let ((chunks nil)
12 (len (length text))
13 (start 0))
14 (loop while (< start len)
15 do (let* ((end (min (+ start chunk-size) len))
16 ;; Try to find a sentence boundary at or before END
17 ;; (searching the window up to 80 chars back from END)
18 (break-pos
19 (if (>= end len)
20 end
21 (or (position #\. text :start (max start (- end 80))
22 :end end :from-end t)
23 (position #\Newline text :start (max start (- end 80))
24 :end end :from-end t)
25 end)))
26 ;; Advance past the break character
27 (actual-end (if (< break-pos end)
28 (1+ break-pos)
29 end)))
30 ;; Guarantee forward progress: if the break search left us
31 ;; at or before START, fall back to a hard cut at CHUNK-SIZE.
32 ;; Without this guard a chunk could be empty or START could
33 ;; fail to advance (looping forever or dropping text).
34 (when (<= actual-end start)
35 (setf actual-end (min (+ start chunk-size) len)))
36 (let ((chunk (string-trim '(#\Space #\Newline #\Tab)
37 (subseq text start actual-end))))
38 (when (> (length chunk) 0)
39 (push chunk chunks)))
40 ;; Never move START backwards: the overlap backtrack must
41 ;; not undo progress made by the forward-progress guard.
42 (setf start (if (>= actual-end len)
43 len
44 (max (1+ start) (- actual-end overlap))))))
45 (nreverse chunks)))
The overlap between chunks (defaulting to 50 characters) ensures that information at chunk boundaries is not lost; a sentence that spans two chunks will appear in both. Note the two progress guards: without them, a document whose only sentence break lands at or before the start of the current window could produce an empty chunk or move start backwards, looping forever. Loops that compute their next position from searched positions always need an explicit monotonic-progress check.
The function add-document reads a file, chunks it, and computes embeddings for all chunks at once. When the default embedding function is in use, get-embeddings makes batched API calls for the whole document instead of one request per chunk:
1 (defun add-document (corpus filepath &key (chunk-size *default-chunk-size*))
2 "Read a text file, split it into chunks, compute embeddings (in
3 batched API calls when the default embedding function is used),
4 and add the chunks to CORPUS. Returns the number of chunks added."
5 (%debug-log "~%DEBUG add-document: loading ~A~%" filepath)
6 (let* ((text (uiop:read-file-string filepath))
7 (chunks (split-into-chunks text :chunk-size chunk-size))
8 (source (file-namestring filepath)))
9 (%debug-log "DEBUG add-document: split into ~A chunks~%" (length chunks))
10 (setf (corpus-chunks corpus)
11 (nconc (corpus-chunks corpus)
12 (loop for chunk-text in chunks
13 for embedding in (get-embeddings chunks)
14 collect (make-document-chunk/embedded chunk-text
15 source
16 embedding))))
17 (%debug-log "DEBUG add-document: added ~A chunks from ~A~%"
18 (length chunks) source)
19 (length chunks)))
Because embedding a corpus costs API calls, save-corpus and load-corpus persist a corpus, embeddings included, as a plain s-expression file. Loading is the risky half: a truncated or corrupt file would otherwise produce chunks with NIL embeddings that fail far from the cause, or worse, score quietly wrong. So load-corpus validates the overall shape and every chunk before trusting it:
1 (defun %valid-chunk-data-p (chunk-data)
2 "True when one saved chunk plist has non-empty TEXT and SOURCE and an
3 EMBEDDING that is a non-empty sequence of numbers."
4 (and (consp chunk-data)
5 (stringp (getf chunk-data :text))
6 (plusp (length (getf chunk-data :text)))
7 (stringp (getf chunk-data :source))
8 (plusp (length (getf chunk-data :source)))
9 (let ((emb (getf chunk-data :embedding)))
10 (and (typep emb 'sequence)
11 (plusp (length emb))
12 (every #'numberp emb)))))
13
14 (defun load-corpus (pathname)
15 "Load a corpus previously written by SAVE-CORPUS. Returns a corpus struct.
16 Signals an error when the file is truncated, corrupt, or contains a
17 chunk missing its text, source, or embedding."
18 (with-open-file (in pathname :direction :input)
19 (with-standard-io-syntax
20 (let* ((*read-eval* nil) ; never evaluate while reading data files
21 (data (read in)))
22 (unless (and (consp data)
23 (getf data :name)
24 (listp (getf data :chunks))
25 (getf data :chunks))
26 (error "Corrupt corpus file ~A: expected (:name ...) (:chunks ...)" pathname))
27 (let ((corpus (make-corpus :name (getf data :name)
28 :description (getf data :description))))
29 (setf (corpus-chunks corpus)
30 (mapcar (lambda (chunk-data)
31 (unless (%valid-chunk-data-p chunk-data)
32 (error "Corrupt chunk in corpus file ~A: ~S"
33 pathname chunk-data))
34 ;; Re-normalize on load: files saved by older
35 ;; versions may hold un-normalized vectors.
36 (make-document-chunk/embedded
37 (getf chunk-data :text)
38 (getf chunk-data :source)
39 (getf chunk-data :embedding)))
40 (getf data :chunks)))
41 corpus)))))
Note the *read-eval* binding when loading: read on an untrusted file must never be allowed to evaluate embedded forms.
The search-corpus and search-corpora functions find the top-K most similar chunks for a given query embedding:
1 (defun score-chunks (chunks query-embedding &key (query-norm 1.0))
2 "Score CHUNKS against QUERY-EMBEDDING. Chunks are stored normalized,
3 so cosine similarity is the dot product divided by the query norm;
4 each chunk's norm is not recomputed."
5 (loop for chunk in chunks
6 collect (cons (/ (dot-product query-embedding
7 (document-chunk-embedding chunk))
8 query-norm)
9 chunk)))
10
11 (defun %top-k-by-score (scored-chunks top-k)
12 "Return the TOP-K entries of SCORED-CHUNKS (sorted by descending car)
13 using a single O(n) selection pass instead of a full O(n log n) sort."
14 (let ((k (min top-k (length scored-chunks))))
15 (when (plusp k)
16 ;; Repeatedly extract the max: k passes, each O(n). Worst case
17 ;; k = n is O(n^2), but k is small (3 by default), so this beats
18 ;; sorting at demo scale and stays O(n) for constant k.
19 (let ((remaining (copy-list scored-chunks))
20 (result nil))
21 (dotimes (i k)
22 (let ((best (loop for entry in remaining
23 maximize (car entry))))
24 (let ((winner (find best remaining :key #'car)))
25 (push winner result)
26 (setf remaining (remove winner remaining :count 1)))))
27 (nreverse result)))))
28
29 (defun search-corpora (corpora query-embedding &key (top-k 3))
30 "Search multiple CORPORA for the TOP-K most similar chunks overall.
31 Returns a list of (score . document-chunk) pairs."
32 (let* ((query (coerce query-embedding 'simple-vector))
33 (query-norm (vector-magnitude query))
34 (all-results
35 (loop for corpus in corpora
36 nconc (%top-k-by-score
37 (score-chunks (corpus-chunks corpus)
38 query
39 :query-norm query-norm)
40 top-k))))
41 (%top-k-by-score all-results top-k)))
Scoring exploits the normalization done at add time: with every chunk at unit length, the cosine similarity between a chunk and the query is the dot product divided by the query’s norm, computed once per query instead of once per chunk. And instead of sorting all n scores to take the top 3, %top-k-by-score extracts the maximum k times, which is O(kn); with small k that beats an O(n log n) sort. Neither optimization matters at 23 chunks, but they keep the search loop harmless at thousands of chunks, and they fall out of the normalized representation naturally.
An important feature for agentic RAG is that search-corpora accepts a list of corpora, enabling cross-corpus retrieval. The Google research article emphasizes this capability: real-world knowledge is often spread across separate databases managed by different teams. Our system searches all corpora simultaneously and returns the best results regardless of source.
The Multi-Agent Pipeline
The file agents.lisp is the heart of the system. Each “agent” is a function that calls Gemini with a specialized prompt. This is a practical and effective pattern: we don’t need an external agent framework to implement agent behaviors, just well-crafted prompts and structured response parsing.
We use gemini-3-flash-preview for all agent calls. This model is very inexpensive while being capable enough for query rewriting, sufficiency assessment, and synthesis. The function rag-generate delegates to gemini:generate from the llm library, going through the special variable *generate-fn* so tests can substitute a stub (the same idiom as *embedding-fn* above). It also wraps the call in call-with-retries, so a transient 500 during assessment or synthesis does not throw away the whole pipeline’s work:
1 (defparameter *rag-model* "gemini-3-flash-preview"
2 "Gemini model used for all agent LLM calls. Override per call with
3 the :model keyword argument to agentic-rag.")
4
5 (defparameter *generate-fn*
6 (lambda (prompt &key (model *rag-model*))
7 (gemini:generate prompt :model-id model))
8 "Function of (prompt &key model) returning generated text. Defaults
9 to a thin wrapper around gemini:generate from the llm library.
10 Rebind this in tests to run the pipeline without network access.")
11
12 (defun rag-generate (prompt &key (model *rag-model*))
13 "Call the LLM through *generate-fn* with retries on transient
14 failures (HTTP 429/5xx, connection errors), so one flaky request
15 does not throw away the whole pipeline's work. Signals an error if
16 the model returns no text."
17 (or (call-with-retries
18 (lambda () (funcall *generate-fn* prompt :model model)))
19 (error "LLM returned no text for prompt: ~A"
20 (subseq prompt 0 (min 80 (length prompt))))))
Agent 1: The Query Rewriter
The Query Rewriter takes a complex user question and decomposes it into 1–3 focused sub-queries. For example, the question “How does the carbon footprint of manufacturing EV batteries compare to the emissions saved by charging EVs from renewable energy?” would be split into sub-queries like:
- “carbon footprint of EV battery manufacturing”
- “emissions saved by charging electric vehicles from renewable energy”
This decomposition improves retrieval because each sub-query targets a specific fact that might appear in a different document or section.
Parsing the model’s response deserves care. The prompt says “no numbering, bullets, or extra text”, but models drift, and a first implementation that trimmed the characters - * 1 2 3 . off both ends of each line mangled legitimate queries: “2024 lithium battery prices” became “024 lithium battery prices”, and “1.5 MW turbine output” became “5 MW turbine output”. Queries are exactly the kind of text with leading digits and decimal points. The fix strips only a leading list prefix: an optional bullet character, or digits followed by . or ) followed by whitespace. The whitespace test is what distinguishes “1. fourth query” from “1.5 MW output”:
1 (defun %strip-list-prefix (line)
2 "Remove an optional markdown/numbered list prefix from LINE and the
3 surrounding whitespace. Only leading list syntax is stripped:
4 interior and trailing digits are part of the query (so
5 \"2024 lithium prices\", \"1.5 MW output\", and \"75-100 kg\" survive
6 intact, while \"4. fourth query\" and \"- bullet\" are cleaned)."
7 (flet ((ws-p (c) (member c '(#\Space #\Tab #\Return #\Newline))))
8 (let* ((len (length line))
9 (i 0))
10 ;; skip leading whitespace
11 (loop while (and (< i len) (ws-p (char line i))) do (incf i))
12 ;; skip an optional bullet character
13 (when (and (< i len) (member (char line i) '(#\- #\* #\+)))
14 (incf i)
15 (loop while (and (< i len) (ws-p (char line i))) do (incf i)))
16 ;; skip an optional numbering: digits followed by . or ) followed
17 ;; by whitespace. "1.5 MW" fails the whitespace test, so it stays.
18 (let ((j i))
19 (loop while (and (< j len) (digit-char-p (char line j))) do (incf j))
20 (when (and (> j i) (< j len)
21 (member (char line j) '(#\. #\)))
22 (< (1+ j) len)
23 (ws-p (char line (1+ j))))
24 (setf i (1+ j))
25 (loop while (and (< i len) (ws-p (char line i))) do (incf i))))
26 (string-trim '(#\Space #\Tab #\Return #\Newline) (subseq line i)))))
27
28 (defun parse-query-lines (response)
29 "Extract one query per line from a rewriter agent RESPONSE, dropping
30 empty lines and list prefixes. Query text itself is untouched."
31 (remove-if (lambda (s) (zerop (length s)))
32 (mapcar #'%strip-list-prefix
33 (uiop:split-string (or response "")
34 :separator '(#\Newline)))))
35
36 (defun rewrite-queries (user-query &key (model *rag-model*))
37 "Decompose USER-QUERY into 1-3 focused sub-queries for retrieval.
38 Returns a list of query strings. The original query is always
39 appended as a fallback so the fanout always searches for what the
40 user actually asked."
41 (%debug-log "~%DEBUG rewrite-queries: decomposing query...~%")
42 (let* ((prompt
43 (format nil
44 "You are a search query rewriter for a RAG system. ~
45 Your job is to break a complex user question into ~
46 1-3 simple, focused search queries that will help ~
47 retrieve relevant information from a document collection.~%~
48 ~%Rules:~
49 ~%- Output ONLY the queries, one per line~
50 ~%- No numbering, bullets, or extra text~
51 ~%- Each query should target a specific fact or concept~
52 ~%- Keep queries concise (under 15 words each)~
53 ~%~%User question: ~A" user-query))
54 (queries (parse-query-lines (rag-generate prompt :model model))))
55 (%debug-log "DEBUG rewrite-queries: generated ~A sub-queries:~%~{ - ~A~%~}"
56 (length queries) queries)
57 (remove-duplicates (append queries (list user-query)) :test #'equal)))
Appending the original query is deliberate. The rewriter’s sub-queries aim at the pieces of a question, but the user’s exact phrasing often matches the document’s phrasing best; searching it directly is one more vector in the fanout and costs one more row in the batch embedding call.
Agent 2: Search Fanout
The Search Fanout agent executes the sub-queries against all corpora. All sub-query embeddings are fetched with one call to get-embeddings (one batched API request for the whole list) instead of one round trip per query. Results are deduplicated by document-chunk-key (source and text together), so the same passage matched by several queries appears once, but identical text from two different files stays as two results:
1 (defun search-fanout (corpora sub-queries &key (top-k 3))
2 "Execute embedding search across CORPORA for each sub-query.
3 All sub-query embeddings are fetched with one batched API call.
4 Returns a deduplicated list of (score . document-chunk) pairs,
5 sorted by descending score. Chunks are deduplicated by
6 (source . text) so identical text in different files stays distinct."
7 (%debug-log "~%DEBUG search-fanout: searching ~A corpora with ~A queries~%"
8 (length corpora) (length sub-queries))
9 ;; One batched embedding call for all sub-queries instead of N round trips
10 (let ((query-embeddings (get-embeddings sub-queries))
11 (all-results nil)
12 (seen-keys (make-hash-table :test 'equal)))
13 (mapc (lambda (query query-embedding)
14 (%debug-log "DEBUG search-fanout: searching with: ~S~%" query)
15 (dolist (result (search-corpora corpora query-embedding
16 :top-k top-k))
17 (let ((key (document-chunk-key (cdr result))))
18 (unless (gethash key seen-keys)
19 (setf (gethash key seen-keys) t)
20 (push result all-results)))))
21 sub-queries query-embeddings)
22 ;; Sort by score descending
23 (let ((sorted (sort all-results #'> :key #'car)))
24 (%debug-log "DEBUG search-fanout: found ~A unique chunks~%" (length sorted))
25 sorted)))
Agent 3: The Sufficient Context Agent
This is the key innovation from the Google research. After retrieval, the Sufficient Context Agent evaluates whether the passages actually contain enough information. It asks Gemini to produce a structured verdict: SUFFICIENT or INSUFFICIENT, with a reason and a description of what’s missing.
The structured output format (VERDICT/REASON/MISSING) makes it straightforward to parse the LLM’s response programmatically. The parsing is factored into its own function, parse-verdict-response, partly so the logic is unit-testable without an API key, and partly so the fallback policy lives in exactly one place: a verdict we cannot parse is treated as SUFFICIENT, because the iteration limit is the only other thing bounding API cost:
1 (defun parse-verdict-response (response)
2 "Parse a Sufficient Context Agent RESPONSE of the form:
3 VERDICT: SUFFICIENT | INSUFFICIENT
4 REASON: ...
5 MISSING: ...
6 Returns two values: SUFFICIENT-P and FEEDBACK (the MISSING text).
7 An unparseable verdict is treated as SUFFICIENT — this bounds API
8 cost because the iteration limit is the only other safeguard."
9 (let* ((lines (uiop:split-string (or response "") :separator '(#\Newline)))
10 (verdict-line (find-if (lambda (line)
11 (search "VERDICT:" line :test #'char-equal))
12 lines))
13 (missing-line (find-if (lambda (line)
14 (search "MISSING:" line :test #'char-equal))
15 lines))
16 (verdict-word (when verdict-line
17 (string-trim
18 '(#\Space #\Tab #\.)
19 (subseq verdict-line
20 (+ (search "VERDICT:" verdict-line
21 :test #'char-equal)
22 8)))))
23 (feedback (if missing-line
24 (string-trim
25 '(#\Space #\Tab)
26 (subseq missing-line
27 (+ (search "MISSING:" missing-line
28 :test #'char-equal)
29 8)))
30 "No specific feedback available")))
31 (cond ((and verdict-word (search "INSUFFICIENT" verdict-word :test #'char-equal))
32 (values nil feedback))
33 ((and verdict-word (search "SUFFICIENT" verdict-word :test #'char-equal))
34 (values t feedback))
35 (t
36 (%debug-log "WARNING parse-verdict-response: unparseable verdict ~S; ~
37 treating as SUFFICIENT~%" verdict-word)
38 (values t feedback)))))
Note the order of the two cond clauses: the string “INSUFFICIENT” contains “SUFFICIENT” as a substring, so we must test for the longer word first. The agent function itself builds the prompt, calls the model, and delegates interpretation to the parser:
1 (defun assess-sufficiency (user-query retrieved-chunks &key (model *rag-model*))
2 "Evaluate whether RETRIEVED-CHUNKS provide sufficient context
3 to answer USER-QUERY. Returns two values:
4 1. SUFFICIENT-P — T if context is sufficient, NIL otherwise
5 2. FEEDBACK — String describing what information is missing."
6 (%debug-log "~%DEBUG assess-sufficiency: evaluating ~A chunks~%"
7 (length retrieved-chunks))
8 (let* ((context (format-retrieved-chunks retrieved-chunks))
9 (prompt
10 (format nil
11 "You are a Sufficient Context Agent in an agentic RAG system. ~
12 Your role is to evaluate whether the retrieved passages ~
13 contain enough information to fully answer the user's question.~%~
14 ~%User Question: ~A~%~
15 ~%Retrieved Passages:~A~%~
16 ~%Evaluate carefully:~
17 ~%1. Does the context contain ALL the specific facts needed?~
18 ~%2. Are there any parts of the question left unanswered?~
19 ~%3. Is any critical information missing?~
20 ~%~%Respond in EXACTLY this format:~
21 ~%VERDICT: SUFFICIENT or INSUFFICIENT~
22 ~%REASON: (one sentence explaining your assessment)~
23 ~%MISSING: (if insufficient, describe what specific ~
24 information to search for next; if sufficient, write NONE)"
25 user-query context))
26 (response (rag-generate prompt :model model)))
27 (%debug-log "DEBUG assess-sufficiency response:~%~A~%" response)
28 (multiple-value-bind (sufficient-p feedback)
29 (parse-verdict-response response)
30 (%debug-log "DEBUG assess-sufficiency: verdict=~A~%"
31 (if sufficient-p "SUFFICIENT" "INSUFFICIENT"))
32 (values sufficient-p feedback))))
The two return values, sufficient-p (a boolean) and feedback (a string describing what’s missing), drive the orchestrator’s decision to either synthesize an answer or refine the search.
Agent 4: The Synthesis Agent
When the context is deemed sufficient, the Synthesis Agent generates the final answer. It is instructed to use only the retrieved passages and to cite source filenames:
1 (defun synthesize-answer (user-query retrieved-chunks &key (model *rag-model*))
2 "Generate a grounded answer to USER-QUERY using RETRIEVED-CHUNKS.
3 The answer cites source documents."
4 (%debug-log "~%DEBUG synthesize-answer: generating answer from ~A chunks~%"
5 (length retrieved-chunks))
6 (let* ((context (format-retrieved-chunks retrieved-chunks))
7 (prompt
8 (format nil
9 "You are a Synthesis Agent in a RAG system. Generate a ~
10 clear, accurate answer to the user's question using ONLY ~
11 the information in the retrieved passages below. ~
12 ~%~%Rules:~
13 ~%- Base your answer strictly on the retrieved passages~
14 ~%- Cite sources by mentioning the source filename~
15 ~%- If the passages don't fully answer the question, ~
16 say what you can answer and note what's missing~
17 ~%- Be concise but thorough~
18 ~%~%User Question: ~A~
19 ~%~%Retrieved Passages:~A"
20 user-query context))
21 (response (rag-generate prompt :model model)))
22 (%debug-log "DEBUG synthesize-answer: generated response (~A chars)~%"
23 (length response))
24 response))
The Orchestrator
The agentic-rag function ties everything together. It runs the full pipeline, iterating when the Sufficient Context Agent determines the retrieved passages are incomplete:
1 (defun agentic-rag (corpora user-query &key (max-iterations 3)
2 (top-k 3)
3 (model *rag-model*)
4 (max-context-chunks 8))
5 "Run the full agentic RAG pipeline:
6 1. Rewrite the user query into sub-queries
7 2. Search corpora for relevant chunks
8 3. Check if context is sufficient (loop if not)
9 4. Synthesize a grounded answer
10
11 CORPORA is a list of corpus structs.
12 MODEL is the Gemini model id used for every agent call.
13 MAX-CONTEXT-CHUNKS caps how many retrieved passages are sent to the
14 LLM (highest-scoring first) no matter how many iterations ran.
15 Returns the synthesized answer string."
16 (%debug-log "~%~%========================================~%")
17 (%debug-log " AGENTIC RAG PIPELINE~%")
18 (%debug-log " Query: ~A~%" user-query)
19 (%debug-log "========================================~%")
20
21 ;; Phase 1: Rewrite queries
22 (let* ((sub-queries (rewrite-queries user-query :model model))
23 ;; Phase 2: Initial search
24 (all-chunks (search-fanout corpora sub-queries :top-k top-k))
25 (iteration 0))
26
27 ;; Phase 3: Iterative sufficiency check
28 (loop
29 (incf iteration)
30 (%debug-log "~%--- Iteration ~A/~A ---~%" iteration max-iterations)
31
32 (when (null all-chunks)
33 (%debug-log "DEBUG agentic-rag: no chunks found, returning empty answer~%")
34 (return-from agentic-rag
35 "I could not find any relevant information in the available documents."))
36
37 ;; Cap prompt size regardless of how many iterations accumulated chunks
38 (let ((context-chunks (%cap-context all-chunks max-context-chunks)))
39
40 ;; At the last allowed iteration both branches end in "synthesize
41 ;; with what we have", so skip the sufficiency LLM call entirely.
42 (when (>= iteration max-iterations)
43 (%debug-log "~%DEBUG agentic-rag: max iterations reached, synthesizing ~
44 with available context~%")
45 (return-from agentic-rag
46 (synthesize-answer user-query context-chunks :model model)))
47
48 (multiple-value-bind (sufficient-p feedback)
49 (assess-sufficiency user-query context-chunks :model model)
50
51 (when sufficient-p
52 (%debug-log "~%DEBUG agentic-rag: context is SUFFICIENT at iteration ~A~%"
53 iteration)
54 ;; Phase 5: Synthesize answer
55 (return-from agentic-rag
56 (synthesize-answer user-query context-chunks :model model)))
57
58 ;; Phase 4: Refine and search again
59 (%debug-log "~%DEBUG agentic-rag: context INSUFFICIENT, refining...~%")
60 (%debug-log "DEBUG agentic-rag: feedback: ~A~%" feedback)
61 (let* ((refined-queries (refine-queries user-query feedback
62 :model model))
63 (new-chunks (search-fanout corpora refined-queries
64 :top-k top-k)))
65 ;; Accumulate new chunks with existing ones (deduplicate by
66 ;; (source . text) so identical text from different files
67 ;; stays distinct)
68 (let ((seen (make-hash-table :test 'equal)))
69 (dolist (scored-chunk all-chunks)
70 (setf (gethash (document-chunk-key (cdr scored-chunk)) seen) t))
71 (dolist (scored-chunk new-chunks)
72 (unless (gethash (document-chunk-key (cdr scored-chunk)) seen)
73 (setf (gethash (document-chunk-key (cdr scored-chunk)) seen) t)
74 (push scored-chunk all-chunks))))
75 ;; Re-sort by score
76 (setf all-chunks (sort all-chunks #'> :key #'car))))))))
Two details of the loop are worth pointing out. First, all progress output goes through %debug-log, so binding *rag-verbose* to NIL silences the entire pipeline, banner and iterations included; the orchestrator prints nothing on its own authority. Second, the max-iterations check runs before assess-sufficiency. At the final iteration, both a SUFFICIENT and an INSUFFICIENT verdict end in “synthesize with what we have”, so the assessment cannot change the outcome; asking the model anyway spends a call to learn nothing. With max-iterations 3 and a query that never converges, the pipeline makes two assessment calls, not three.
Notice how each iteration accumulates new chunks with the existing ones, deduplicating by (source . text). The accumulated context grows richer with each iteration, increasing the likelihood that the Sufficient Context Agent will be satisfied. The max-context-chunks keyword caps how many top-scoring passages are actually sent to the model, so a long refinement loop cannot grow the prompt without bound.
Top-Level API and Demo
The file rag.lisp provides convenience functions and a built-in demo. The test function creates three separate corpora (renewable energy, electric vehicles, and climate science) and runs three progressively harder queries:
1 (defun test ()
2 "Run a demo of the Agentic RAG system with sample documents.
3 Creates three corpora (energy, vehicles, climate) and runs
4 multi-hop queries that require cross-corpus retrieval."
5 (format t "~%~%============================================~%")
6 (format t " Agentic RAG Demo — Loading Documents~%")
7 (format t "============================================~%")
8
9 ;; Create three separate corpora to demonstrate cross-corpus retrieval
10 (let ((energy-corpus (make-corpus :name "renewable-energy"
11 :description "Renewable energy sources and technologies"))
12 (ev-corpus (make-corpus :name "electric-vehicles"
13 :description "Electric vehicle technology and infrastructure"))
14 (climate-corpus (make-corpus :name "climate-science"
15 :description "Climate science and carbon emissions")))
16
17 ;; Load documents into their respective corpora
18 (add-document energy-corpus (data-path "renewable-energy.txt"))
19 (add-document ev-corpus (data-path "electric-vehicles.txt"))
20 (add-document climate-corpus (data-path "climate-science.txt"))
21
22 (let ((all-corpora (list energy-corpus ev-corpus climate-corpus)))
23 (format t "~%~%Loaded ~A total chunks across ~A corpora.~%"
24 (loop for c in all-corpora sum (corpus-chunk-count c))
25 (length all-corpora))
26
27 ;; Query 1: Single-corpus question (should find answer easily)
28 (format t "~%~%===== TEST QUERY 1 (single topic) =====~%")
29 (let ((answer (query all-corpora
30 "What is the current cost of lithium-ion battery storage per kilowatt-hour?")))
31 (format t "~%~%ANSWER 1:~%~A~%~%" answer))
32
33 ;; Query 2: Multi-hop question requiring cross-corpus retrieval
34 (format t "~%~%===== TEST QUERY 2 (multi-hop, cross-corpus) =====~%")
35 (let ((answer (query all-corpora
36 "How does the carbon footprint of manufacturing EV batteries compare to the emissions saved by charging EVs from renewable energy sources?")))
37 (format t "~%~%ANSWER 2:~%~A~%~%" answer))
38
39 ;; Query 3: Complex question that may need iterative retrieval
40 (format t "~%~%===== TEST QUERY 3 (complex, iterative) =====~%")
41 (let ((answer (query all-corpora
42 "What role could solid-state batteries and pumped-storage hydroelectricity play together in solving the intermittency problem of wind and solar energy?")))
43 (format t "~%~%ANSWER 3:~%~A~%~%" answer))
44
45 (format t "~%~%============================================~%")
46 (format t " Demo Complete~%")
47 (format t "============================================~%")
48
49 ;; Return corpora for interactive use
50 all-corpora)))
The test queries are designed to demonstrate different capabilities:
- Query 1 is a simple factual lookup: the answer exists in a single document chunk.
- Query 2 requires combining information from the electric vehicles corpus (battery manufacturing emissions) with the climate science corpus (emissions data), demonstrating cross-corpus retrieval. The documents do not actually contain the break-even data the question asks for, so this query also shows the refinement loop running to exhaustion and the synthesis agent reporting what it can and cannot answer.
- Query 3 combines solid-state batteries (EV corpus) with pumped-storage hydro and hybrid storage (renewable energy corpus). Here the initial retrieval already finds the hybrid-storage passage, and the Sufficient Context Agent accepts on the first iteration.
Running the Example
Load the system and run the demo:
1 $ sbcl
2 * (load "project.lisp")
3
4 --- rag project loaded ---
5
6 * (rag:test)
7
8 ============================================
9 Agentic RAG Demo — Loading Documents
10 ============================================
11
12 DEBUG add-document: loading .../data/renewable-energy.txt
13 DEBUG add-document: split into 9 chunks
14
15 DEBUG get-embeddings: batch-fetching 9 embeddings
16 DEBUG add-document: added 9 chunks from renewable-energy.txt
17
18 DEBUG add-document: loading .../data/electric-vehicles.txt
19 DEBUG add-document: split into 7 chunks
20
21 DEBUG get-embeddings: batch-fetching 7 embeddings
22 DEBUG add-document: added 7 chunks from electric-vehicles.txt
23
24 DEBUG add-document: loading .../data/climate-science.txt
25 DEBUG add-document: split into 7 chunks
26
27 DEBUG get-embeddings: batch-fetching 7 embeddings
28 DEBUG add-document: added 7 chunks from climate-science.txt
29
30 Loaded 23 total chunks across 3 corpora.
31
32
33 ===== TEST QUERY 1 (single topic) =====
34
35 ========================================
36 AGENTIC RAG PIPELINE
37 Query: What is the current cost of lithium-ion battery storage
38 per kilowatt-hour?
39 ========================================
40
41 DEBUG rewrite-queries: decomposing query...
42 DEBUG rewrite-queries: generated 3 sub-queries:
43 - lithium-ion battery storage cost per kWh 2024
44 - recent trends in lithium-ion battery pack prices per kilowatt-hour
45 - average cost per kWh for utility-scale lithium-ion batteries
46
47 DEBUG search-fanout: searching 3 corpora with 4 queries
48
49 DEBUG get-embeddings: batch-fetching 4 embeddings
50 DEBUG search-fanout: searching with: "lithium-ion battery storage cost per kWh 2024"
51 DEBUG search-fanout: searching with: "recent trends in lithium-ion battery pack prices per kilowatt-hour"
52 DEBUG search-fanout: searching with: "average cost per kWh for utility-scale lithium-ion batteries"
53 DEBUG search-fanout: searching with: "What is the current cost of lithium-ion battery storage per kilowatt-hour?"
54 DEBUG search-fanout: found 5 unique chunks
55
56 --- Iteration 1/3 ---
57
58 DEBUG assess-sufficiency: evaluating 5 chunks
59 DEBUG assess-sufficiency response:
60 VERDICT: SUFFICIENT
61 REASON: The first retrieved passage provides a specific current cost
62 figure, stating that the price of lithium-ion battery storage has
63 fallen to under $140 per kilowatt-hour.
64 MISSING: NONE
65 DEBUG assess-sufficiency: verdict=SUFFICIENT
66
67 DEBUG agentic-rag: context is SUFFICIENT at iteration 1
68
69 DEBUG synthesize-answer: generating answer from 5 chunks
70
71
72 ANSWER 1:
73 Based on the provided passages, the current cost of lithium-ion
74 battery storage is under $140 per kilowatt-hour (source:
75 renewable-energy.txt). The cost has fallen by approximately 90% since
76 2010, when prices were over $1,100 per kilowatt-hour (source:
77 renewable-energy.txt).
Two things are worth noticing in this run. The rewriter produced three sub-queries, but the fanout searched with four: the original question is appended to the sub-query list, and all four embeddings are fetched with one batched call (“batch-fetching 4 embeddings”). Each document load is likewise one batched request.
Query 2 is the interesting case for the refinement loop. The question asks for a comparison the documents cannot fully support, so the Sufficient Context Agent keeps finding the gap and the loop runs to the iteration limit:
1 ===== TEST QUERY 2 (multi-hop, cross-corpus) =====
2
3 --- Iteration 1/3 ---
4
5 DEBUG assess-sufficiency response:
6 VERDICT: INSUFFICIENT
7 REASON: While the passages provide the carbon footprint for battery
8 manufacturing (75-100 kg CO2/kWh), they lack specific quantitative
9 data on the emissions saved per mile or year to allow for a direct
10 comparison or break-even analysis.
11 MISSING: Quantitative data on CO2 emissions from internal combustion
12 engine vehicles or the "break-even" distance/time required for an EV
13 charged by renewables to offset its manufacturing carbon footprint.
14
15 DEBUG agentic-rag: context INSUFFICIENT, refining...
16 DEBUG refine-queries: generated 2 refined queries:
17 - lifecycle CO2 emissions per mile internal combustion engine vs EV
18 battery manufacturing data
19 - break-even driving distance EV charged by renewables to offset
20 battery production emissions
21
22 DEBUG search-fanout: found 4 unique chunks
23
24 --- Iteration 2/3 ---
25
26 DEBUG assess-sufficiency: evaluating 6 chunks
27 DEBUG assess-sufficiency response:
28 VERDICT: INSUFFICIENT
29 REASON: While the passages provide the carbon footprint for battery
30 manufacturing (75-100 kg CO2/kWh), they lack the specific emission
31 data for gasoline vehicles needed to calculate the "emissions saved"
32 or a direct break-even analysis.
33 MISSING: Average carbon emissions of internal combustion engine
34 vehicles per mile and the specific distance or timeframe required for
35 an EV charged on renewables to offset its manufacturing carbon debt.
36
37 --- Iteration 3/3 ---
38
39 DEBUG agentic-rag: max iterations reached, synthesizing with available context
40
41 DEBUG synthesize-answer: generating answer from 7 chunks
42
43
44 ANSWER 2:
45 Manufacturing EV batteries produces approximately 75-100 kg of CO2
46 per kilowatt-hour of battery capacity (source: electric-vehicles.txt).
47 When an EV is charged using renewable energy sources like solar or
48 wind, it produces zero operational emissions (source:
49 electric-vehicles.txt). Even when accounting for manufacturing, EVs
50 produce roughly 50-60% fewer lifecycle greenhouse gas emissions than
51 comparable gasoline vehicles when charged from the average US grid
52 mix (source: electric-vehicles.txt).
53
54 Missing Information: The retrieved passages do not provide a specific
55 "break-even" point (such as the number of miles or years) at which
56 the emissions saved by renewable charging fully offset the initial
57 CO2 generated during battery manufacturing.
Notice that iteration 3/3 goes straight to synthesis: the sufficiency check is skipped entirely at the last iteration because its verdict cannot change the outcome. The pipeline honestly reports what it found and what is missing, which is the right behavior when the documents simply do not contain the requested data.
After the test completes, you can use the returned corpora for interactive queries:
1 * (defvar *corpora* (rag:test))
2 ;; ... test output ...
3
4 * (rag:interactive-demo *corpora*)
5
6 ============================
7 Agentic RAG Interactive Demo
8 ============================
9
10 Loaded 3 corpora with 23 total chunks.
11 Type your question (or 'quit' to exit):
12
13 RAG> What is the Paris Agreement temperature target?
14
15 ===== ANSWER =====
16 The Paris Agreement aims to limit warming to 1.5°C above
17 pre-industrial levels (source: climate-science.txt).
18 ==================
19
20 RAG> quit
When rag:test returns the corpora and the REPL prints the return value, the custom print functions from the vector store section keep the output readable: each corpus is one line with its name, description, and chunk count, and inspecting a single chunk shows only the first 10 embedding values plus the dimension count. Without those printers, this one expression would dump all 23 chunks with their full 3072-value embeddings.
Offline Tests
Because *embedding-fn*, *batch-request-fn*, and *generate-fn* are special variables holding functions, the whole pipeline can be exercised without network access or an API key:
1 * (asdf:test-system :rag)
2
3 All RAG tests passed.
The test system (tests.lisp, package rag-tests) covers the chunking edge cases (including the forward-progress guard discussed above), query line parsing (queries that begin with digits or decimal numbers survive; list prefixes of any length are stripped), vector math (including the dimension-mismatch error), retrieval ranking and deduplication (identical text from different files stays distinct), batched query embedding (one call for the whole fanout), batch splitting at the 100-text API cap, cache eviction at the cap, retry behavior (transient 429/5xx and connection errors retry; permanent 4xx signal immediately), verdict parsing (including the INSUFFICIENT-contains-SUFFICIENT substring trap), corpus save/load round-trips with corrupt-file rejection, and a full agentic-rag run against a stubbed LLM, including the checks that the last iteration skips the sufficiency call and that a quiet pipeline prints nothing. There is no testing framework dependency; a small check macro records failures and run-tests signals an error if any occurred, which is all ASDF needs to report test failure.
Wrap Up for Agentic RAG
The key takeaway from this chapter is that agentic RAG dramatically improves answer quality compared to vanilla RAG, especially for complex queries that require information from multiple sources. The Sufficient Context Agent is the critical innovation: by explicitly checking whether enough information has been retrieved before generating an answer, we avoid the common failure modes of hallucination and incomplete responses.
The implementation is deliberately simple: each “agent” is just a function with a well-crafted prompt. You don’t need an elaborate agent framework to get the benefits of multi-agent architectures. What matters is the pattern: decompose, search, assess, refine, synthesize.
The engineering around that pattern is deliberately practical as well: embeddings are stored as normalized simple-vectors computed with batched API calls and memoized, all sub-queries in a fanout share one embedding request, the API key travels in a header instead of the URL, transient HTTP and connection failures are retried while permanent client errors surface immediately, corpora can be saved to disk, validated on load, and reloaded without re-embedding, and every network-facing function sits behind a rebindable special variable so the entire pipeline is testable offline.
For production use, consider these enhancements:
- Persistent vector store: Replace the in-memory lists with a dedicated vector database (Chroma, Qdrant, or Pinecone) for larger document collections. Because chunks are stored normalized, a store that indexes unit vectors can use plain dot-product scoring directly.
- Document loaders: Add support for PDF, HTML, and other formats beyond plain text.
- Structured agent outputs: Request JSON Schema-constrained responses from the Interactions API instead of parsing VERDICT lines (see practice problem 5).
- Parallel search: Use threads to search multiple corpora simultaneously (see practice problem 6).
The Google research reports that their production agentic RAG system achieves up to 34% higher accuracy than vanilla RAG on factuality benchmarks, with cross-corpus retrieval nearly matching single-corpus accuracy. Our Common Lisp implementation demonstrates the same architecture on a smaller scale.
Optional Practice Problems
Custom Chunk Size and Overlap Strategy: The logic inside
split-into-chunksin vector-store.lisp uses the package constants*default-chunk-size*(500 characters) and*chunk-overlap*(50 characters). Modifyadd-documentin vector-store.lisp andagentic-ragin agents.lisp to support dynamic configuration of these parameters. Write a helper function that measures the sensitivity of retrieval relevance scores to different chunk configurations.Deduplication with Embedding Similarity (Soft Deduplication): In
search-fanout(in agents.lisp), the retrieved chunks are deduplicated by exactdocument-chunk-keymatching (source plus text). In large datasets, different documents might contain near-identical chunks or rephrased content. Implement a “soft” deduplication mechanism insearch-fanoutthat uses thecosine-similarityfunction from embeddings.lisp to discard any retrieved chunk that has a similarity score greater than 0.9 with an already selected chunk. Remember that stored embeddings are normalized, so the comparison is a plain dot product.Multi-Turn Chat Interface Integration: The current
interactive-demoloop in rag.lisp and the orchestratoragentic-ragin agents.lisp are stateless: each query is processed independently. Extend the pipeline to support a conversation history (list of past QA turns). Pass the history to theQuery Rewriterso it can resolve pronouns and context (e.g., rewriting “How does it compare to hydro?” following “What is the cost of battery storage?”).Self-Correction and Web Search Fallback on Refinement: In
agentic-rag(in agents.lisp), if the context remains insufficient after refining queries, the system continues to search the same corpus. Write a fallback mechanism that, when the Sufficient Context Agent reportsINSUFFICIENTfor the second time, switches to an external API (like a local DuckDuckGo lookup or Ollama Cloud web search helper) to gather external context, appending the results to the RAG vector store dynamically.JSON Schema Schema Enforcement for Agent Verdicts: The Sufficient Context Agent in agents.lisp relies on string matching (searching for
"VERDICT:"and"MISSING:") to parse Gemini’s response. This is fragile if the model outputs code blocks, explanations, or formatting deviations. Modifyassess-sufficiencyto request structured output using a JSON Schema (by specifying schema parameters in the request payload to the Gemini Interactions API). Parse the returned structured JSON reliably usingcl-json.Parallelized Search Fanout: The
search-fanoutfunction in agents.lisp embeds all sub-queries in one batched API call, but still searches them sequentially: each sub-query’ssearch-corporacall runs one after another. When there are several sub-queries and multiple corpora, searching one by one adds latency. Use a threading library such asbordeaux-threadsto parallelize thesearch-corporacalls per sub-query, gathering and deduplicating results once all threads terminate.