AI Engineering Visualized, Volume 3: RAG & Knowledge Systems

The best match was true once. Now it's wrong.

Asha asks whether order #2001 can still be refunded. The best match in the assistant's new search index is an old ticket saying the parcel is still on its way. That ticket was true when it closed. Now it's stale: the order record says #2001 was delivered. A better search would still find that ticket. The code around the search decides which sources may answer, and this book builds that code.

For a retrieval feature whose answers you now have to trust. The sources are manuals with tables, an order database, past tickets and rules that change by region. Volume 1 built the Northwind Goods support assistant with a small hybrid search over one policy page and nine handbook passages. Volume 2 put a harness around it so it could act. Volume 3 gives it real knowledge and makes every answer traceable to a source.

One assistant, carried forward. Chapter by chapter you build the knowledge system behind the model:

- a contract and an access path for every source, and an ingestion pipeline with stable ids, provenance and safe reruns

documents with tables and scans, cut into chunks that know where they apply

permissions from the session, checked at search and at every later step, with poisoned pages treated as a signal and never as the boundary

embeddings and vector indexes, hybrid search with an exact-match lane, reranking, and queries that are rewritten, routed and decomposed

a clear rule for which source wins when two disagree, when to abstain, and an evidence pack the model can cite

safe, scoped SQL over records, and multi-hop retrieval over a knowledge graph

index updates, deletes and rebuilds, retrieval metrics and faithfulness judges, and test sets with a regression gate

A capstone appendix then sends one ticket through every stage on one corpus.

Visual first. Every idea is one of 35 named mental models, such as The Ingestion Conveyor, Small to Find, Big to Read, The Two-Stage Funnel, The Evidence Pack, The Stale Index or The Regression Gate. Each one has a hand-drawn doodle, a plain definition, a failure you can trigger and the fix.

Real code, no API key. The examples are short Python excerpts from the free companion repository (MIT licence), built from plain stages on the Python standard library with no framework. By default they run on a stubbed model, with no API key, no network and no cost. You can point a chapter at a real local model with Ollama if you like. Each Model experiment reports one real llama3.2 run with its settings and log, not a rate; runs not made yet are marked [TO MEASURE].

Chapters 3, 7 and 10 are in the free sample.