LLM Engineering, from Component to Production

LLM Engineering, from Component to Production is a practical, measurement-first guide to building a local-first agentic retrieval-augmented generation system. Across sixteen chapters, one evolving codebase, and a sequence of hands-on labs, it treats the language model as a fallible software component rather than a magical application layer. The central promise is not that the reader will memorize a framework, but that they will finish with a system they have built, measured, secured, observed, and prepared to deploy.

The book begins at the component level. It explains tokenization, sampling, constrained decoding, tool calling, and context management from first principles. The reader writes the tool-calling loop by hand before using an orchestration framework, making the runtime contract explicit: the model proposes a structured action, ordinary code validates and executes it, and the result is returned to the model. This foundation supports a broader engineering principle repeated throughout the manuscript: inference should be used for judgment, while loops, conditionals, data transformation, validation, and orchestration belong in deterministic code.

The retrieval section builds a strong baseline before introducing more elaborate techniques. It covers layout-aware document parsing, structural chunking, provenance, idempotent ingestion, dense and lexical retrieval, reciprocal rank fusion, cross-encoder reranking, and contextual retrieval. Particular emphasis is placed on silent failure modes: indexes built with the wrong embedding model, BM25 constructed over an empty store, lost page metadata, duplicated model calls, and retrieval pipelines that appear to work while returning weak evidence. GraphRAG and query expansion are assessed against measured costs rather than adopted as defaults.

The agency section explains when an agent is justified and when a fixed pipeline is the better design. It covers current LangChain and LangGraph patterns, middleware, durable state, human-in-the-loop interrupts, Model Context Protocol, protocol alternatives, and tool-surface security. The recurring design constraint is containment: read-only tools should remain simple, privileged actions require narrow interfaces and validation, and untrusted model output must never be dispatched through unrestricted reflection or unchecked arguments.

Evaluation, security, observability, and cost form the rigor layer. The book advocates a hand-written golden set, deterministic retrieval metrics such as recall at k and mean reciprocal rank as CI gates, and judged generation metrics as noisy monitoring signals rather than binary tests. It includes prompt pinning, embedding-index skew checks, injection regression tests, OpenTelemetry tracing, latency decomposition, token accounting, and cost-per-query measurement. The reader is repeatedly asked to record before-and-after numbers so that architectural claims remain falsifiable.

The production chapters move from local development to serving and deployment. They cover vLLM, continuous batching, PagedAttention, KV-cache sizing, quantization, concurrency testing, blue-green index releases, canaries, rollback, on-premises and cloud options, data residency versus sovereignty, and EU AI Act implications. The final chapters turn the completed system into a capstone, a defensible CV project, and a set of technically precise interview explanations. Two appendices reinforce the book’s deeper lesson: audit confident technical advice critically, and prefer architectures whose behavior can be inspected, measured, and explained.

The intended outcome is a working system and an evidence trail: a corpus chosen by the reader, a golden set written by hand, retrieval and generation metrics, trace data, security tests, deployment artifacts, and a concise record of which changes improved the system and which were not worth their cost.