Eval Engineering

Specification Engineering turns complex ideas into clear, workable specifications. This practical guide takes you from core principles to advanced methods for defining, validating, managing and evolving requirements across products, software, systems and services. Built for real-world use, it gives you the tools to create specifications that people can understand and act on.

Vectors, embeddings, retrieval, agents, and evaluation are all built from first principles inside the chapter that needs them. No mathematics. No machine learning background. No prior AI experience and no framework knowledge is required. We build with plain Python and small, single-purpose libraries.  

Eight agents, zero coordination, one identical error. That's not a coincidence. It's a signature. Learn to read the five families of failure behaviors in multi-agent systems, reproduce each one in TypeScript, and turn every bias into a CI assertion before it ships.

A hands-on, failure-first guide to evaluating LLM agents, RAG, tool use, grounding, and production release gates—with executable Python examples and tests.

Great software is built through great reviews. This book shows you how to spot hidden bugs, improve design, strengthen security, and review AI-generated code with confidence. Packed with practical examples and proven techniques, it helps you write better software in a world where humans and AI build code together.

Evaluation Engineering for AI Systems is a practical guide to building reliable AI evaluations for production. Learn how to measure performance, compare models, detect regressions and create evaluation pipelines using real-world examples, modern Python and proven industry practices.

Your AI-generated code passes 18,000 tests and reports healthy — while entire data pipelines silently produce nothing. Silent Wiring names the failure mode nobody's tooling catches, and shows you how to find it before your users do.