- Preface
- Part I. Foundations
- 1. What Is an Agent? The Governed Loop
- 1.1 The naive loop
- 1.2 The governed loop
- 1.3 Taxonomy
- 1.4 The minimal agent loop
- 1.5 The agent
- 1.6 The environment
- 1.7 Running the loop
- 1.8 Inspecting step records
- 1.9 What the loop does and does not do
- 2. Knowledge Graphs and the Geometric Memory Substrate
- 2.1 From a policy document to a knowledge graph
- 2.2 Querying the graph
- 2.3 What GMS adds: plausibility as a distance
- 2.4 Exact numbers, recalled losslessly
- 2.5 The operator set
- 2.6 The substrate the book runs on
- 3. Reasoning Traces Are Not Evidence
- 3.1 Trust levels
- 3.2 The unstructured approach
- 3.3 The structured approach
- 3.4 Verification
- 3.5 Invariants enforced at construction time
- 3.6 The independent verifier
- 1. What Is an Agent? The Governed Loop
- Part II. Tasks, State and Tools
- 4. Tasks, State and Actions
- 4.1 TaskSpec carries more than a prompt
- 4.2 AgentState round-trips losslessly
- 4.3 Actions as a discriminated union
- 4.4 parse_action: replay from JSON
- 4.5 Refitting the loop
- 5. Tools as Typed Actions
- 5.1 Define typed tools
- 5.2 Registration and validation
- 5.3 Three routers, one benchmark
- 5.4 When to pick each router
- 6. Safe Tool Execution and Tool Testing
- 6.1 A simple tool to gate
- 6.2 The three gates
- 6.3 Policy as code
- 6.4 Plausibility: refusing the outlier
- 6.5 The geometric plausibility gate
- 6.6 Tool contract testing
- 6.7 Exercises
- 7. Cost, Latency and Budgets
- 7.1 A Budget caps each axis
- 7.2 Why synthesize an Escalate
- 7.3 Model adapters report their own usage
- 7.4 Where the budget is enforced
- 7.5 Exercises
- 4. Tasks, State and Actions
- Part III. Planning and Memory
- 8. Planning, Decomposition and Replanning
- 8.1 A reference task
- 8.2 WorkflowPlanner: predefined steps
- 8.3 LMPlanner: JSON plan from a language model
- 8.4 GraphSearchPlanner: BFS over an action graph
- 8.5 Decomposition: one task becomes many
- 8.6 Replanning fires on tool failure
- 8.7 Validating the plan before it runs
- 8.8 Exercises
- 9. Memory: Types, Retrieval and Hybrid Stores
- 9.1 Memory kinds
- 9.2 The embedder
- 9.3 VectorMemory: chunk, embed, retrieve
- 9.4 GraphMemory: triples and multi-hop
- 9.5 ShortTermMemory: bounded recency
- 9.6 Hybrid: combine the three
- 9.7 A fourth tier: exact numbers and contradiction rejection
- 9.8 Exercises
- 8. Planning, Decomposition and Replanning
- Part IV. Evaluation
- 10. Trajectory Evaluation and Metrics
- 10.1 Three trajectories ending three ways
- 10.2 Summaries are pure functions over a Trajectory
- 10.3 Groundedness: extract claims, match evidence
- 10.4 A non-LLM verifier: groundedness as distance
- 10.5 Scoring a retrieval step
- 10.6 Exercises
- 11. Failure Modes and Design of Experiments
- 11.1 The failure-mode catalog
- 11.2 Balanced design over a small factor table
- 11.3 Generate test cases from the design
- 11.4 Adversarial users versus adversarial inputs
- 11.5 Space-filling designs: covering interactions, not just levels
- 11.6 A benchmark is a designed experiment, not a scoreboard
- 11.7 Verifying answers against graph truth
- 11.8 Fault injection at the tool boundary
- 11.9 Release gates: turning scores into a ship decision
- 10. Trajectory Evaluation and Metrics
- Part V. Governance
- 12. Runtime Governance: Gates, Policy-as-Code and Audit
- 12.1 The scenario: a malicious email
- 12.2 Config 1: no policy gate
- 12.3 Config 2: add a PolicyGate with two policies
- 12.4 Config 3: full harness, audit log produced
- 12.5 Policy-as-code in practice
- 12.6 The geometric gate: is this call a legal next step?
- 12.7 Why prompt-only fails
- 12.8 Auditability is not optional
- 13. Governed Retrieval
- 13.1 The retrieval channel as a control surface
- 13.2 Retrieval returns facts, not documents
- 13.3 The sensitivity map and the retrieval contract
- 13.4 Enforcement on the retrieved facts
- 13.5 Least-context synthesis
- 13.6 Injection in the retrieved channel
- 13.7 The output-disclosure gate: geometry against a trained classifier
- 13.8 Audit on provenance
- 13.9 Purpose-based access and the control stack
- 14. Human-in-the-Loop and Escalation UX
- 14.1 The setup: a tool call that triggers escalation
- 14.2 What the reviewer sees
- 14.3 APPROVE: the gate is overridden
- 14.4 DENY: keep the failure, the loop continues
- 14.5 DEFER: end the loop in escalated state
- 14.6 The three decisions are the contract
- 14.7 The latency cost of human review
- 12. Runtime Governance: Gates, Policy-as-Code and Audit
- Part VI. Systems and Capstone
- 15. Multi-Agent: When, How and When Not To
- 15.1 Build two workers, each with its own harness
- 15.2 A supervisor routes by name
- 15.3 Each worker has an independent audit chain
- 15.4 When not to use multi-agent
- 15.4.1 Question 1: Can a single well-engineered agent with planning and tools do this?
- 15.4.2 Question 2: Do the workers really have different capabilities, policies or audit boundaries?
- 15.4.3 Question 3: Are the costs justified?
- 15.5 Typed messages versus free-form chatter
- 15.6 Conflict resolution by evidence, not by vote
- 16. Capstone: Governed Banking Complaint Agent
- 16.1 What the agent does
- 16.2 Build the harness
- 16.3 The five tools: how they are built and registered
- 16.4 The gate stack
- 16.5 Run a routine inquiry
- 16.6 An adversarial overdraft case escalates via UDAAP
- 16.7 A PII case is caught at the first tool call
- 16.8 Fact extraction as a policy-grounded query
- 16.9 The regulatory guard: GMS as a governance backstop
- 16.9.1 The knowledge graph
- 16.9.2 Reading the message geometrically
- 16.9.3 Calibration
- 16.9.4 Verify and correct
- 16.9.5 Escalation as a multi-hop traversal
- 16.9.6 Fail loud, not silent
- 16.10 The reasoning record: typed claims, not free-form thought
- 16.11 Verifying the draft: exact numbers and unauthorized promises
- 16.12 Aggregate over all cases
- 16.13 Inspect the drafts
- 16.14 What the agent does and does not do
- 16.15 The thesis, restated
- 15. Multi-Agent: When, How and When Not To
- Part VII. Testing
- 17. Testing the Capstone Agent
- 17.1 Coverage, judgment, attribution
- 17.2 Designing the complaint suite
- 17.3 Running the governed agent
- 17.4 Judgment: correctness and groundedness
- 17.5 Attribution
- 17.6 Which component failed: per-tool decomposition
- 17.7 Resilience: faulting each tool in turn
- 17.8 The hardest fault: a reversed stance
- 17.9 What stays for the testing monograph
- 17.10 Exercises
- 17. Testing the Capstone Agent
- A. Frontier Topics
- A.1 Reflection as a hypothesis
- A.2 When reflection drifts
- A.3 Trajectory preferences
- A.4 Learning from trajectories more generally
- B. Framework Comparison
- B.1 The agentlab assembly, recap
- B.2 What a LangChain version looks like
- B.3 Side-by-side
- B.4 Recommendation matrix
- C. The GMS Substrate and Its Calibration
- C.1 The agent's view of GMS
- C.2 Building the store
- C.3 The calibrated thresholds
- C.4 A labeled cohort
- C.5 The sweep
- C.6 Pick the operating point
- C.7 Seal it as a deployable artifact
- C.8 When more than one threshold is in play
- C.9 When to recalibrate
- D. The Capstone Tools in Depth
- D.1 A common structure
- D.2 classify_complaint
- D.3 extract_facts
- D.4 search_policy
- D.5 flag_regulatory
- D.6 draft_response
- D.7 Training the draft adapter: the MeMo v3 corpus
- D.8 Two judges for the draft: groundedness and fluency
- D.9 The harness and the gate stack
- D.10 Reproducing the tools
- About the Authors
- About KnowlytiX
Beyond "Prompt and Pray"
Building Governed Agentic Systems with Geometric Memory and Verification A Practical Tutorial on Perception, Planning, Tools, Memory, Evaluation and Runtime Governance
A language model predicts tokens. An agent acts—and must be held accountable. Learn to replace “prompt and pray” with governed tools, geometric memory, independent verification and auditable runtime controls.
Minimum price
$9.95
$29.95
You pay
Author earns
About
About the Book
Most AI agents are built on a fragile premise: give a language model a prompt, connect it to tools and trust another model to judge whether it behaved correctly. That approach may produce an impressive demonstration but it does not produce a system that is safe, auditable or ready for regulated environments.
Beyond “Prompt and Pray” presents a practical architecture for building governed agentic AI. It shows how to surround the model with typed actions, controlled tools, structured memory, calibrated gates, independent verification, human escalation and replayable audit trails. Through working Python examples and a complete banking complaint agent, readers learn to build AI systems whose actions are bounded, evidence is traceable and failures can be detected before they become consequences.
Author
About the Author
Agus Sudjianto
Agus Sudjianto is the Chief Scientist at KnowlytiX. He has spent more than two decades building, governing and validating quantitative models inside major financial institutions. He was Executive Vice President and Head of Model Risk at Wells Fargo, where he served on the Management Committee and led enterprise model risk management. Earlier in his career he held senior quantitative risk roles at Lloyds Banking Group and Bank of America. Since leaving corporate industry, he has continued this work as an advisor, builder and researcher across banking, fintech and AI.
Agus's work sits at the intersection of machine learning, model risk and governed AI systems. He created PiML and MoDeVa, toolkits for interpretable model development and validation, and his more recent work extends that same discipline into agentic AI, graph-grounded retrieval and geometric memory. Across these projects, the through-line is consistent: high-stakes AI should be built with the same rigor expected of high-stakes statistical models.
He is also co-author of Design and Modeling for Computer Experiments, holds several U.S. patents and has long worked across engineering, quantitative finance and applied machine learning. His current research centers on learning as geometry discovery in both predictive machine learning and generative AI.
In this series, Agus brings the perspective of someone who has spent a career asking not only whether a model works, but whether it can be governed, defended and trusted in practice.
WingYan Lau
Wing Yan Lau is the Chief Technology Officer at KnowlytiX. Her work centers on the systems layer that makes GMS usable in practice: document ingestion, knowledge-store construction, query infrastructure, verification pathways and the interfaces that connect governed AI to real enterprise data. She is a co-author of KnowlytiX's research on graph-verified evaluation and structured financial-document retrieval, including work reflected in FinStructBench and in the company's broader knowledge and testing stack.
Wing brings more than two decades of database and data-platform engineering experience to that work. She has contributed to core systems at IBM, SAP and Workday, with technical work spanning query optimization, storage systems and execution infrastructure. That background is visible throughout the KnowlytiX platform, where the challenge is not only to generate answers, but to connect models to structured knowledge in ways that remain exact, inspectable and operationally reliable.
In this series, Wing brings implementation discipline to every layer of the system: how documents become structured stores, how numeric facts remain exact, how graph-backed retrieval is made usable and how governed workflows are turned into code rather than left as intentions in prose. Her contribution is what turns the ideas in the architecture into systems an engineer can actually build, test and run.
Contents
Table of Contents
The Leanpub 60 Day 100% Happiness Guarantee
Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.
See full terms...
Earn $8 on a $10 Purchase, and $16 on a $20 Purchase
We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.
(Yes, some authors have already earned much more than that on Leanpub.)
In fact, authors have earned over $15 million writing, publishing and selling on Leanpub.
Learn more about writing on Leanpub
Free Updates. DRM Free.
If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).
Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.
Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.
Learn more about Leanpub's ebook formats and where to read them
Write and Publish on Leanpub
You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!
Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.
Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.