Leanpub Header

Skip to main content

The AI Diagnostic Manual

Symptom-Based Troubleshooting for LLM Applications

The AI Diagnostic Manual

It is three in the morning and something is wrong with your LLM application. This is the book you open. Not a tutorial — a reference manual, organised by symptom, for the moment something you built has already broken.

Minimum price

$34.99

$34.99

You pay

Author earns

$

Also available for 2 book credits with a Reader Membership

PDF
EPUB
About

About

About the Book

It is three in the morning and something is wrong with your LLM application. This is the book you open.

Not a tutorial. A reference manual, organised by symptom, for the moment when something you built has already broken.

Everything else on this shelf teaches you to build. Almost nothing helps you debug what you have already built. This book is the other half: eighteen chapters, each one a symptom in the words you would actually use.

The output looks correct and is wrong. The agent is looping. It worked yesterday. The tool call never fires. It works in testing and fails in production. Costs are spiking. It answered from the wrong document. The JSON will not parse. It is too slow. It works for me and not for them.

Hallucination, retrieval failures in a RAG pipeline, prompt regressions, runaway token spend, latency, agents that will not stop — each one gets a chapter, and each chapter gets you to a check in under two minutes.

Every chapter follows the same structure, built for lookup under pressure. The recognition lines, so you know in seconds whether you are in the right place. The first two minutes — the small number of actions that halve the search space before you understand anything. Then the candidate causes, ordered by how common they are and how cheap they are to test, and for each one the part that makes this a diagnostic rather than a list: how to tell it apart from the cause next to it. A specific test, with a specific outcome, that separates them.

And every chapter can tell you that it has nothing for you. A diagnostic that always finds something is not a diagnostic. Chapter 18 is the honest dead ends — when it is not the model, when it is your data, when the requirement changed, when it is the provider and there is nothing to do, and how to know you have passed the point where more investigation helps.

The cases are real and they were read at source. A tribunal decision against an airline whose chatbot invented a refund rule, and whose wrong answer carried a link to the very page that contradicted it. A support bot that explained a genuine bug with a cause that did not exist, and the subscriptions cancelled because of it. Two vendor postmortems in which the models changed underneath applications nobody had touched — including one where the evaluations passed, the A/B test passed, and the only instrument pointing at the real problem was a tester saying it felt slightly off. And two code-repository issues where the fault was nowhere near where anyone was looking: one a single field quietly rewritten in transit, one a loop with no memory of what it had already done.

Not one of them is a story about a model being stupid. Every one is a failure in a seam — between a system that expects guarantees and a component that does not offer them. That is where these bugs live, and that is what this book is a map of.

Also inside: the minimum trace record, in six fields — the observability you actually need, which is far less than a platform will sell you, and which makes every technique here possible. The bisect protocol. A one-page triage. A fast check for every symptom. A pre-incident checklist of what to have in place before you need it. And a closing chapter on reasoning about a failure that is not in this book or any other — because that is where you will be soon enough.

Version-dependent material is marked and date-stamped, and kept separate from the durable material, which is most of it. The arithmetic does not change. Neither does the method.

Written by Cynked Press under its own name, with no pen name and no invented credentials. Every source is cited. Where the book does not know, it says so.

Author

About the Author

Cynked Press

Cynked Press publishes non-fiction. Its technical titles go out under the imprint's own name, with no pen name and no invented credentials, because the expertise behind them is ordinary and real: daily work building and operating these systems. Every case in its books is public, cited, and read at source rather than taken from a summary. Where a book doesn't know something, it says so.

Contents

Table of Contents

  1. The four questions that come before everything
  2. You cannot debug what you cannot see
  3. The output looks correct and is wrong
  4. It worked yesterday
  5. The agent is looping
  6. The tool call never fires
  7. It works in testing and fails in production
  8. It works for me and not for them
  9. Costs are spiking
  10. It answered from the wrong source
  11. It fails only sometimes
  12. The output will not parse
  13. It is too slow
  14. It forgets, or ignores the middle
  15. Something in the pipeline is wrong and you do not know which step
  16. It did something it was told not to do
  17. How to reason about a failure you have never seen
  18. This is not your problem

Appendices

  • A — The bisect protocol
  • B — The minimum trace record
  • C — Fast checks, by symptom
  • D — The pre-incident checklist
  • E — Triage, in one page

Plus: Glossary · Sources · On what will go out of date

Get the free sample chapters

Click the buttons to get the free sample in PDF or EPUB, or read the sample online here

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earned over $15 million writing, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub