Join us live on Zoom for a GhostAI workshop TODAY! 1PM Pacific / 4PM Eastern. Learn more…

Leanpub Header

Skip to main content

Failure Driven Development

Software Development Habits Haven't Caught Up to Cheap Compute

This book is 21% completeLast updated on 2026-05-26

Failure Driven Development Designing Software the Way Engineers Design Circuits — Powered by Throwaway CPU Cycles

Software Development Habits Haven't Caught Up to Cheap Compute. Design software the way electrical engineers design circuits — probes, test jigs, and a bowl of trusted parts — and field defects follow the same trajectory as hardware: toward zero.

Minimum price

$19.00

$29.00

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
EPUB
WEB
About

About

About the Book

Current programming languages address exactly one paradigm: synchronous and sequential. That paradigm served the 20th century well, when CPUs were expensive and every cycle had to count.

The 21st century changed the ground rules. CPUs are now effectively free. The internet, robotics, IoT, and control systems are now mainstream. Problems that cannot be forced into a synchronous, sequential mold are no longer edge cases — they are the common case.

Yet our development workflow has not changed. We still spend our time shaping workarounds, bending non-sequential problems into a sequential mold, and discarding the problem variations that won't fit. We optimize for code quality over design quality. We treat premature perfection as a virtue.

This book does not claim to correct that imbalance. It points at it, examines it, and argues that attention to design is at least as important as the correctness of the code that implements it. Accepting the first design that comes to mind is the same mistake as accepting the first code an LLM generates without subjecting it to critique. We would not do the latter. We routinely do the former.

Our current practice of delegating design feedback to paying customers — through quarterly, sometimes daily, updates — produces an unnecessarily slow design validation loop. Our belief that a single device and a single paradigm can be stretched to address every problem produces self-inflicted, accidental complexity that trends toward decreased robustness.

We have also forgotten that the functional paradigm is a choice, not a law of nature. Other choices exist. Computers running operating systems with libraries of function-based code are not the only way to produce electronic solutions. Operating systems and function libraries exist to support the functional paradigm — they may be unnecessary when a different paradigm is the better fit. They are development tools. We should be using and culling them to produce solutions. Instead, we develop inside a soup of development tools and ship the entire kit as a product to non-developers.

Exploration of the design space deserves at least as much emphasis as producing correct code. No developer can explore every corner of a design space — the fullness of that exploration will always depend on the developer's own judgment about how far to push. But we settle for far less exploration than is warranted, and our tools and workflows are a primary reason why. They discourage backtracking. They make trying out an alternative design idea feel expensive. They treat every restart as a failure rather than as progress. If we can recognize these obstacles clearly, we can begin thinking about programming differently — not as a monolithic activity aimed at a single correct solution, but as a process that alternates deliberately between exploration and consolidation, between design and perfection, with honest awareness of the limits of our initial understanding.

The model this book proposes is the electrical engineer's bench: snap together trusted black boxes, probe the results, iterate without friction, test compositions with test jigs before committing to a full design. Small tools, little languages, and scripted loops replace the monolithic toolchain. The design space gets explored — as fully as the developer's judgment allows — instead of being prematurely collapsed. The book makes this concrete with a step-by-step worked example: a small PBP project building a "Five Whys" tool using two instances of an LLM. The example demonstrates designing from the top down and from the bottom up, and shows what backtracking looks like in practice when new knowledge is acquired mid-design.

This is not new capability. We have had the CPU headroom to work this way for some time. What has held us back is economics and a "make it work with what we have" attitude rather than a "design for what the problem actually requires" attitude — and an underemphasis on design validation as a result.

This book does not claim to have the final answer, or to relieve programmers from the obligation to think hard and explore design spaces to their own satisfaction. The invisible hand of the market should do what it did in the early days of game programming — cull poor designs, poor code, and anti-robustness driven by featurism. What this book argues is that we need to be improving our workflow on many fronts simultaneously, rather than deep-diving into a single aspect of coding — correctness of code — while leaving the prior question of design quality largely unexamined. Shining a light on more of the issues may itself change the way we think about when to explore and when to perfect.

Field reliability follows as a natural consequence of getting the design right — the same way it does in hardware.

Share this book

Author

About the Author

Paul Tarvydas

Who Am I and Why Should You Care?

I'm a retired electrical engineer who ran a software consultancy for over 25 years. Embedded systems (credit card terminals, the kind of hardware that has to work every single time), Y2K renovation of COBOL codebases using advanced parsing and backtracking tools and design recovery, tax platforms, business strategy tools — I've built the unglamorous stuff that actually runs the world. None of it was glamorous. All of it taught me something.

What it taught me, mostly, is that software is far more complicated than it needs to be.

Hardware designers work with schematics and ICs — reliable, composable, and honest about time. Software designers inherited notation from the 15th century printing press, then the 19th century QWERTY typewriter, and never stopped to ask whether those tools were the right fit for a fundamentally different medium.

They aren't.

There's a trap worth naming early: casting deep thinking into mathematical form is a one-way operation, not two-way. Once you understand something, you can express it in math — but the math doesn't contain everything you understood. Whatever was elided to make the expression concise is gone. The map is not the territory, and the equation is not the insight. I try to keep that distinction front of mind in everything I write here.

The core thesis

Real life is asynchronous. Software pretends it isn't. That pretense is the source of most of the accidental complexity we spend our careers managing — callback hell, promises, mutex locks, race conditions, the whole zoo.

I've spent years asking one question: why are hardware designs more robust than software designs? Hardware is fundamentally asynchronous — components react to signals, they don't wait their turn. If we expressed software the same way, would reliability go up? I think yes. And I don't think the current answers — threads, promises, async/await — are the right path. Bolting asynchrony onto a synchronous foundation feels fundamentally backwards. What we're doing today is like writing assembler. We need better notations for expressing compositions of asynchronous components. Can we just start from asynchrony instead? I'm now convinced the answer is yes, and that doing so leads to something almost embarrassingly simple. If you know how to build a queue, you're most of the way there.

What I publish

I've put out hundreds of articles on GitHub Pages and Substack, and hundreds of videos on YouTube. Most of it is free. I believe in partial marks — publish the unpolished thought, not just the finished paper. A lot of what I write reads more like a working diary than a journal article, and I think that's a feature, not a bug.

The topics look scattered from the outside. Diagrams as code. Prolog. Forth. Functional programming's hidden assumptions.

They're connected. The connecting thread is always the same question: what is being elided, and what does that cost us?

What I'm driving at

We are still using 2D notation — text, math symbols, sequential syntax — to describe programs that run on hardware that lets us think in four dimensions. x, y, z, and time. We have the technology to express programs in 4D. We just haven't built the habit.

Functional programming elides time and ordering. That's fine for ballistics computations. It is not fine for systems that need to respond to the world as it actually is — concurrent, event-driven, asynchronous by nature.

I'm not arguing that functions are wrong. I'm arguing that functions are not enough, and that treating them as the only legitimate notation is what got us into this mess.

The path out involves asynchronous-by-default design, diagram-based notations, staged computation, and the willingness to admit that no single notation covers all bases. Software is catching up slowly.

I'm trying to help it catch up faster.

Launch

Launch Video

Contents

Table of Contents

Introduction

Overview

  1. Who Is This Book For?
  2. What You Can Do Differently
  3. The Design Problem
  4. Zero Defects In The Field

Are We Really Better Than Sketchpad?

Concurrency and Parallelism

  1. Concurrency Is Not Parallelism

Code Perfection vs. Design Perfection

The Failure Driven Development Workflow - Top Down and Bottom Up

The Synchronous Straightjacket - Why Today’s Programming Cannot Solve Tomorrow’s Problems

  1. Programming Is a Subset of What a CPU Can Do
  2. The Economics of the Mistake
  3. Symptoms: Thread Safety and Concurrency
  4. Symptoms: Callback Hell
  5. The Internet Is Not Synchronous
  6. The Dark Matter Analogy
  7. The Notation Trap
  8. The Workflow, Not the Language
  9. The 20th Century as Stepping Stone
  10. What Needs to Change
  11. Changing The Software Development Workflow

Multiple Notations in a Single Project

Making Erasure Cheap

Black Boxes, Not Functions

Little Languages and Small Tools

T2T: Text To Text Transmogrification

Bench Testing and Component Palettes

  1. Testing Examples

Knowing When to Commit

  1. FDD in Practice

Example FDD Workflow - The Five Whys Example

  1. Getting the LLM Running
  2. Wrapping the LLM as a Part
  3. What I Learned From This Test Jig
  4. Build a Stub Test Jig (No API Calls)
  5. Count To Five
  6. Rethink and Retrench
  7. Continue Count To Five
  8. New Insights, Pass 2
  9. Different Ways to Probe
  10. Internally Rephrasing Answers as Questions
  11. New Part For Rephrasing Answers
  12. Make the Debug Output More Concise
  13. Conclusion
  14. Parts Based Programming
  15. Suggested Starting Rules
  16. Resources
  17. Everything is a Black Box Part
  18. Drafting Rules For Node-and-Wire Diagrammatic Languages
  19. Building Multi-Language Tools
  20. @make

Appendix A - How We Got Here

Historical Choices

Get the free sample chapters

Click the buttons to get the free sample in PDF or EPUB, or read the sample online here

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earned over $15 million writing, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub