Leanpub Header

Skip to main content
Go to Leanpub.com

Engineering Distributed Systems on the BEAM

Building Reliable, Scalable Applications with Erlang/OTP and the Ecosystem

Engineering Distributed Systems on the BEAM
This book is 100% completeLast updated on 2026-08-20
Steve Publications

What does it take to build distributed systems that keep working when things go wrong? Engineering Distributed Systems on the BEAM goes beyond syntax to show how Erlang/OTP turns concurrency, failure and recovery into practical design tools. Build a real system from the ground up, then learn how to scale, deploy and operate it with confidence.

Minimum price

$19.00

$29.00

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
EPUB
WEB
APP
161
Pages
About

About

About the Book

This book takes you from the fundamentals of the BEAM virtual machine through to designing, implementing, deploying, and operating production-grade distributed systems. Using a realistic reference application built incrementally throughout each chapter, you will learn how to translate distributed-system requirements into process boundaries, supervision strategies, message protocols, persistence models, and recovery mechanisms. The coverage includes BEAM distribution, clustering, consistency trade-offs, fault tolerance, observability, deployment on Kubernetes, and honest comparisons with other ecosystems. By the end, you will understand not only how to build distributed systems on the BEAM but also how to reason about their correctness, reliability, scalability, performance, security, and operational behavior in production.

Share this book

Categories

Distributed SystemsSoftware ArchitectureSystems EngineeringComputers and ProgrammingSoftware EngineeringErlang

Feedback

Email the Author

Author

About the Author

Steve Publications

Steve is a technology professional with more than 20 years of experience in software development, server infrastructure, cybersecurity, vulnerability research and reverse engineering. Throughout his career, he has designed, secured, analyzed and tested complex software and infrastructure, with a particular focus on understanding how systems fail and how they can be made more secure.

Outside of work, Steve enjoys sharing knowledge with the technology community. He collaborates with researchers, industry experts and technology professionals to write practical books covering software development, cybersecurity, cloud computing, networking, DevOps, artificial intelligence and enterprise technologies. His books focus on practical learning through clear explanations, real-world examples and hands-on exercises. With more than two decades of industry experience, his goal is to help IT professionals, students and technology enthusiasts build useful skills and stay current in a rapidly changing industry.

We believe readers deserve to know how our books are created. Most of our authors are not native English speakers, so we use AI to help translate, proofread manuscripts, fix grammar, improve sentence structure and make technical explanations easier to read. AI is used as an editing tool only. It does not replace the research, technical knowledge or hands-on experience behind our books. Some of our authors also prefer to remain anonymous for privacy or professional reasons. In those cases, we publish their work under a different name. The author's name may be different, but the quality of the content and our review process remain the same.

Every book is written, reviewed and maintained by experienced technology professionals, with contributions from our private technical community of more than 420 engineers and researchers. We spend far more time validating technical accuracy and keeping our content up to date than generating text. We are always interested in working with experienced professionals who have deep expertise in a particular technology or domain. If you would like to publish a book with us or help review an existing manuscript, we'd love to hear from you. Send us a message describing your area of expertise. We are especially interested in niche technologies, specialized skills and emerging topics that are underrepresented in existing technical literature.

If you look through the contents of our books, you'll see practical examples, detailed explanations and material that is regularly updated. Our goal is to publish books that professionals can actually rely on, not low-effort AI-generated content. If you ever feel that one of our books does not meet that standard, Leanpub offers a 60-day money-back guarantee. Feel free to request a refund if you are not satisfied with your purchase.

Contents

Table of Contents

Building Reliable, Scalable Applications with Erlang/OTP and the Ecosystem

Introduction

  1. The Promise of This Book
  2. What Makes the BEAM Different
  3. The Reference Application: Nexus
  4. How to Read This Book
  5. Prerequisites and Tooling
  6. A Note on Guarantees

Chapter 1: The BEAM Execution Model

  1. Virtual Machine Architecture and Schedulers
  2. Lightweight Processes: Creation, Cost, and Lifecycle
  3. Mailboxes and Message Passing Semantics
  4. Immutability and Concurrency Safety
  5. Garbage Collection: Generational, Concurrent, and Stop-the-World
  6. Process Isolation as the Foundation of Fault Tolerance
  7. Summary

Chapter 2: OTP Design Principles and Core Behaviours

  1. The OTP Philosophy: Let It Crash and Why It Works
  2. GenServer: Stateful Processes with Structured Interfaces
  3. Supervisors: Trees, Strategies, and Recovery Semantics
  4. Applications as Composable Units of Deployment
  5. Task Behaviours: OneShot, Sync, and Async Patterns
  6. Designing Process Hierarchies for a Real Application
  7. Summary

Chapter 3: Building the Reference Application — Architecture Foundations

  1. Requirements: Throughput, Latency, Availability, Consistency Goals
  2. Domain Decomposition: Services, Bounded Contexts, Process Ownership
  3. Supervision Topology Design for the Order System
  4. Message Protocol Design: Requests, Responses, Events, Timeouts
  5. State Ownership and Data Flow Architecture
  6. Initial Implementation: Local Single-Node Version
  7. Summary

Chapter 4: BEAM Distribution Fundamentals

  1. Nodes and Naming: Short Names, Long Names, and DNS
  2. The Distribution Protocol: epmd, Handshake, and Connection Management
  3. Cookies, Authentication, and Node Security
  4. Remote Process Calls and Distributed Registries
  5. Links and Monitors Across Nodes
  6. When BEAM Distribution Is (and Isn’t) the Right Tool
  7. Summary

Chapter 5: Clustering and Service Discovery

  1. Static vs Dynamic Clusters: Trade-offs and Use Cases
  2. libcluster Strategies: DNS, Kubernetes, Consul, Etcd Integration
  3. Horde: Distributed ETS, Registry, and Application Controller
  4. Running BEAM on Kubernetes: Pods, Services, and Node Discovery
  5. Cluster Membership Changes: Joins, Leaves, Partitions
  6. Security in Dynamic Clusters: TLS, mTLS, and Cookie Management
  7. Summary

Chapter 6: State Management, Replication, and Partitioning

  1. State Placement: In-Memory vs Persistent vs External Stores
  2. ETS and DETS: High-Performance Local Storage
  3. Mnesia: Distributed Database on BEAM — Capabilities and Limits
  4. Replication Strategies: Primary-Backup, Multi-Master, Quorum-Based
  5. Partitioning and Sharding: Keys, Ranges, and Hot Spots
  6. Choosing Between BEAM-Native and External Databases
  7. Summary

Chapter 7: Consistency Models and Distributed Transactions

  1. The CAP Theorem in Practice: What It Actually Means for Your System
  2. Consistency Models: Strong, Sequential, Causal, Eventual
  3. Distributed Transactions on BEAM: Mnesia DTC and Alternatives
  4. Saga Patterns and Compensating Actions for Long-Running Workflows
  5. Idempotency Design: Keys, Tokens, and Deduplication
  6. Retries, Timeouts, and Circuit Breakers in Concurrent Systems
  7. Summary

Chapter 8: Fault Detection, Recovery, and Network Partitions

  1. Failure Detection: Monitors, Heartbeats, and Timeouts
  2. Handling Node Crashes and Process Restarts in a Cluster
  3. Network Partitions: Detection, Diagnosis, and Recovery Strategies
  4. Split-Brain Prevention: Quorums, Leaders, and Tiebreakers
  5. Graceful Degradation: Read-Only Modes, Queuing, and Fallbacks
  6. Designing for Partial Outages: Dependency Failure Handling
  7. Summary

Chapter 9: Event-Driven Architectures and Message Flows

  1. Event Sourcing vs Event-Carried State Transfer: Choosing Wisely
  2. Pub/Sub on BEAM: :rpc, GenEvent, Phoenix PubSub, and Beyond
  3. Queues and Workers: Oban, Broadway, and Custom Implementations
  4. Backpressure and Flow Control: Preventing Overwhelm
  5. RabbitMQ and External Message Brokers in BEAM Systems
  6. Event Ordering Guarantees and Their Costs
  7. Summary

Chapter 10: Testing Distributed Systems

  1. Unit Testing OTP Behaviours: Isolating Processes and Time
  2. Integration Testing Across Nodes: Spin-Up, Teardown, Cleanup
  3. Simulating Failures: Crashes, Delays, Partitions, and Slow Links
  4. Property-Based Testing for Concurrent and Distributed Code
  5. Chaos Engineering on BEAM: Tools and Practices
  6. Testing Idempotency, Retries, and Recovery Paths
  7. Summary

Chapter 11: Observability, Profiling, and Performance Tuning

  1. Structured Logging in Distributed BEAM Applications
  2. Metrics: Counters, Gauges, Histograms, and Cardinality
  3. Distributed Tracing with OpenTelemetry on BEAM
  4. Profiling: Scheduler Utilization, Reductions, and Hot Paths
  5. Memory Analysis: Mailbox Growth, Process Spawns, and Leaks
  6. GC Tuning and Performance Optimization Techniques
  7. Summary

Chapter 12: Deployment, Releases, and Operations

  1. Building Production Releases: Mix Release, Rebar3, and Docker
  2. Configuration Management: Environments, Overrides, and Hot Reloading
  3. Secrets Management: Vault, KMS, and Runtime Injection
  4. TLS Termination and Node-to-Node Encryption
  5. Rolling Upgrades and Zero-Downtime Deployments
  6. Backups, Disaster Recovery, and Multi-Region Strategies
  7. Summary

Chapter 13: BEAM in the Modern Ecosystem — Comparisons and Trade-offs

  1. BEAM vs Microservices: Monolithic Clusters vs Distributed Services
  2. BEAM vs Actor Systems: Akka, Orleans, and Other Frameworks
  3. BEAM in Kubernetes: Fit, Friction, and Best Practices
  4. When to Choose BEAM: Strengths, Weaknesses, and Honest Trade-offs
  5. Hybrid Architectures: BEAM as One Component of a Larger System
  6. Architecture Decision Checklist for Distributed Systems
  7. Summary

Conclusion

References

Get the free sample chapters

Click the buttons to get the free sample in PDF or EPUB, or read the sample online here

Download Sample PDFDownload Sample EPUB

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earned over $15 million writing, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub