Designing Decentralized Communication Protocols
- About this book
Introduction: The Decentralization Imperative
- A Conversation You Do Not Control
- The Promise of Decentralization
- The Central Thesis
- How This Book Is Organized
- What You Will Be Able to Do
- How to Read This Book
Chapter 1: Centralized Messaging Under the Microscope
- How Centralized Messaging Works Today
- Trust Concentration and the Operator as God
- Single Points of Failure and Cascading Outages
- Privacy, Surveillance and Metadata Exposure
- Scalability Walls and Operational Coupling
- The Case for Decentralization (and Its Costs)
Chapter 2: Foundations of Distributed Systems
- Partial Failure and the Death of the Perfect System
- Time, Clocks and Causality
- Consistency Models from Strong to Eventual
- CAP and PACELC: The Real Trade-Offs
- Consensus: When It Is Needed and When It Is Not
- Failure Detectors, Heartbeats and Membership
Chapter 3: Peer-to-Peer Networking and Network Overlays
- The P2P Spectrum: From Pure to Hybrid
- Distributed Hash Tables (Kademlia, Chord)
- Gossip and Anti-Entropy Protocols
- Overlay Construction, Churn and Topology Control
- Structured vs. Unstructured Discovery
Chapter 4: Number Theory and Elliptic Curves
- Finite Fields and Modular Arithmetic
- The Elliptic Curve Group Law
- Discrete Logarithms and the Difficulty Assumption
- Curve Selection: Ed25519, Curve25519, X448 and NIST
- Side Channels and Constant-Time Implementation
Chapter 5: Public-Key Cryptography, Signatures and Key Management
- Asymmetric Encryption vs. Key Encapsulation
- Digital Signatures: ECDSA and EdDSA
- Identity Keys, Device Keys and Trust Models
- Key Derivation Functions and PRFs
- Key Rotation, Expiry and Lifecycle
- Hashing, MACs and AEAD
Chapter 6: Key Exchange and the Double Ratchet
- Diffie-Hellman and Authenticated Key Exchange
- The Noise Protocol Framework
- X3DH: Asynchronous Key Agreement
- The Double Ratchet: Symmetric-Key Ratcheting
- Forward Secrecy, Post-Compromise Security and Continuity
- Session Establishment and Key Scheduling
Chapter 7: Group Messaging and Collaborative Cryptography
- Why Groups Are Hard
- Tree-Based Group Encryption and the MLS Protocol
- Key Distribution, Ratcheting and Member Changes
- Backward and Forward Secrecy in Groups
- Message Ordering, Fragmentation and Large Groups
Chapter 8: Decentralized Identity and Addressing
- The Identity Problem in P2P
- Decentralized Identifiers (DIDs) and Verifiable Credentials
- Key-Based Identities and Human-Readable Aliases
- Naming Systems: ENS, Nostr, ActivityPub Actors
- Verifying Identity: Fingerprints, Social Trust and Notarization
Chapter 9: Network Topologies, NAT Traversal and Relays
- The Last Mile: NATs, Firewalls and Cone Types
- STUN, TURN and ICE: The Traversal Toolbox
- Hole Punching and Direct Connectivity
- Relay Networks, Mix Layers and Censorship Circumvention
- Transport Choice: TCP, QUIC, WebTransport and UDP
Chapter 10: Transport, Framing and Connection Lifecycle
- Framing and Delimiting on Stream and Datagram Transports
- Multiplexing and Flow Control
- Connection Lifecycle: Establishment, Keepalive, Teardown
- Graceful Degradation and Reconnection
- Backpressure and Resource Limits
Chapter 11: Reliable Asynchronous Messaging Semantics
- Delivery Guarantees: At-Most-Once, At-Least-Once, Exactly-Once
- Acknowledgements, Retransmission and Timeouts
- Ordering: FIFO, Causal and Total Order
- Deduplication and Idempotency
- Sequence Numbers, Gaps and Recovery
Chapter 12: Offline Operation, Store-and-Forward and Synchronization
- Offline-First Design and Local Queues
- Store-and-Forward via Relays and Peers
- Anti-Entropy Synchronization and Merkle Trees
- Conflict Resolution and Last-Writer-Wins
- Synchronizing After Prolonged Disconnection
Chapter 13: Reference Architecture: System Design and Threat Model
- Design Principles and Non-Goals
- Trust Boundaries and the Threat Model
- Security Invariants and Requirements
- Component Responsibilities and Architecture Diagram
- Node Lifecycle and States
Chapter 14: Protocol Specification: Wire Formats, Envelopes and Message Types
- Version Negotiation and Capability Exchange
- Message Envelopes and Addressing
- Wire Formats: Protobuf Schemas and Encodings
- Cryptographic Handshakes on the Wire
- Error Codes and Error Handling
Chapter 15: Protocol Specification: Routing, State Machines and Delivery
- Routing Rules and Forwarding Logic
- State Machines for Sessions, Channels and Peers
- Store-and-Forward Delivery Flow
- Synchronization and Catch-Up Protocols
- Backward Compatibility and Evolution Strategy
Chapter 16: Metadata Privacy, Anonymity and Censorship Resistance
- Why Metadata Matters
- Traffic Analysis and Its Defenses
- Onions, Mixes and Plausible Deniability
- Censorship Circumvention Techniques
- Designing for Low-Profile Metadata
Chapter 17: Threat Model, Attacks and Defenses
- Threat Modeling Methodology
- Sybil Attacks and Spam Prevention
- Denial-of-Service at Every Layer
- Replay, Impersonation and Message Injection
- Compromised Endpoints and Key Compromise Recovery
- Mapping Defenses to Threats
Chapter 18: Implementation Foundations: Stack, Repository and Core Primitives
- Technology Choices and Rationale
- Repository Structure and Module Boundaries
- Cryptographic Primitives Layer
- Identity and Key Store
- Serialization and Configuration
Chapter 19: Building the Peer Node: Transport, Handshake and Reliable Delivery
- Async Runtime and Connection Manager
- Noise Handshake Implementation
- Framed Transport and Multiplexing
- Reliable Delivery Engine: Acks, Sequences and Retries
- Local Store, Offline Queue and Sync
Chapter 20: Scaling Up: Groups, Multi-Device and Production Hardening
- Group Messaging Integration
- Multi-Device and Secondary Devices
- Discovery and Routing Service
- Hardening: Rate Limits, Resource Caps, Telemetry
- End-to-End Example: A Working Conversation
Chapter 21: Testing, Verification and Security Validation
- Testing Pyramid: Unit, Integration, End-to-End
- Property-Based Testing and Fuzzing
- Cryptographic Test Vectors and Interoperability
- Fault Injection and Concurrency Testing
- Load Testing, Benchmarking and Security Audits
Chapter 22: Deployment, Operations and Performance
- Development, Staging and Production Environments
- Self-Hosted Infrastructure and Node Topologies
- Monitoring, Logging and Observability
- Upgrades, Migration and Backward Compatibility
- Performance Engineering and Capacity Planning
Conclusion: The Road Ahead
- What We Have Built
- Where It Falls Short
- Open Research Problems
- Interoperability and Future Evolution
- A Final Word