Leanpub Header

Skip to main content

Envoy Proxy: The Definitive Production Guide

Architecture, Configuration, Operations, and Advanced Patterns for Cloud-Native Infrastructure and AI Workloads

This book is 100% completeLast updated on 2026-07-21

Envoy Proxy is the foundation of modern cloud-native networking, powering service meshes, API gateways and AI infrastructure. This practical guide shows you how to design, deploy and operate Envoy in production with expert guidance, real-world configurations and advanced patterns for scalable, resilient systems.

Minimum price

$19.00

$29.00

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
EPUB
WEB
APP
289
Pages
About

About

About the Book

Envoy Proxy has become the foundational data plane for modern cloud-native infrastructure, powering service meshes, API gateways, edge proxies, and increasingly, AI inference platforms. This book provides a comprehensive, production-focused guide to understanding, deploying, and operating Envoy at scale. Beginning with core architectural concepts and progressing through advanced traffic management, security, observability, and extensibility patterns, every chapter delivers detailed explanations alongside real-world configuration examples. A dedicated section covers modern AI and LLM workloads, including multi-provider routing, intelligent model selection, streaming responses, cost optimization, and inference-specific observability. Whether you are a platform engineer building service meshes, an SRE operating critical infrastructure, or an architect designing AI-native systems, this book equips you with the depth needed to use Envoy effectively in production environments.

Author

About the Author

Steve Publications

Steve is a technology professional with more than 20 years of experience in software development, server infrastructure, cybersecurity, vulnerability research and reverse engineering. Throughout his career, he has designed, secured, analyzed and tested complex software and infrastructure, with a particular focus on understanding how systems fail and how they can be made more secure.

He currently works in the advanced research division of a leading cybersecurity company, where he performs vulnerability research alongside a team of experienced researchers and engineers. His work includes discovering security vulnerabilities, reverse engineering software and malware, analyzing emerging threats and developing new techniques to improve the security of modern computing environments.

Outside of work, Steve enjoys sharing knowledge with the technology community. He collaborates with researchers, industry experts and technology professionals to write practical books covering software development, cybersecurity, cloud computing, networking, DevOps, artificial intelligence and enterprise technologies. His books focus on practical learning through clear explanations, real-world examples and hands-on exercises. With more than two decades of industry experience, his goal is to help IT professionals, students and technology enthusiasts build useful skills and stay current in a rapidly changing industry.

We believe readers deserve to know how our books are created. Most of our authors are not native English speakers, so we use AI to help translate, proofread manuscripts, fix grammar, improve sentence structure and make technical explanations easier to read. AI is used as an editing tool only. It does not replace the research, technical knowledge or hands-on experience behind our books.

Some of our authors also prefer to remain anonymous for privacy or professional reasons. In those cases, we publish their work under the a different name. The author's name may be different, but the quality of the content and our review process remain the same.

Every book is written, reviewed and maintained by experienced technology professionals, with contributions from our private technical community of more than 400 engineers and researchers from Ukraine, Belarus and Russia. We spend far more time validating technical accuracy and keeping our content up to date than generating text.

If you look through the contents of our books, you'll see practical examples, detailed explanations and material that is regularly updated. Our goal is to publish books that professionals can actually rely on, not low-effort AI-generated content. If you ever feel that one of our books does not meet that standard, Leanpub offers a 60-day money-back guarantee. Feel free to request a refund if you are not satisfied with your purchase.

Contents

Table of Contents

Architecture, Configuration, Operations, and Advanced Patterns for Cloud-Native Infrastructure and AI Workloads

Introduction: Why Envoy Matters

  1. The Proxy Problem in Modern Infrastructure
  2. How Envoy Got Here: From Lyft to CNCF
  3. What Envoy Is (and What It Is Not)
  4. Who Should Read This Book
  5. How This Book Is Organized

Chapter 1: Architecture and Core Design Principles

  1. The Filter Chain: Envoy’s Central Abstraction
  2. Threading Model and Event Loop Architecture
  3. Memory Management and Resource Constraints
  4. Process Isolation and Sandboxing Patterns
  5. Comparing Envoy to NGINX, HAProxy, and Traefik

Chapter 2: Request Lifecycle and Networking Model

  1. Connection Lifecycle: From SYN to FIN
  2. Protocol Detection and Upgrading
  3. The Request Path Through Listeners and Filters
  4. Response Handling and Connection Reuse
  5. Graceful Shutdown and Draining

Chapter 3: Configuration Fundamentals — Listeners, Filter Chains, and Routes

  1. The Bootstrap Configuration File
  2. Listeners: Accepting Connections
  3. Filter Chains and Protocol Matching
  4. The HTTP Connection Manager
  5. Routes, Virtual Hosts, and Route Actions

Chapter 4: Clusters, Endpoints, and Service Discovery

  1. Clusters: Logical Upstream Groupings
  2. Endpoint Types and Resolution Modes
  3. Static and File-Based Discovery
  4. DNS-Based Service Discovery
  5. EDS and the xDS Protocol Suite

Chapter 5: Load Balancing Algorithms and Strategies

  1. Round Robin and Weighted Variants
  2. Least Connection Load Balancing
  3. Ring Hash Consistent Hashing
  4. Maglev and Random Strategies
  5. Locality-Aware and Multi-Zone Routing

Chapter 6: Reliability Patterns — Health Checking, Retries, and Circuit Breaking

  1. Active and Passive Health Checking
  2. Outlier Detection and Ejection Policies
  3. Retry Policies and Backoff Strategies
  4. Circuit Breaking and Resource Quotas
  5. Timeout Configuration and Deadline Propagation

Chapter 7: Traffic Management — Rate Limiting, Fault Injection, and Canaries

  1. Local Rate Limiting Filters
  2. Global Rate Limiting Service Integration
  3. Fault Injection for Chaos Testing
  4. Traffic Splitting and Canary Deployments
  5. Shadow Traffic and Request Mirroring

Chapter 8: Security — TLS, mTLS, Authentication, and Authorization

  1. TLS Termination and Configuration
  2. Mutual TLS (mTLS) in Service Meshes
  3. Certificate Management and Rotation
  4. JWT Authentication and Validation
  5. RBAC and External Authorization

Chapter 9: Observability — Logging, Metrics, and Distributed Tracing

  1. Access Logging: Formats and Destinations
  2. Structured Logging for Machine Parsing
  3. Metrics Collection with Prometheus
  4. Distributed Tracing Integration
  5. Building Observability Dashboards

Chapter 10: Extensibility — WebAssembly, Custom Filters, and Plugins

  1. The Filter Architecture Deep Dive
  2. WebAssembly (WASM) Filter Development
  3. Lua Scripting for Lightweight Extensions
  4. External Processing Filters
  5. Choosing the Right Extension Mechanism

Chapter 11: Dynamic Configuration and the xDS APIs

  1. The xDS Protocol Architecture
  2. Discovery Services: CDS, LDS, RDS, EDS, SDS
  3. Resource Versioning and Incremental Updates
  4. Control Plane Design Patterns
  5. Building a Custom xDS Control Plane
  6. Configuration Validation and Rollout Strategies

Chapter 12: Service Mesh Integration and Kubernetes Deployments

  1. Envoy as a Service Mesh Data Plane
  2. Istio Integration Patterns
  3. Sidecar Proxy Deployment Models
  4. Kubernetes Ingress and Gateway API
  5. Egress Gateway Patterns

Chapter 13: Edge Proxies, API Gateways, and Multi-Region Deployments

  1. Envoy as an Edge Proxy
  2. API Gateway Patterns and Capabilities
  3. Multi-Region Traffic Routing
  4. Global Load Balancing and Geo-Routing
  5. CDN Integration and Caching Strategies

Chapter 14: AI and LLM Workloads — Building Intelligent Inference Gateways

  1. The AI Gateway Pattern: Why Envoy Fits
  2. Routing to Multiple LLM Providers and Self-Hosted Models
  3. Intelligent Request Routing and Model Selection
  4. Prompt-Aware Routing Strategies
  5. A/B Testing, Canary Deployments, and Traffic Splitting for Models
  6. Streaming Responses and Long-Running Inference
  7. Caching Strategies for AI Inference
  8. Cost Optimization and Vendor Diversification
  9. Latency-Aware Routing and Multi-Region AI Deployments
  10. Observability for AI Inference Workloads
  11. Securing AI Workloads with Envoy
  12. Implementing AI Gateway Patterns with Vanilla Envoy

Chapter 15: Performance Tuning, High Availability, and Production Operations

  1. Performance Tuning: Threads, Buffers, and Connection Pools
  2. Scalability Patterns and Horizontal Scaling
  3. High Availability Architectures
  4. Debugging Envoy in Production
  5. Troubleshooting Common Issues
  6. Advanced Troubleshooting: Complex Incident Walkthroughs
  7. Migration Strategies from Other Proxies
  8. Production Best Practices
  9. Real-World Case Studies

Conclusion: The Future of Envoy and Proxy Architecture

  1. Key Principles Recap
  2. Emerging Patterns in Proxy Architecture
  3. Envoy’s Role in AI-Native Infrastructure
  4. Staying Current with the Ecosystem

References

Get the free sample chapters

Click the buttons to get the free sample in PDF or EPUB, or read the sample online here

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earned over $15 million writing, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub