Leanpub Header

Skip to main content

Building High-Performance Network Stacks

From First Principles to Production-Grade Systems

Building High-Performance Network Stacks
This book is 100% completeLast updated on 2026-11-12

What does it take to build a network stack that can handle millions of packets per second? This book takes you from the fundamentals of networking to the inner workings of Linux, showing how protocols, kernel internals and low-level optimization techniques come together in real-world systems. Learn by building, measuring and tuning code in C and Rust.

Minimum price

$25.00

$35.00

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
EPUB
WEB
APP
263
Pages
About

About

About the Book

This book provides a complete, implementation-level understanding of modern networking systems. Beginning with the physics and mathematics that constrain all network communication, it progresses through the entire TCP/IP protocol stack, the internal architecture of the Linux kernel networking subsystem and the advanced techniques required to build systems that process millions of packets per second. You will learn how to implement networking protocols from scratch, optimize production services for throughput and tail latency, leverage kernel bypass frameworks like AF_XDP and DPDK and apply eBPF for programmable packet processing. The treatment is deliberately deep and practical: every major concept is grounded in protocol specifications, kernel source code, measured performance data and working implementations in C and Rust.

Author

About the Author

Steve Publications

Steve is a technology professional with more than 20 years of experience spanning software development, server infrastructure, cybersecurity, vulnerability research and reverse engineering. His work focuses on understanding complex systems, uncovering security weaknesses and developing practical solutions that make technology more secure, reliable and effective.

Through Steve Publications, he collaborates with experienced researchers, engineers and industry experts to create practical, technically rigorous books on various topics not limited to self-improvement, software development, cybersecurity, cloud computing, DevOps, artificial intelligence and enterprise technologies. His mission is to make complex technical subjects accessible through clear explanations, real-world examples and hands-on exercises that help IT professionals, students and technology enthusiasts build valuable, practical skills.

Steve Publications embraces AI as a supporting tool for translation, proofreading and editorial refinement, not as a substitute for original research, technical expertise, or hands-on experience. Some contributors choose to publish under pseudonyms to protect their privacy or maintain professional boundaries.

Behind every book is a commitment to technical accuracy, practical relevance and continuous improvement. Our publications are written, reviewed and maintained by experienced professionals, with support from a private community of more than 420 engineers and researchers. Rather than prioritizing volume or mass-produced AI-generated content, we focus on delivering reliable, useful resources that readers can apply in real-world environments.

We also welcome experienced professionals who want to share their knowledge by writing or reviewing books, particularly in specialized and emerging technology fields. If you have expertise to contribute, we invite you to get in touch and explore opportunities to collaborate.

Our commitment is simple: To deliver books that respect our readers' time, support their professional growth and provide lasting practical value. If a book does not meet your expectations, Leanpub offers a 60-day money-back guarantee.

Contents

Table of Contents

From First Principles to Production-Grade Systems

Introduction: Why Network Stack Depth Matters

  1. The thesis
  2. What this book assumes
  3. The scope
  4. How to use this book
  5. What this book is not
  6. The performance focus

Chapter 1: The Physics and Mathematics of Network Communication

  1. Signals, bandwidth and the Nyquist theorem
  2. Shannon capacity and fundamental limits
  3. Bit error rates and error detection codes
  4. Encoding schemes: NRZ, Manchester, 8b/10b, 64b/66b
  5. From analog to digital: modulation, sampling and clock recovery
  6. How physical constraints shape protocol design

Chapter 2: Ethernet: The Foundation of Modern Networking

  1. Ethernet history and design philosophy
  2. Frame format: preamble, MAC addresses, EtherType, payload, FCS
  3. CSMA/CD legacy and why modern Ethernet abandoned collision detection
  4. MAC addresses and switch forwarding
  5. Speed evolution: from 10 Mbps to 800 Gbps and beyond
  6. Why Ethernet endures

Chapter 3: Internet Protocol: Addressing and Routing

  1. IPv4 header structure and field semantics
  2. IP addressing, subnet masks, CIDR and hierarchical allocation
  3. The design goals of IP: simplicity, universality, best-effort semantics
  4. IPv6: header redesign, extension headers and adoption reality
  5. Routing fundamentals: longest prefix match, routing tables, BGP basics
  6. Fragmentation, MTU and Path MTU Discovery

Chapter 4: Transmission Control Protocol: Reliable Byte Streams

  1. The TCP header and every flag and field
  2. The three-way handshake and connection teardown
  3. Sequence numbers, acknowledgments and reliable reassembly
  4. Flow control via the receive window
  5. Retransmission strategies: timeout, fast retransmit, fast recovery
  6. State machine: every state and transition

Chapter 5: User Datagram Protocol and Transport Alternatives

  1. UDP header and semantics: why unreliable can be desirable
  2. Checksums and reliability without ordering
  3. QUIC architecture: multiplexing, 0-RTT, connection migration
  4. SCTP: multi-homing and multi-streaming for telephony
  5. DCCP and rate-limited unreliable delivery
  6. Choosing a transport: decision framework

Chapter 6: The Socket Interface and Application Programming

  1. Socket types: STREAM, DGRAM, RAW, SEQUENTIAL PACKET
  2. Domain addressing: AF_INET, AF_INET6, AF_UNIX, AF_PACKET
  3. The Berkeley sockets API: bind, listen, accept, connect, send, recv
  4. Blocking vs non-blocking semantics
  5. Socket options: tuning socket behavior
  6. The design philosophy of sockets and where it breaks down

Chapter 7: The Linux Kernel Networking Architecture

  1. Overview of net/ directory structure and subsystem layout
  2. The packet journey: ingress and egress flow through the kernel
  3. Protocol registration and family initialization
  4. Network namespaces and virtualization foundations
  5. The net_device structure and device abstraction layer
  6. sysctl and runtime tuning knobs

Chapter 8: The Complete Packet Lifecycle: Reception Path

  1. PHY reception and MAC frame validation
  2. NIC descriptor rings and DMA buffer management
  3. MSI-X interrupt vectors and affinity
  4. Interrupt moderation and coalescing
  5. NAPI: poll mode and its performance implications
  6. sk_buff allocation, initialization and metadata population
  7. GRO and packet coalescing in the driver

Chapter 9: The Complete Packet Lifecycle: Transmission Path

  1. Application write: copy from user to kernel
  2. Socket buffers: sk_buff construction and queuing
  3. TCP segmentation: MSS, SACK blocks, options
  4. IP header construction and checksum offload
  5. Queue discipline: fq_codel, pfifo_fast, HTB
  6. TSO/LRO/GSO offloads and their interactions
  7. TX completion paths and buffer recycling

Chapter 10: TCP Implementation in the Linux Kernel

  1. TCP socket structure: struct tcp_sock and its fields
  2. The receive path: tcp_v4_rcv and packet reassembly
  3. The send path: tcp_sendmsg and queue management
  4. Timers: RTO, keepalive, delay ACK, time-wait
  5. Retransmission queue management and fast retransmit
  6. Congestion control hooks: tcp_cong_ops and algorithm selection

Chapter 11: TCP Congestion Control Algorithms

  1. The congestion control problem and the AIMD principle
  2. Reno, NewReno: congestion window and slow start
  3. SACK and its transformations on recovery behavior
  4. CUBIC: the Linux default and its cubic growth curve
  5. BBR: bandwidth probing and model-based control
  6. Algorithm selection and interaction with middleboxes

Chapter 12: Networking Performance Measurement

  1. Metrics: throughput, latency, jitter, packet loss, CPU utilization
  2. Percentiles and tail latency: p50, p95, p99, p99.9 distributions
  3. Measurement tools: perf, ftrace, bpftrace, bpftool
  4. Network measurement: iproute2, ethtool, tcpdump, Wireshark
  5. Traffic generators: pktgen, Tcpreplay, TRex
  6. Methodology: baselines, controlled variables, reproducibility

Chapter 13: Processor-Architecture Foundations for Networking

  1. Cache hierarchy: L1, L2, L3, TLB and their costs
  2. NUMA topology and its impact on packet processing
  3. False sharing and cacheline padding in networking structures
  4. Memory barriers, acquire-release semantics, lock-free programming
  5. Branch prediction and its impact on hot packet paths
  6. SIMD acceleration for crypto and checksum computation

Chapter 14: High-Performance I/O: epoll, io_uring and Beyond

  1. The evolution of event notification: select, poll, epoll
  2. epoll internals: red-black trees, ready lists, EPOLLET
  3. io_uring architecture: submission/completion rings, fixed buffers
  4. Comparing latency and throughput across I/O interfaces
  5. Application design patterns for high-concurrency networking

Chapter 15: Zero-Copy Techniques and Memory Efficiency

  1. The cost of copying in the traditional path
  2. sendfile and splice: kernel-level file-to-socket transfer
  3. MSG_ZEROCOPY: zero-copy send
  4. UDP zerocopy and its limitations
  5. Pinning pages and getting memory to the NIC
  6. Buffer management strategies for high-performance stacks

Chapter 16: Kernel Bypass: DPDK, AF_XDP and Direct NIC Access

  1. Why kernel bypass: eliminating overhead for extreme performance
  2. DPDK architecture: EAL, poll mode drivers, memory pools
  3. Building a DPDK application: lifecycle and memory management
  4. AF_XDP: in-kernel ring-based socket interface
  5. Comparing DPDK, AF_XDP and XDP for production use

Chapter 17: eBPF in Networking: Programmability at Scale

  1. eBPF origins and virtual machine architecture
  2. BPF programs for networking: XDP, socket filters, TC, kprobe
  3. Map types and their networking applications
  4. Verifier constraints and safe kernel execution
  5. Observability with bpftrace and custom eBPF programs
  6. Production patterns: traffic steering, DDoS mitigation, load balancing

Chapter 18: RDMA and Hardware Offloading

  1. RDMA principles: zero-copy, kernel bypass, CPU offload
  2. InfiniBand protocol architecture and verbs API
  3. RoCEv2: RDMA over Converged Ethernet
  4. iWARP: RDMA over TCP and its trade-offs
  5. Hardware offloads: checksums, TSO, RSS, flow steering
  6. When RDMA makes sense: cluster interconnects and storage

Chapter 19: Programmable NICs, SmartNICs and DPU Architectures

  1. NIC evolution: from passive DMA to programmable engines
  2. SmartNIC architecture and compute fabric
  3. Data Processing Units (DPUs) and offloading servers
  4. P4 and programmable packet processing
  5. Production use cases: encryption offload, protocol acceleration
  6. Trade-offs: flexibility, cost, vendor lock-in

Chapter 20: Virtualization and Container Networking

  1. Virtual switches: Linux bridge, Open vSwitch, DPDK-based switches
  2. Tunnels and overlays: VXLAN, Geneve, GRE, IPIP
  3. Network namespaces and veth pairs
  4. Container networking: CNI, kube-proxy, service meshes
  5. SR-IOV: direct device assignment to guests
  6. Performance impact: overhead measurement and mitigation

Chapter 21: Designing a Network Stack from Scratch: Architecture

  1. Defining requirements: protocols, platforms, performance targets
  2. Layered architecture and module boundaries
  3. Buffer management design: owning, sharing and recycling packets
  4. Event-driven vs synchronous processing models
  5. Platform abstraction: porting across operating systems
  6. Testing strategy: unit tests, fuzzing, protocol verification

Chapter 22: Implementing the Data Link and Network Layers

  1. Packet buffer abstraction in C
  2. Ethernet frame parsing and construction
  3. Raw socket packet I/O on Linux
  4. ARP implementation with caching and timeout
  5. IPv4 header parsing, checksums and fragmentation handling
  6. Basic routing table and longest prefix match

Chapter 23: Implementing Transport Protocols: UDP and TCP

  1. UDP socket abstraction and datagram handling
  2. TCP connection table and lookup structures
  3. The TCP state machine: SYN_SENT, ESTABLISHED, FIN_WAIT_2, etc.
  4. Sequence number management and wraparound handling
  5. Retransmission logic: timers, RTO calculation, fast retransmit
  6. Congestion control: slow start, congestion avoidance, cubic
  7. Flow control and window updates

Chapter 24: Building a User-Space TCP/IP Stack

  1. Architecture choices: raw sockets vs kernel bypass
  2. Integration with a packet capture interface
  3. Complete socket API implementation
  4. Concurrency model for the stack
  5. Integration testing with real networks
  6. Performance comparison with kernel TCP

Chapter 25: Implementing an AF_XDP Packet Processing Engine

  1. Setting up AF_XDP sockets and ring buffers
  2. Batch receive and transmit for throughput
  3. Zero-copy receive mode configuration
  4. eBPF program for redirecting packets to AF_XDP
  5. Application logic: filtering, modification, forwarding
  6. Performance benchmarking and optimization

Chapter 26: Implementing a DPDK-Based Router

  1. DPDK setup: EAL initialization, memory pools, port configuration
  2. Poll loop architecture for multi-core forwarding
  3. LPM table implementation and lookup
  4. NAT translation table management
  5. Load balancing: consistent hashing and flow affinity
  6. Performance optimization and scalability testing

Chapter 27: Building a High-Performance HTTP Server

  1. Request/response parsing and state management
  2. Connection handling with io_uring or epoll
  3. Memory management and zero-copy file serving
  4. TLS acceleration and session resumption
  5. Benchmarking against nginx and specialized servers

Chapter 28: Implementing a Custom Load Balancer

  1. Layer 4 vs layer 7 load balancing architecture
  2. Health check implementation and failure detection
  3. Load balancing algorithms: round-robin, least connections, consistent hashing
  4. Connection tracking and session affinity
  5. Failover and high availability design
  6. Integration with service discovery and orchestration

Chapter 29: Optimizing a Linux TCP Service for Production

  1. Baseline measurement and profiling with perf
  2. Identifying hot paths with flame graphs
  3. Kernel parameter tuning: somaxconn, tcp_tw_reuse, tcp_max_syn_backlog
  4. Application optimization: buffer sizes, batching, batching writes
  5. eBPF-based observability for production diagnosis
  6. Continuous performance monitoring in production

Chapter 30: Production Networking: Reliability, Security and Operations

  1. Observability pillars for networking: metrics, logs, traces
  2. Packet loss and congestion in production environments
  3. Denial-of-service: types, detection, mitigation strategies
  4. Resource exhaustion: SYN floods, memory pressure, file descriptors
  5. Fault isolation: circuit breakers, timeouts, bulkheads
  6. Long-term evolution: protocol upgrades, security patches, capacity planning

Conclusion: The Future of Network Stack Engineering

  1. The lessons
  2. Where the field is heading
  3. What to focus on

References

  1. Protocol Specifications (RFCs)
  2. Ethernet and Link Layer
  3. Linux Kernel Networking Documentation
  4. Kernel Bypass and High-Performance Frameworks
  5. Congestion Control Research
  6. Performance Measurement and Profiling
  7. Hardware and NIC Documentation
  8. Virtualization and Container Networking
  9. RDMA and High-Performance Transports
  10. General Networking and Systems References

Get the free sample chapters

Click the buttons to get the free sample in PDF or EPUB, or read the sample online here

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub