eBPF for Systems Engineers

From First Principles to Production Deployments

Introduction: The eBPF Revolution

1. The Observability Problem in Modern Infrastructure
2. Before eBPF: What We Used to Do
3. What eBPF Gives You
4. Who Should Read This Book
5. How This Book Is Organized

Chapter 1: History and Evolution of BPF

1. The Original Berkeley Packet Filter
2. Linux Socket Filters: The First Step
3. The Birth of cBPF and eBPF
4. Alexei Starovoitov and the eBPF Project
5. Major Milestones: From 3.18 to 6.x

Chapter 2: Linux Kernel Architecture for eBPF

1. Kernel Space vs User Space: The Boundary
2. System Calls and Context Switching
3. Kernel Hooks and Tracing Infrastructure
4. Memory Management Basics for eBPF
5. The BPF Virtual Machine Architecture
6. How Programs Attach to the Kernel

Chapter 3: The eBPF Execution Model

1. The eBPF Instruction Set Architecture
2. Registers and Calling Conventions
3. The Fixed-Size Stack
4. Control Flow: Loops and Conditionals
5. Execution Contexts and Program Inputs
6. Deterministic Termination Requirements

Chapter 4: Writing Your First eBPF Programs

1. Setting Up the Development Environment
2. The LLVM/Clang Compilation Pipeline
3. A Hello World Tracepoint Program
4. Building with libbpf Skeletons
5. Running and Reading Output
6. Understanding the Generated Code

Chapter 5: The eBPF Verifier

1. Why the Verifier Exists
2. Safety Guarantees: Memory, Execution Time, Privileges
3. How the Verifier Works: State Machine Analysis
4. Common Verification Failures and Fixes
5. Loop Unrolling and Complexity Limits
6. Understanding Verifier Logs

Chapter 6: JIT Compilation

1. Why JIT Compilation Matters
2. The JIT Compilation Pipeline
3. Architecture-Specific JITs (x86, ARM, RISC-V)
4. JIT vs Interpreter Performance
5. Enabling, Disabling, and Tuning JIT
6. When Native Performance Is Critical

Chapter 7: eBPF Maps — Data Structures Between Kernel and User Space

1. What Maps Are and Why They Matter
2. Hash Maps: The Workhorse
3. Array Maps and Perf Arrays
4. Ring Buffers for High-Throughput Data
5. LRU and Per-CPU Variants
6. Map Performance and Capacity Planning

Chapter 8: Helper Functions — Talking to the Kernel

1. The Helper Function Calling Convention
2. Core Helpers: Time, Task, and Map Operations
3. Networking Helpers for Packets and Sockets
4. Tracing Helpers for kprobes and Tracepoints
5. Security and Permission Helpers
6. Choosing the Right Helper

Chapter 9: BTF and CO-RE — Kernel-Agnostic eBPF Programs

1. The Kernel Version Compatibility Problem
2. What BTF Is: Type Information in the Kernel
3. How CO-RE Relocation Works
4. Writing CO-RE-Compatible Programs
5. libbpf’s Role in CO-RE
6. Debugging CO-RE Relocations

Chapter 10: Tracing with kprobes, uprobes, and Tracepoints

1. kprobes: Instrumenting Kernel Functions
2. Finding Safe Probe Locations
3. uprobes for User-Space Tracing
4. Tracepoints: Stable Instrumentation Points
5. Raw Tracepoints for Maximum Performance
6. Choosing the Right Tracing Mechanism

Chapter 11: fentry, fexit, and Modern Function Tracing

1. How fentry/fexit Differ from kprobes
2. Performance Advantages of fentry/fexit
3. Attaching to Kernel Functions Safely
4. Return Value Tracing with fexit
5. Limitations and Gotchas
6. Migration Guide: From kprobes to fentry

Chapter 12: XDP — Extreme Packet Processing at Line Rate

1. The Networking Performance Problem
2. XDP Architecture and Execution Points
3. Driver, Hardware, and Generic Modes
4. Writing Your First XDP Program
5. Packet Manipulation and Actions
6. XDP in Production: Load Balancing and DDoS Protection

Chapter 13: Traffic Control, Socket Filters, and cgroups Programs

1. Traffic Control (TC) Hooks
2. Socket Filters for Application Traffic
3. cgroup Attach Points: v1 vs v2
4. Socket Operations Tracing
5. Container-Aware Networking with cgroups
6. Combining Multiple Network Hooks

Chapter 14: Security Monitoring with eBPF and LSM Hooks

1. Why Traditional Security Tools Fall Short
2. Syscall Monitoring and Auditing
3. File System Access Tracking
4. Process Lifecycle Monitoring
5. LSM Hooks: Deep Kernel Security Integration
6. Building a Security Monitoring Stack

Chapter 15: Observability, Profiling, and Performance Analysis

1. The Three Pillars: Metrics, Traces, Logs
2. Building Custom Observability Tools
3. CPU Profiling with eBPF
4. Latency Analysis and Histograms
5. Integration with Prometheus and OpenTelemetry
6. Production Monitoring Patterns

Chapter 16: Kubernetes and Container Visibility

1. Containers as a Tracing Challenge
2. Namespace-Aware eBPF Programs
3. Pod and Container Identification
4. Service Mesh Integration (Istio, Cilium)
5. Runtime Security in Kubernetes
6. Multi-Tenant Observability

Chapter 17: Production Operations — Deploying, Securing, and Maintaining eBPF

1. Security Model and Privilege Management
2. Deployment Strategies and Automation
3. Debugging Production eBPF Programs
4. Performance Tuning and Resource Limits
5. Version Compatibility and Rollout Planning
6. Monitoring Your eBPF Infrastructure

Conclusion: The Future of eBPF

1. Where eBPF Has Been
2. Emerging Capabilities and Active Development
3. Beyond Linux: Cross-Platform Prospects
4. Skills That Will Matter
5. Final Thoughts on Systems Engineering with eBPF

References