SIMD Programming for Modern Software Engineers

From Fundamentals to Advanced Optimization

Introduction

1. What This Book Is About
2. How This Book Is Organized
3. What You Will Be Able to Do After Reading
4. Prerequisites and Conventions
5. A Note on What This Book Does Not Cover

Chapter 1: The Case for SIMD; Why Data-Level Parallelism Matters

1. A Simple Example: Tenfold Speedup in Three Lines
2. Where Performance Is Actually Spent Today (Amdahl’s Law in Practice)
3. Forms of Parallelism: Threads, Pipelines, Superscalar, SIMD, GPUs; How They Relate
4. The Memory Wall and Why Vectorization Helps
5. When SIMD Wins, When It Does Not, and When to Skip It Entirely
6. A Practical Decision Framework Before You Optimize

Chapter 2: CPU Architecture Essentials for SIMD Programmers

1. The Modern CPU as a Dataflow Machine (Not a Von Neumann Diagram)
2. Registers, Execution Ports, and Instruction Scheduling
3. Pipelining, Superscalar Execution, and Instruction-Level Parallelism
4. The Memory Hierarchy: Registers, L1/L2/L3 Cache, RAM; Latencies That Matter
5. Branch Prediction and Why SIMD Loves Straight-Line Code
6. Clock Frequency, Power Budgets, and Turbo Boost Reality

Chapter 3: Understanding Vector Execution from First Principles

1. Scalar Versus Vector: The Same Operation, Different Granularity
2. What Is a Lane and How Data Lives Inside a Vector Register
3. Loading, Transforming, Storing: The Basic SIMD Execution Model
4. Endianness, Alignment, and Memory Layout Fundamentals
5. Instruction Encoding Widths (128-bit, 256-bit, 512-bit) and What They Mean
6. Throughput Versus Latency in Vector Operations

Chapter 4: Compiler Auto-Vectorization and Vectorization Reports

1. How Modern Compilers Discover Vectorization Opportunities
2. Reading Compiler Vectorization Reports (GCC, Clang, MSVC, ICC)
3. Writing Auto-Friendly Code: Loop Structure, Alignment, Dependencies
4. Common Reasons Auto-Vectorization Fails and How to Fix Them
5. Floating-Point Semantics, Reassociation, and Precision Trade-offs
6. When to Trust the Compiler and When to Take Over Manually

Chapter 5: Writing Your First SIMD Code; Intrinsics and Portable Approaches

1. Development Environment Setup: Compilers, Flags, Debugging Tools
2. Your First Intrinsic: A Complete Scalar-to-SIMD Walkthrough
3. How Intrinsics Map to Assembly (and Why That Matters)
4. Portable SIMD: C++20 std::simd Proposals, ISPC, and Libraries
5. Language Support Beyond C/C++: Rust, Go, Zig, and Others
6. Feature Detection: CPUID, IsA Detection, and Runtime Dispatch Basics

Chapter 6: Memory, Alignment, and Data Layout for SIMD

1. Aligned Versus Unaligned Loads: Costs, Faults, and Mitigations
2. Structure of Arrays (SoA) Versus Array of Structures (AoS): The Fundamental Choice
3. Data Rearrangement: When and How to Transpose Between Layouts
4. Gather and Scatter: Flexible but Expensive Memory Access
5. Prefetching and Streaming Loads for Bandwidth-Bound Code
6. Cache Line Effects, False Sharing, and Alignment Strategy

Chapter 7: Vector Operations; Loading, Transforming, Comparing, and Reducing

1. Load Patterns: Streaming, Non-Temporal, Masked, and Gather Loads
2. Arithmetic Operations: Addition, Subtraction, Multiplication, Division Avoidance
3. Comparison and Masking: Generating Masks from Vector Comparisons
4. Shuffles, Permutations, and Cross-Lane Data Movement
5. Reductions: Summing, Finding Min/Max, Horizontal Operations
6. Widening, Narrowing, Saturation, and Type Conversions

Chapter 8: x86 SIMD; SSE Through AVX-512

1. The x86 SIMD Evolution: SSE, SSE2, SSE3, SSSE3, SSE4.x Timeline
2. AVX and AVX2: 256-Bit Vectors, VEX Encoding, and Integer Multiplies
3. AVX-512: Masking, EVEX Encoding, and the Frequency Penalty Debate
4. Instruction Selection Strategy: Which Set to Target on Modern Hardware
5. Microarchitecture Differences: Intel Versus AMD SIMD Execution
6. ABI Considerations and Calling Conventions for SIMD Code

Chapter 9: Arm SIMD; NEON and SVE/SVE2

1. NEON Basics: Register File, Data Types, and Instruction Patterns
2. Programming NEON: Intrinsics, Assembly, and Portable Approaches
3. SVE and SVE2: Predicate-Based Vectoring and Variable Length
4. Key Differences Between Arm and x86 SIMD Design Philosophies
5. Mobile Versus Server: Power Constraints and Performance Targets
6. Cross-Platform SIMD: Writing Code That Runs on Both Families

Chapter 10: Performance Analysis and Optimization Methodology

1. Building Trustworthy Microbenchmarks: Pitfalls and Best Practices
2. Reading Assembly Output: Disassemblers and Compiler Explorer
3. Hardware Performance Counters: What to Measure and How
4. Identifying Compute-Bound Versus Memory-Bound Kernels
5. The Roofline Model and Where Your Code Lives
6. Iterative Optimization: Profile, Hypothesize, Implement, Measure

Chapter 11: Advanced Vectorization Patterns and Software Pipelining

1. Loop Unrolling and Multiple Accumulators for Latency Hiding
2. Software Pipelining by Hand: Overlapping Independent Operations
3. Predicated SIMD: AVX-512 Masks and Arm SVE Predicates
4. Branch Elimination Through Vector Comparison and Blending
5. Tail Handling Strategies for Non-Multiple-of-Vector-Length Data
6. Instruction Scheduling for Maximum Port Utilization

Chapter 12: Case Studies in SIMD Optimization; Part I (Numerical and Array Processing)

1. Case Study 1: Image Convolution and Pixel Processing
2. Case Study 2: Matrix Multiplication Kernel Optimization
3. Case Study 3: Signal Processing ; FFT Butterfly Operations
4. Case Study 4: Scientific Computing ; N-Body Force Calculation

Chapter 13: Case Studies in SIMD Optimization; Part II (Systems and Data Processing)

1. Case Study 5: Text Parsing, String Search, and Pattern Matching
2. Case Study 6: Serialization and Binary Encoding Acceleration
3. Case Study 7: Hashing Functions and Cryptographic Primitives
4. Case Study 8: Database Predicate Evaluation and Vectorized Execution

Chapter 14: Building, Testing, and Maintaining SIMD Code in Production

1. Multiversioning Strategies: Compile-Time Versus Runtime Dispatch
2. Testing Across Architectures and Vector Widths
3. Debugging Optimized SIMD Code: Tools and Techniques
4. Numerical Reproducibility and Floating-Point Determinism
5. ABI Stability and Binary Compatibility with SIMD Libraries
6. Documentation, Code Review, and Knowledge Transfer for SIMD Teams

Chapter 15: The SIMD Optimization Workflow; A Repeatable Methodology

1. The Complete Optimization Decision Tree: Scalar, Auto, Intrinsics, Libraries, GPU
2. Step-by-Step SIMD Optimization Checklist for New Codebases
3. Integrating SIMD Work Into CI/CD and Performance Regression Testing
4. When to Stop Optimizing: Diminishing Returns and Maintenance Cost
5. Reference: SIMD Terminology Glossary
6. Reference: Common SIMD Operations Cheat Sheet
7. Reference: Compiler Flags and Feature Detection Summary
8. Reference: Architecture Capability Quick Reference
9. Further Resources and Authoritative References

Conclusion

References