CPU Performance Engineering

From Architecture to Optimization - A Practical Guide to Making Software Faster

Introduction: The Machine Beneath the Abstraction

1. Why Performance Still Matters
2. Why This Book Is Different
3. How to Read This Book
4. About the Repository Analysis
5. What This Book Will Not Do
6. Getting Started
7. Chapter Summary

Chapter 1: Why Performance Matters - and Why It Is Hard

1. The End of Free Lunch - Moore’s Law and the Performance Wall
2. Latency vs. Throughput - Two Different Problems
3. The Cost of Performance Mistakes - Real-World Economics
4. Why Intuition Fails - Counterintuitive Behavior of Modern CPUs
5. A Scientific Approach - The Performance Engineering Methodology
6. What This Book Will and Will Not Cover
7. Chapter Summary

Chapter 2: Inside the Modern CPU - A Mental Model

1. The Von Neumann Architecture and Its Limitations
2. The Fetch-Decode-Execute Cycle - And Why It Is Not That Simple
3. Cores, Threads and SMT - Understanding Compute Resources
4. The Processor as a Factory - Pipeline, Buffer and Queue Analogies
5. Why Clock Speed Is Only Part of the Story
6. Instruction Set Architectures - x86-64 and ARM64 Compared
7. Chapter Summary

Chapter 3: Instructions and the Instruction Set

1. What Is an Instruction - Anatomy and Encoding
2. RISC vs. CISC - History, Philosophy and Modern Reality
3. Micro-ops - How x86 Instructions Become Executable Work
4. Instruction Latency vs. Instruction Throughput - Two Different Numbers
5. Addressing Modes and Their Performance Cost
6. Reading Agner Fog and Processor Reference Manuals
7. Chapter Summary

Chapter 4: Pipelines and Superscalar Execution

1. The Pipeline - Stages, Hazards and Throughput
2. Superscalar Design - Issuing Multiple Instructions Per Cycle
3. Execution Ports and Resource Constraints
4. Pipeline Depth - Benefits and the Cost of Mispredictions
5. Frontend vs. Backend Bottlenecks
6. Architectural Implications - How to Write Code for Superscalar CPUs
7. Chapter Summary

Chapter 5: Out-of-Order Execution and Instruction-Level Parallelism

1. The Problem - Dependencies and Stall Cycles
2. Out-of-Order Execution - Scoreboarding and Tomasulo’s Algorithm
3. The Reorder Buffer and Commit Stage
4. True Dependencies, Anti-Dependencies and Output Dependencies
5. Instruction-Level Parallelism - Measuring and Exploiting It
6. What You Can Control - Software Techniques to Increase ILP
7. Chapter Summary

Chapter 6: Branch Prediction and Speculative Execution

1. The Branch Problem - Why Control Flow Kills Throughput
2. Branch Prediction Mechanisms - Static and Dynamic Predictors
3. Speculative Execution - Running Before You Know
4. Branch Misprediction Penalty - Quantifying the Cost
5. Writing Predictable Code - Data-Dependent Branches and Conditional Moves
6. Security Implications - Spectre, Meltdown and Their Microcode Fixes
7. Chapter Summary

Chapter 7: Instruction Scheduling and Code Layout

1. Instruction Fetch - Width, Alignment and Cache Lines
2. Code Alignment and Its Impact on Pipeline Performance
3. Software Pipelining - Manual Instruction Scheduling
4. Compiler Scheduling - What Compilers Do and What They Cannot See
5. Loop Layout and Branch Placement
6. Reading and Optimizing Assembly Output
7. Chapter Summary

Chapter 8: Caches - The Memory Hierarchy

1. The Memory Wall - Why Main Memory Is Too Slow
2. Cache Levels - L1, L2, L3 and Their Roles
3. Cache Internals - Lines, Sets, Ways and Associativity
4. Cache Access Types - Hits, Misses and Conflict Misses
5. False Sharing - A Subtle Performance Killer
6. Cache-Friendly Programming - Locality, Stride and Data Structures
7. Chapter Summary

Chapter 9: Cache Coherency and Multi-Core Communication

1. The Cache Coherency Problem - Multiple Copies, One Truth
2. MESI and MOESI Protocols - How CPUs Keep Caches in Sync
3. The Snooping Bus and Directory-Based Coherency
4. Coherency Traffic - Measuring Its Impact
5. Writing Cache-Coherency-Aware Code
6. NUMA and Cache Coherency in Large Systems
7. Chapter Summary

Chapter 10: Virtual Memory and Translation Lookaside Buffers

1. Virtual Memory - Isolation, Abstraction and Performance Cost
2. Page Tables and Page Faults - Walking the Tree
3. TLBs - Purpose, Hierarchy and Miss Penalties
4. TLB Shootdowns - The Multi-Core Cost
5. Huge Pages - Reducing Translation Overhead
6. Memory Mapping Strategies for Performance
7. Chapter Summary

Chapter 11: Memory Access Patterns and Bandwidth

1. Sequential vs. Random Access - The Performance Gap
2. Spatial and Temporal Locality - Exploiting Hardware Prefetching
3. Memory Bandwidth - Measuring and Maximizing It
4. Prefetching - Hardware and Software Techniques
5. Data Layout Transformations - AOS vs. SOA and Beyond
6. Benchmarking Memory Performance - Real Numbers on Real Hardware
7. Chapter Summary

Chapter 12: SIMD and Vectorization

1. SIMD Fundamentals - Doing More Work Per Instruction
2. x86 SIMD Families - SSE, AVX, AVX2, AVX-512
3. ARM NEON and SVE - Vectorization on ARM
4. Auto-Vectorization - When Compilers Succeed and Fail
5. Manual Vectorization - Intrinsics and Assembly
6. Common Pitfalls - Alignment, Reductions and Control Flow
7. Chapter Summary

Chapter 13: Multithreading and Hardware Concurrency

1. Hardware Multithreading - Simultaneous Multithreading (SMT)
2. Software Threads - Mapping to Hardware
3. Scaling Performance - Amdahl’s Law and Gustafson’s Law
4. Thread Affinity and NUMA Awareness
5. Load Balancing - Dynamic and Static Scheduling
6. When Multithreading Hurts - Resource Contention and False Sharing
7. Chapter Summary

Chapter 14: Synchronization and Atomic Operations

1. Atomic Operations - The Hardware Foundation
2. Memory Ordering and Barriers - Sequential Consistency vs. Weaker Models
3. Locks - Mutexes, Spinlocks and Their Overhead
4. Lock-Free and Wait-Free Algorithms - Possibilities and Limits
5. Read-Write Locks and Fine-Grained Locking
6. Lock Coarsening, Elimination and False Sharing
7. Chapter Summary

Chapter 15: Performance Counters and Hardware Monitoring

1. What Are Performance Counters - Hardware Events and Counters
2. Intel PMU, AMD IBS and ARM PMU - Vendor-Specific Features
3. Key Metrics - IPC, Cache Miss Rates, Branch Mispredictions
4. Using perf - The Linux Performance Toolkit
5. Statistical Sampling vs. Precise Events
6. Designing Counter-Based Experiments
7. Chapter Summary

Chapter 16: Profiling - Understanding What Your Code Actually Does

1. Sampling vs. Instrumentation - Trade-Offs
2. Flame Graphs - Visualizing Execution Profiles
3. CPU Profiling - Finding Hot Paths
4. Cache Profiling - Identifying Memory Bottlenecks
5. Branch Profiling - Understanding Control Flow Performance
6. Profiling in Production - Sampling, Tracing and Overhead
7. Chapter Summary

Chapter 17: Benchmarking - Measuring What Matters

1. Microbenchmarks vs. Macrobenchmarks - When to Use Each
2. Experimental Design - Controls, Variables and Repetition
3. Statistical Variation - Warmup, Cooling and Outliers
4. Measuring Latency - Precision Timers and Statistical Analysis
5. Measuring Throughput - Sustained Performance and Sustaining It
6. Common Benchmarking Mistakes - And How to Avoid Them
7. Chapter Summary

Chapter 18: Compiler Optimization - Working With the Compiler

1. Compiler Optimization Pipeline - What Happens Under the Hood
2. Optimization Levels - What -O2 and -O3 Actually Do
3. Compiler Flags - Tuning for Your Target
4. Reading Optimization Reports - -Rpass and -Rpass-missed
5. LTO and PGO - Link-Time and Profile-Guided Optimization
6. When the Compiler Fails - Inline Assembly, Intrinsics and Hand-Tuning
7. Chapter Summary

Chapter 19: CPU Frequency, Power Management and Thermal Limits

1. Turbo Boost and Dynamic Frequency Scaling - How It Works
2. Power Limits - TDP, PL1, PL2 and Their Real-World Impact
3. Thermal Throttling - When Heat Becomes the Bottleneck
4. Measuring Real Frequency - Is Your CPU Running at Full Speed?
5. Disabling Power Management for Benchmarks
6. Performance per Watt - Optimizing for Efficiency, Not Just Speed
7. Chapter Summary

Chapter 20: Operating System Interactions and System Configuration

1. Scheduler Effects - Context Switches and Priority Inversion
2. Interrupt Handling - Softirqs, Hardirqs and Latency
3. CPU Isolation - Isolating Cores for Latency-Sensitive Workloads
4. Kernel Configuration - Tuning for Performance
5. cgroups and Resource Controls - Containment and QoS
6. Virtualization Overhead - When Your VM Hurts Performance
7. Chapter Summary

Chapter 21: Performance Analysis Methodologies - A Scientific Approach

1. The Performance Engineering Workflow - Measure, Hypothesize, Optimize, Validate
2. Establishing a Baseline - Reproducibility and Regression Prevention
3. Formulating Hypotheses - From Symptoms to Root Causes
4. The Bottleneck Taxonomy - CPU-Bound, Memory-Bound, I/O-Bound
5. Prioritizing Optimizations - Impact vs. Effort
6. Validating Results - Avoiding Premature Celebration
7. Chapter Summary

Chapter 22: Case Studies in Performance Engineering

1. Case Study 1 - Optimizing a Hash Table: Cache Locality and False Sharing
2. Case Study 2 - Vectorizing Image Processing: From Naive to AVX-512
3. Case Study 3 - Reducing Lock Contention in a High-Throughput Server
4. Case Study 4 - NUMA-Aware Memory Allocation in a Database
5. Lessons Learned - Common Patterns Across Cases
6. Chapter Summary

Chapter 23: Critical Analysis of the CPU-Performance-Engineering Repository

1. Repository Overview - Structure and Scope
2. Strengths - What It Gets Right
3. Weaknesses and Omissions - What Is Missing or Under-Explained
4. Accuracy Assessment - Comparing to Primary Sources and Reference Manuals
5. Outdated or Misleading Material - Specific Examples and Corrections
6. How to Use This Repository - Recommendations and Caveats
7. Chapter Summary

Conclusion: The Performance Engineering Mindset

1. The Key Principles - A Condensed Checklist
2. What Changes and What Stays the Same
3. Future Trends - Chiplets, AI Accelerators and New Memory Technologies
4. Continuing Your Journey - Resources and Communities
5. Final Thoughts

References