High-Performance Python with Cython

A Practical Guide from Fundamentals to Production Systems

Introduction: Why Cython, Why Now

1. What You Are About to Learn
2. Who This Book Is For
3. How This Book Is Structured
4. A Note on Versions
5. The Philosophy of This Book

Chapter 1: The Performance Landscape: When Cython Matters

1. Why Python Is Slow: A Brief Diagnosis
2. The Performance Toolchain: Options and Trade-offs
3. When to Reach for Cython
4. How Cython Fits Into the Ecosystem
5. Case Study: Diagnosing a Real Bottleneck

Chapter 2: Getting Started: Installation, Setup, and First Steps

1. Installing Cython and Dependencies
2. Your First Cython Program
3. Build Systems: setup.py, pyproject.toml, and setuptools
4. The Compilation Pipeline Explained
5. Running and Importing Cython Modules

Chapter 3: Cython Syntax and Static Typing

1. From Python to Cython: The Minimal Changes
2. Declaring Types: Variables, Parameters, and Returns
3. Typed vs Untyped Code: What the Compiler Does Differently
4. Functions and Methods — cdef, cpdef, and def
5. Compiler Directives — Controlling Behavior with Annotations
6. Reading the Generated C Code
7. Common Pitfalls with Static Typing

Chapter 4: Memoryviews — Fast Array Access Without NumPy Dependency

1. What Are Memoryviews and Why They Matter
2. Declaring and Using Memoryviews
3. Ownership Models — Borrowed vs Owned Memory
4. Multidimensional Arrays and Broadcasting
5. Memoryviews vs NumPy Arrays vs Typed Memoryviews
6. Performance Example — Matrix Multiplication
7. Common Pitfalls with Memoryviews

Chapter 5: Loops, Data Structures, and Algorithmic Patterns

1. Optimizing Python Loops with Cython
2. Parallel Loops with prange
3. Native C Data Structures in Cython
4. Efficient String and Dictionary Operations
5. Common Algorithmic Patterns — Sorting, Searching, Accumulation

Chapter 6: Extension Types — Object-Oriented Cython

1. Defining Extension Types with cdef class
2. Properties and Methods — cdef vs cpdef vs def
3. Inheritance and Polymorphism in Cython
4. Memory Management for Custom Objects
5. When to Use Extension Types vs Python Classes

Chapter 7: Interfacing with C and C++ Libraries

1. Declaring External C Functions with cdef extern
2. Using pxd Files for Interface Definitions
3. Calling C Libraries — Complete Workflow
4. C++ Integration — Classes, Templates, and Namespaces
5. ABI Compatibility and Binary Distribution Challenges

Chapter 8: The GIL, Parallelism, and Concurrency

1. Understanding the GIL — What It Is and Why It Matters
2. Releasing the GIL with nogil
3. Thread Safety in Cython Code
4. Multiprocessing vs Threading with Cython
5. Practical Patterns for Parallel Computation

Chapter 9: Memory Management and Native Data Types

1. Native C Data Types in Cython
2. Manual Memory Allocation with malloc and free
3. Stack vs Heap Allocation Strategies
4. Buffer Protocol and Zero-Copy Access
5. Common Memory Bugs and How to Avoid Them

Chapter 10: Exception Handling and Error Management

1. Python Exceptions in Cython Code
2. C Errors and Return Codes
3. Crossing the Boundary — Converting Between Systems
4. noexcept and Performance Implications
5. Defensive Programming Patterns

Chapter 11: Profiling, Benchmarking, and Diagnosing Bottlenecks

1. Profiling Python Code to Find Hotspots
2. Cython-Specific Profiling Tools
3. Designing Valid Benchmarks
4. Interpreting Results and Avoiding Pitfalls
5. Regression Testing for Performance

Chapter 12: Optimization Strategies — From Incremental to Radical

1. The Optimization Ladder — A Systematic Approach
2. Compiler Optimization Flags and Their Impact
3. Code Restructuring for Cython Performance
4. Advanced Techniques — Inlining, Bounds Checking, Wraparound
5. When to Stop Optimizing

Chapter 13: Build Systems, Packaging, and Distribution

1. Modern Python Packaging with Cython
2. Building Wheels for Distribution
3. Managing C Dependencies in Packages
4. Cross-Compilation and Manylinux
5. Publishing to PyPI — Complete Workflow

Chapter 14: Debugging, Testing, and Maintainability

1. Debugging Cython Code — Tools and Techniques
2. Testing Compiled Extensions
3. Maintaining Readable Cython Code
4. API Design Principles for Cython Modules
5. Versioning and Breaking Changes

Chapter 15: Real-World Case Studies — Production Cython in Action

1. Case Study — High-Frequency Data Processing Pipeline
2. Case Study — Computational Geometry Library
3. Case Study — Integrating a C++ Machine Learning Library
4. Lessons from Production Deployments

Conclusion — The Cython Mindset

1. The Principles of Effective Cython Use
2. Looking Ahead — Cython in the Evolving Python Ecosystem
3. Your Next Steps
4. Final Thought

References