Writing GPU Kernels with CUDA Rust

A Practical Guide to High-Performance GPU Computing from Rust

Introduction

Chapter 1: Why GPU Computing, Why Rust

1. The Parallel Computing Revolution
2. Where GPUs Excel and Where They Do Not
3. The State of GPU Programming in 2025
4. Why Rust for GPU Development
5. How This Book Is Organized

Chapter 2: GPU Architecture and Execution Model

1. From CPU to GPU: A Hardware Perspective
2. Streaming Multiprocessors and Execution Units
3. The Thread Block and Grid Execution Model
4. Warp Execution and SIMT
5. Memory Hierarchies on the GPU
6. How a Kernel Launch Traverses the Hardware

Chapter 3: CUDA Fundamentals: The C++ Reference Model

1. The CUDA C++ Kernel Syntax
2. Host Code Versus Device Code
3. Launch Configuration and Execution Dimensions
4. Device Memory Allocation and Data Transfer
5. A Complete CUDA C++ Example Walked Through
6. What CUDA Rust Will Build Upon

Chapter 4: Introducing CUDA Rust: History and Vision

1. The Road to GPU Programming in Rust
2. NVIDIA’s CUDA Rust Initiative
3. The Two Tracks: Native and C Bindings
4. Design Goals and Philosophy
5. Current Status, Stability, and Versioning
6. What CUDA Rust Is Not

Chapter 5: Toolchain Setup and Project Configuration

1. Prerequisites: Hardware, CUDA Toolkit, and Drivers
2. Installing the CUDA Rust Compiler and Dependencies
3. Project Layout with cargo-cuda
4. Build Configuration and Environment Variables
5. Your First CUDA Rust Project
6. Troubleshooting Common Setup Issues

Chapter 6: The CUDA-Rust API Surface

1. The core_cuda and cuda-rs Crates
2. Device Pointer Types and Abstractions
3. The Kernel Trait and Launch Mechanics
4. Memory Allocation and Transfer APIs
5. Utility Types: Dimensions, Streams, Events
6. How the API Maps to CUDA C++ Concepts

Chapter 7: Device Code Fundamentals

1. Marking Code as Device Code
2. Computing Thread and Block Indices
3. Your First Kernel: Vector Addition
4. Understanding the Launch Dimensions
5. Compilation: How Device Code Becomes PTX and SASS
6. Safety Considerations in Device Code

Chapter 8: Memory Management: Allocation and Transfer

1. Device Memory Allocation Patterns
2. Host-to-Device and Device-to-Host Transfers
3. Pinned Memory and Asynchronous Transfers
4. The Cost Model of Data Movement
5. Memory Safety at Host and Device Boundaries
6. Complete Example: Matrix Initialization and Transfer

Chapter 9: Kernel Launch Patterns

1. Launch Configuration and Grid Size Calculations
2. Occupancy and Heuristics for Block Size
3. Streams and Concurrent Execution
4. Synchronous Versus Asynchronous Launches
5. Kernel Chaining and Dependencies
6. Example: Multi-Kernel Pipeline

Chapter 10: Shared Memory and Block-Level Optimization

1. Shared Memory: Purpose and Performance
2. Declaring and Using Shared Memory in Kernels
3. The Matrix Transpose Optimization
4. Warp Coalescing and Memory Access Patterns
5. A Reduction Kernel with Shared Memory
6. Barriers and Synchronization Within a Block

Chapter 11: Global Memory and Data Access Patterns

1. Global Memory Architecture and Caching
2. Coalesced Versus Uncoalesced Access
3. Bank Conflicts and Shared Memory Layout
4. Texture and Constant Memory
5. Designing Kernels for Memory-Bound Workloads
6. Example: Image Filter Kernel

Chapter 12: Synchronization and Coordination

1. Thread-Level Synchronization: __syncthreads
2. Warp-Level Primitives
3. Cross-Block Synchronization Limitations
4. Stream-Based Synchronization
5. Host-Device Synchronization and Events
6. Deadlock and Starvation Pitfalls

Chapter 13: Ownership, Lifetimes, and GPU Safety

1. Ownership Semantics for Device Resources
2. Lifetimes at the Host-Device Boundary
3. RAII for CUDA Handles
4. Aliasing and the Device Pointer Model
5. Avoiding Use-After-Free on the GPU
6. Safe Abstractions Over Unsafe CUDA Operations

Chapter 14: Error Handling and Robust GPU Code

1. CUDA Error Types and Categories
2. Propagating Errors in Host Code
3. Detecting Kernel Launch Errors
4. Runtime Versus Compile-Time Errors
5. Defensive Patterns for Production Kernels
6. Logging and Diagnostic Infrastructure

Chapter 15: Numerical Computing on the GPU

1. Float32, Float64, and Mixed Precision
2. GPU Floating Point Semantics
3. Denormals, NaN, and Infinity
4. Atomic Operations and Race Conditions
5. Numerical Stability in Parallel Algorithms

Chapter 16: Interoperability: CUDA Rust and the Wider World

1. Calling CUDA C++ Libraries from Rust
2. FFI Patterns and Safety
3. Interfacing with cuBLAS and Linear Algebra
4. cuFFT and Transform Libraries
5. CUDA Graphs and Advanced APIs
6. Building Hybrid CUDA C++ and CUDA Rust Projects

Chapter 17: Integration with Rust Application Ecosystems

1. Multi-Threaded Hosts and CUDA Contexts
2. CUDA with Rust Async Runtimes
3. CPU-GPU Hybrid Algorithms
4. Designing a GPU-Accelerated Rust Library
5. Example: Async GPU Processing Pipeline

Chapter 18: Debugging GPU Kernels

1. Common GPU Kernel Bugs
2. Using cuda-memcheck and Sanitizers
3. Debug Builds and GDB for CUDA
4. Print-Based and Trace-Based Debugging
5. Debugging Deadlocks and Timing Issues
6. Systematic Approaches to Bug Isolation

Chapter 19: Profiling and Performance Analysis

1. NVIDIA Nsight Systems and Compute Profilers
2. Occupancy Metrics and Utilization
3. Memory Bandwidth Analysis
4. Identifying Kernel Bottlenecks
5. Roofline Model and Performance Reasoning
6. Iterative Optimization Case Study

Chapter 20: Performance Optimization Patterns

1. Register Pressure and Spilling
2. Instruction-Level Parallelism
3. Loop Unrolling and Predicate Elimination
4. Warp Divergence and Control Flow
5. Using Specialized Instructions
6. Example: Optimizing a Convolution Kernel

Chapter 21: Production Deployment and Best Practices

1. Testing GPU Kernels
2. Continuous Integration for CUDA Rust
3. API Design for Reusable Kernels
4. Documentation and Team Conventions
5. Portability and Hardware Targeting
6. When Not to Use CUDA Rust

Conclusion

1. The CUDA Rust Journey So Far
2. Where the Ecosystem Is Heading
3. Complementary Technologies
4. A Final Word on GPU Computing with Rust

References