GPU Programming & Architecture Super Bundle

THIRD EDITION - now with pedagogy & color illustrations

GPU Parallel Computing: From Basics to Breakthroughs — A Technical Guide to GPU Programming

If you want to understand how modern GPUs work and how to use them effectively for high-performance workloads, this book provides the technical foundation required.

This book assumes no prior exposure to GPU internals; however, a working knowledge of electronics and general computer architecture is recommended.It is written for students, engineers, researchers, and data scientists who are new to GPU architecture and parallel programming and want a rigorous introduction before progressing into optimization and large-scale GPU systems.

If you are already an experienced CUDA performance engineer or low-level GPU architect seeking a specialized microarchitectural reference manual, this book is not positioned for that purpose.

What You Will Learn

• GPU Architecture Fundamentals
• Streaming multiprocessors and SIMT execution
• Warp scheduling and instruction flow
• GPU memory hierarchy and bandwidth considerations
• GPU Programming Models
• CUDA programming principles
• OpenCL fundamentals
• Kernel structure and execution behavior
• Performance Optimization
• Memory access patterns and coalescing
• Warp divergence and latency hiding
• Occupancy principles and kernel configuration
• Real-World Applications
• Scientific simulations
• Machine learning workloads
• Graphics and visualization pipelines

Advanced Topics

• Multi-GPU communication
• Tensor cores and mixed precision
• Profiling, debugging, and performance analysis

The early chapters establish architectural clarity and programming fundamentals.

Later chapters address optimization strategies, scalability, and applied GPU workloads.

Who This Book Is For

1. Students entering GPU computing
2. Engineers transitioning into parallel architecture
3. Researchers and data scientists adopting GPU acceleration

This is a technical book. It builds understanding from architectural principles upward and focuses on performance-oriented reasoning rather than superficial overview.

Why This Book?

Many GPU resources either assume too much prior knowledge or remain overly abstract.This book emphasizes structured technical understanding:

• How GPUs execute threads
• Why performance bottlenecks occur
• How architectural constraints shape results
• How programming decisions map to hardware behavior

Clear explanations. Practical code examples. Architectural context.

Modern GPU Architecture THIRD EDITION — Volume One
Graphics Pipeline Design and Hardware Implementation

Modern GPUs are the most complex and efficient parallel processors ever created—and this book shows you exactly how they work at the hardware level.

Unlike typical graphics or programming guides, this volume takes you inside the GPU itself:
how instructions flow through pipelines, how memory hierarchies sustain bandwidth, how shader cores and fixed-function units cooperate to render billions of pixels per second.

You’ll explore every major stage of the graphics pipeline in depth—geometry, rasterization, shading, texturing, and render output—all supported by clear mathematical models and synthesizable Verilog examples. This is not “theory for theory’s sake”; it’s engineering detail you can apply directly in design, simulation, or hardware verification.

By reading this book, you’ll gain:

• Architectural intuition — understand how throughput, latency, and bandwidth interact in real GPUs.
• Practical RTL-level insight — see how each stage can be implemented with clean, synthesizable Verilog.
• A foundation for advanced design — build the knowledge required for AI acceleration, compute architectures, or FPGA-based GPU prototyping.
• Confidence to analyze real silicon — reason about performance, bottlenecks, and tradeoffs like a hardware architect.

Every chapter bridges concept and implementation, making it invaluable for anyone designing graphics hardware, studying computer architecture, or seeking mastery of parallel computation systems.

Dense, detailed, and unapologetically technical, this book is written for those who want to understand modern GPUs—not just use them.

⚠️ This isn’t entertainment. It’s engineering.
If that excites you, welcome aboard.
If it intimidates you, this book isn’t for you.

From the Editor at Burst Books — Gareth Thomas

A Smarter Kind of Learning Has Arrived — Thinking on Its Own.

Forget tired textbooks from years past. These AI-crafted STEM editions advance at the speed of discovery. Each page is built by intelligence trained on thousands of trusted sources, delivering crystal-clear explanations, flawless equations, and functional examples — all refreshed through the latest breakthroughs.

Best of all, these editions cost a fraction of traditional texts yet surpass expectations. You’re gaining more than a book — you’re enhancing the mind’s performance.

Explore BurstBooksPublishing on GitHub to find technical samples, infographics, and additional study material — a complete hub that supports deeper, hands-on learning.

In this age of AI, leave the past behind and learn directly from tomorrow.

Modern GPU Architecture THIRD EDITION — Volume Two
Graphics Pipeline Design and Hardware Implementation

Modern GPUs are the most complex and efficient parallel processors ever created—and this book shows you exactly how they work at the hardware level.

Unlike typical graphics or programming guides, this volume takes you inside the GPU itself:
how instructions flow through pipelines, how memory hierarchies sustain bandwidth, how shader cores and fixed-function units cooperate to render billions of pixels per second.

You’ll explore every major stage of the graphics pipeline in depth—geometry, rasterization, shading, texturing, and render output—all supported by clear mathematical models and synthesizable Verilog examples. This is not “theory for theory’s sake”; it’s engineering detail you can apply directly in design, simulation, or hardware verification.

By reading this book, you’ll gain:

• Architectural intuition — understand how throughput, latency, and bandwidth interact in real GPUs.
• Practical RTL-level insight — see how each stage can be implemented with clean, synthesizable Verilog.
• A foundation for advanced design — build the knowledge required for AI acceleration, compute architectures, or FPGA-based GPU prototyping.
• Confidence to analyze real silicon — reason about performance, bottlenecks, and tradeoffs like a hardware architect.

Every chapter bridges concept and implementation, making it invaluable for anyone designing graphics hardware, studying computer architecture, or seeking mastery of parallel computation systems.

Dense, detailed, and unapologetically technical, this book is written for those who want to understand modern GPUs—not just use them.

⚠️ This isn’t entertainment. It’s engineering.
If that excites you, welcome aboard.
If it intimidates you, this book isn’t for you.

From the Editor at Burst Books — Gareth Thomas

A Smarter Kind of Learning Has Arrived — Thinking on Its Own.

Forget tired textbooks from years past. These AI-crafted STEM editions advance at the speed of discovery. Each page is built by intelligence trained on thousands of trusted sources, delivering crystal-clear explanations, flawless equations, and functional examples — all refreshed through the latest breakthroughs.

Best of all, these editions cost a fraction of traditional texts yet surpass expectations. You’re gaining more than a book — you’re enhancing the mind’s performance.

Explore BurstBooksPublishing on GitHub to find technical samples, infographics, and additional study material — a complete hub that supports deeper, hands-on learning.

In this age of AI, leave the past behind and learn directly from tomorrow.

THIRD EDITION

Uncover the fundamentals of GPU architecture and assembly programming with Advanced GPU Assembly Programming, a resource designed for enthusiasts and professionals who want to explore the intricate workings of modern GPUs. This book is not a step-by-step manual but a gateway to understanding GPU architecture and assembly programming at a foundational level. It’s ideal for readers who are ready to invest their own effort to experiment and grow their expertise.

What You’ll Gain:

1. Deep Insights into GPU Architecture

• Explore the fundamental differences between GPUs and CPUs, with a focus on parallelism, memory hierarchies, and threading models.
• Learn the principles underlying the instruction set architectures (ISAs) of NVIDIA and AMD GPUs.

2. Foundations of Assembly Programming

• Delve into the mechanics of low-level GPU programming, including execution models, memory access optimization, and pipeline management.
• Understand the core concepts of assembly programming while preparing to apply them with additional resources and practice.

3. Tools and Techniques

• Get an overview of key debugging and profiling tools such as NVIDIA Nsight and AMD Radeon GPU Profiler.
• Gain the contextual knowledge to optimize GPU performance through careful analysis and tuning.

4. Future-Focused Knowledge

• Stay ahead of emerging trends in GPU technology, from next-generation architectures to AI-driven optimization tools.

Who This Book is For:

• Assembly Enthusiasts: Those eager to understand GPUs at their core and explore low-level programming.
• Developers and Engineers: Professionals optimizing GPU-driven systems in gaming, AI, and scientific computing.
• Researchers and Students: Anyone seeking a foundational understanding of GPU architectures and programming approaches.

What This Book is Not:

This is not a hands-on, step-by-step guide. Instead, it provides a conceptual framework and architectural insights to set readers on the right path. It encourages further exploration and learning through personal effort and experimentation.

Whether you’re a developer, researcher, or assembly enthusiast, Advanced GPU Assembly Programming will give you the knowledge needed to deeply understand GPU architecture and programming. Equip yourself with the foundational tools to explore, experiment, and achieve mastery in the fascinating world of GPU assembly.

Order your copy today and take your first step into the realm of GPU programming mastery!

UPDATE: This book now has a github repository with all source code samples, infographics, exercise manual and more.

Most CUDA programmers never see the real program their GPU actually runs.

They write CUDA C++. They launch kernels. They profile. They tune block sizes, adjust memory access, stare at Nsight reports, and hope the compiler has done what they think it has done.

But the truth is lower down.

The truth is in PTX. The truth is in SASS.

Mastering PTX and SASS is for CUDA programmers, GPU performance engineers, ML systems developers, compiler-minded programmers, and high-performance computing specialists who want to understand NVIDIA GPU execution below the source level.

This is not a beginner CUDA book.

It is for readers who already understand the basic GPU programming model and now want to inspect the layer where performance is really exposed: instruction streams, register allocation, memory transactions, predicate logic, compiler output, synchronization, tensor pipelines, and architecture-specific machine code.

Inside, you will learn how to:

• Understand how CUDA source becomes PTX, and how PTX becomes SASS
• Use PTX as a portable low-level view of compiler intent
• Use SASS as evidence of what the GPU actually executes
• Interpret register pressure, spills, occupancy, instruction scheduling, and memory traffic
• Diagnose coalescing problems, bank conflicts, branch divergence, dependency stalls, and synchronization costs
• Work with tensor cores, MMA, WGMMA, TMA, low-precision formats, and modern NVIDIA architecture features
• Use nvcc, ptxas, cuobjdump, nvdisasm, Nsight Compute, Nsight Systems, and Compute Sanitizer as practical engineering tools
• Decide when CUDA C++, compiler flags, intrinsics, inline PTX, full PTX, or SASS inspection is the right level to touch

The goal is not assembly for its own sake.

The goal is diagnosis.

The goal is control.

The goal is to look at a slow kernel and know whether the limiting factor is memory movement, instruction throughput, register pressure, tensor-pipeline starvation, branch behavior, synchronization overhead, launch cost, or compiler transformation.

Modern GPU performance is not won by writing code that merely runs.

It is won by understanding how the machine schedules work, moves data, allocates registers, forms instructions, hides latency, feeds tensor units, and exposes bottlenecks through measurable evidence.

This book is for you if you already write CUDA code and want to understand what happens after compilation.

If you are still learning what a thread block is or how kernels launch, start with an introductory CUDA book first.

But if you want to read below the source level, connect profiler symptoms to architectural causes, and move closer to the real hardware ceiling, Mastering PTX and SASS was written for you.

Correctness is only the beginning.

The real question is: how close can you get to the hardware ceiling?