LLM Inference Engineering Memory, Kernels, and Production Serving

About the Book

Running an LLM is one problem. Understanding its memory footprint, execution phases, and serving behavior is another.

LLM Inference Engine Architecture explores the engineering behind language-model inference—from prefill and decode to KV-cache management, attention kernels, quantization, and request scheduling.

Written for software engineers and ML engineers with a working knowledge of Python and transformer fundamentals, the book connects mathematical explanations with implementation examples and serving-system design considerations.

What You Will Explore

• Inference fundamentals: Trace the request lifecycle and understand the different computational characteristics of prefill and decode.
• Memory budgeting: Work through model-weight and KV-cache estimates, including the effects of context length, KV-head count, precision, and concurrent sequences.
• Attention and memory management: Explore PagedAttention, non-contiguous KV allocation, and the relationship between attention computation and memory access.
• Quantization: Examine weight and KV-cache precision choices, including FP8 and low-bit weight-quantization approaches.
• Serving architecture: Study continuous batching, request scheduling, multi-GPU execution, and latency and throughput metrics.
• Kernel development: Explore CUDA and Triton concepts, kernel fusion, and a quantized matrix-multiplication case study.

The book includes numerical examples, reference code, and an educational asynchronous serving simulator. These examples are intended to explain mechanisms and trade-offs—not to replace hardware-specific profiling or production validation.

Read the sample to review the mathematical level, coding style, and scope before purchasing.