[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6,"_7":8},"actionData","errors","root",{"_28":-5,"_38":-5,"_39":40,"_41":42,"_43":44,"_45":46,"_47":48,"_49":50,"_51":52,"_53":54,"_55":56,"_57":58},"routes/_store+/_readers+/email_author.$slug/route",{"_9":10,"_11":-5},"book",{"_12":13,"_14":15,"_16":17,"_18":19,"_20":21,"_22":23},"currentUser","id","Z2lkOi8vbGVhbnB1Yi9TdG9yZTo6Qm9va1Byb2R1Y3QvODA1NTI","title","LLM Quantization","state","published","contactAuthorMethod","email","aboutText","

Most explanations of model quantization hand you a recipe and a number: \"use INT4, you'll lose about 1% accuracy.\" This book refuses to stop there. It asks why — and answers from the bits up.

Starting from a single weight and the question of how few bits can represent it, the book builds the entire field from first principles. Every mechanism is coded from scratch in tested Python and PyTorch, deliberately pushed until it breaks, and measured with real numbers produced by executed code. Nothing is asserted; everything is demonstrated. When the book says GPTQ makes 3-bit weights free, there is a measured perplexity behind it. When it says one outlier channel costs 40 dB, there is a derivation and a verification.

What you'll build and measure

The Δ²/12 noise law and the 6.02 dB-per-bit rule — derived, then verified in code
A proof of why output error, not weight error, is the objective that matters
GPTQ, built from the Optimal Brain Surgeon equations, rescuing 3-bit weights
AWQ, SmoothQuant, and Hadamard rotation — implemented and compared
The GGUF K-quant byte layout that powers llama.cpp, reconstructed byte for byte
The quality cliff where low-bit quantization collapses — and exactly why no algorithm can save it
A capstone that quantizes a real model end to end and explains every number in the final size × accuracy × speed table

Who it's for

Engineers and researchers who want to understand quantization, not just apply it — people who would rather know why INT4 works than memorize that it does. If you can read Python and basic linear algebra, you can follow every derivation and reproduce every experiment.

How it's taught

The reference model throughout is TinyGPT, a small GPT-style decoder trained from scratch on Shakespeare — chosen so every experiment runs on a single CPU core and every result is reproducible. Seventeen chapters, six parts, fifty-plus figures, and not a single invented number.

This is quantization as a small set of principles — representation, noise, propagation, outliers, algorithms, kernels — that together let you predict exactly what happens when you take the bits away.

Build it, break it, measure it.

","authors",[24],{"_12":25,"_26":27,"_28":29},"Z2lkOi8vbGVhbnB1Yi9TdG9yZTo6UHJvZHVjdEF1dGhvci84NjExMA","name","Hatem M.","user",{"_30":31},"avatar",{"_32":33,"_34":35,"_36":37},"url","https://secure.gravatar.com/avatar/2ab5c51a2b78e985dc0c76be7fe841e5?r=pg&d=mm","styleName","medium","exists",true,"shoppingCartId","shoppingCart",{"_12":71,"_72":73,"_74":37,"_75":76,"_77":78,"_79":80,"_81":37},"locale","en","readerTheme","light","constants",{"_69":70},"publishers",{"_65":66,"_67":68},"learnerOrganizations",[],"needsLocaleCookieSync",false,"announcements",{"_59":60,"_61":-5},"sentryEnvironment","production","sentryRelease","f5548b0fa7c03e0c3ddcc40793859fac31237ca7","announcement_store",{"_62":52,"_63":64},"announcement_authors","flag","text","

Kick off your book project, get started with GhostAI, get better at marketing, or spend the day doing all three! Free live workshops on Zoom. Saturday, June 27, 2026.

\n","companies",[],"universities",[],"paypalClientId","AX53Cwz8jR6rVVzzZiZNWWXl0dGagi-GZrjG-S3Ua14E5nrKMn7DxDQ730O4R8LusLPp49ALn6aRVaEy","no-cart","count",0,"free","updatedAt","","paymentGateway","STRIPE","cartItems",[],"loading"]

Email the Author