Kick off your book project in 3 hours! Live workshop on Zoom. You’ll leave with a real book project, progress on your first chapter, and a clear plan to keep going. Saturday, June 6, 2026. Learn more…

Leanpub Header

Skip to main content
Go to Leanpub.com

Under The Hood

Build Every Layer of a Large Language Model from Scratch

This book is 100% completeLast updated on 2026-05-21
Ramchand Kumaresan

A practical, project-driven manual for engineers who want to understand how modern language models are built — and where they fail — by writing every layer themselves. From a scalar autograd engine to RLHF to fused specialists, in 35 hands-on projects with deliberate sabotage experiments. Build it. Break it. Measure it.

Minimum price

$19.99

$35.00

You pay

Author earns

$
PDF
EPUB
974
Pages
256,586Words
About

About

About the Book

Most LLM books teach you to *use* models. This one teaches you to *build* them — every layer, every optimizer step, every cache, every quantization scheme — and then to deliberately break each piece so you understand why it exists in the first place.

It is a workshop in book form. 35 hands-on projects, ~250,000 words, one tight spiral that takes you from a single autograd scalar all the way to fusing independently trained specialists into a routed system. No black boxes. No "import library, call method." You write the code, you run it, you break it, you measure what broke.

Each project follows the same disciplined rhythm: Hook → The Concept → Why It Matters → The Build → **BREAK IT** → Optional Homework → Questions To Answer → Go Further → What You Now Know. Reading the book without breaking the code is half the experience. The breaks are where the lessons actually live.

A public code companion lives at github.com/mechramc/Under-the-hood with runnable build.py, tests, and captured outputs for every project.

If you have ever read a transformer paper and felt that the diagram and the code were in two different universes — this book closes that gap.

**Build it. Break it. Measure it.**

Share this book

Categories

Computers and ProgrammingLarge language modelsArtificial Intelligence

Feedback

Email the Author

Installments completed

1 / 3

Author

About the Author

Ramchand Kumaresan

Ramchand Kumaresan is a Senior Program Manager at Procore Technologies and the founder of Murai Labs, a one-person AI research lab built on the idea that disciplined engineering matters more than hype.

His published work includes KALAVAI (cooperative LoRA fusion for routing across specialist adapters), UYIR (evolutionary lifecycles for LoRA adapters, currently under review at TMLR), and Orion (the first open end-to-end system for programming Apple's Neural Engine for LLM inference and training). His current research focuses on heritable sparse mutability masks (MARMAM), grounded in Tamil Siddha marmam-point therapy.

He wrote Under the Hood to teach himself what he didn't know — and then kept writing as he learned more. The book is the trail he marked along the way.

Contents

Table of Contents

Under The Hood

  1. The LLM Engineering Manual

Copyright

Preface: What This Book Is Actually Asking You To Do

Using the Code Repository

  1. How the repo is organized
  2. The intended workflow per chapter
  3. Getting it running
  4. What the repo is not
  5. Reporting issues

Preflight: Python, Tensors, and What a Language Model Actually Is

  1. What a Language Model Actually Is
  2. How a Model Learns: The Three Moves
  3. Python You Need to Read This Book
  4. Numbers as Tensors
  5. The Shape of the Book
  6. Quick Reference: What to Do If You Get Stuck
  7. What You Now Know

Project 1: The Learning Machine

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 2: Predicting The Next Character

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. Building the neural character model
  6. BREAK IT
  7. Optional Homework
  8. Questions To Answer
  9. Go Further
  10. What You Now Know
  11. Starting Point

Project 3: Building A Tokenizer

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 4: Attention From Scratch

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 5: Your GPT From A Blank File

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 6: From Prototype to nanoGPT

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 7: The Details That Matter

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 8: Flash Attention and Tiled Kernels

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 9: Pretraining On The Real Web

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 10: Data Curation and Contamination

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 11: Training Debugging: Spikes, NaNs, and Profiling

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 12: Distributed Training: FSDP and ZeRO (Single-Box Proxy)

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 13: Fast Inference: The KV Cache

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 14: Speculative Decoding

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 15: Grouped Query Attention

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 16: Long-Context Extension (RoPE, YaRN, NTK-Aware)

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 17: Production Serving: Continuous Batching and PagedAttention

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 18: Mixture Of Experts

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 19: Scaling Laws

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 20: Autonomous Experimentation

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 21: Fine-Tuning And Instruction Tuning

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 22: Evaluation Methodology

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 23: Reward Models And RLHF

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 24: DPO and Preference Optimization

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 25: Test-Time Reasoning (CoT, Self-Consistency, Best-of-N)

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 26: Tool Use and Function Calling

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 27: Quantization and Deployment

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 28: Retrieval-Augmented Generation

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 29: Multimodal: A Tiny Vision-Language Model

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 30: Non-Transformer Architectures (Mamba, RWKV)

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 31: Layer Freezing and Transfer

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 32: Fusing Independently Trained Specialists

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Starting Point

Project 33: The Interface Specification

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Research Anchors
  11. Starting Point

Project 34: Incremental Assembly

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. What You Now Know
  10. Research Anchors
  11. Starting Point

Project 35: Your Architecture

  1. Hook
  2. The Concept
  3. Why It Matters
  4. The Build
  5. BREAK IT
  6. Optional Homework
  7. Questions To Answer
  8. Go Further
  9. Research Anchors
  10. What You Now Know
  11. Where The Field Is Now
  12. What To Sound Like In A Strong Interview
  13. Frontier Reading Map
  14. What The Book Now Gives You
  15. Starting Point

Appendix A: Lecture Companions

  1. How to use this appendix
  2. Preflight companion: Python and tensor foundations
  3. Part I companion: learning mechanics, tokenization, attention
  4. Part II companion: building and training a transformer
  5. Part III companion: inference, efficiency, and scaling
  6. Part IV companion: post-training, alignment, deployment
  7. Part V companion: transfer, modularity, interfaces, research
  8. Suggested study rhythm

Appendix B: Free Resources

  1. Reference architecture by topic
  2. Stability rule

Appendix C: Notes, Sources, And Bibliography

  1. Project Sources

Glossary

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

Now, this is technically risky for us, since you'll have the book or course files either way. But we're so confident in our products and services, and in our authors and readers, that we're happy to offer a full money back guarantee for everything we sell.

You can only find out how good something is by trying it, and because of our 100% money back guarantee there's literally no risk to do so!

So, there's no reason not to click the Add to Cart button, is there?

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earned over $15 million writing, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub