Leanpub Header

Skip to main content
Go to Leanpub.com

LM from First Principles

A Notebook-Driven Guide to Building and Deploying Language Models

LM from First Principles
Amarpreet Singh Bassan

Minimum price

Free!

$1.00

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
EPUB
About

About

About the Book

LM from First Principles is a notebook-driven guide to building and deploying language models from the ground up. Starting from a 9,025-parameter bigram that predicts one character at a time, each chapter adds one layer of capability — feedforward context, sampling strategies, self-attention, scaling — until you have a model you can push to Hugging Face. Every concept is grounded in a runnable Jupyter notebook, so you do not just read about how language models work; you watch them train, fail, improve, and generate. The book is written for practitioners who want to understand what is happening inside the model, not just call an API. You need Python and curiosity. Everything else is built from first principles.

Share this book

Categories

Computer Programming/Artificial Intelligence/Python

Feedback

Email the Author

Author

About the Author

Amarpreet Singh Bassan

Amarpreet Singh Bassan is a senior software engineer with 19 years of experience building reliable software systems. His current work focuses on production debugging and improving the reliability of production systems at scale. Outside of his core engineering work, he builds open-source proof-of-concept projects to explore AI engineering, agentic workflows, memory, observability, grounded retrieval, and practical debugging support. He is an IEEE Senior Member and writes about his learning journey in software engineering, AI systems, and reliable production architecture.

Launch

Launch Video

Subscribe on YouTube

Clips

Clips

View all clips →

Contents

Table of Contents

LM from First Principles

  1. A Notebook-Driven Guide to Building and Deploying Language Models

Chapter 1: The Data Pipeline — Text, Tensors, and GPU

  1. What You Will Learn
  2. Setup
  3. Getting the Corpus
  4. Building the Vocabulary
  5. The Lookup Tables
  6. From List to Tensor
  7. Train and Validation Split
  8. Moving to the GPU
  9. Building Batches
  10. A First Model: The Bigram
  11. Training
  12. Evaluation: Loss and Perplexity
  13. The Greedy Collapse
  14. How This Connects to Modern LMs and Inference Engineering
  15. Chapter Summary
  16. What’s Next
  17. The Journey
  18. Review Questions

Chapter 2: Vectors — The Geometry Behind Language Models

  1. What You Will Learn
  2. Setup
  3. 1. Revisit Chapter 1: What Was That Row of 95 Numbers?
  4. 2. Scalar, Vector, Matrix, Tensor
  5. 3. Rank, Shape, Number of Elements, and Dtype
  6. 4. Reading (B, T, C)
  7. 5. Coordinates, Magnitude, and Direction
  8. 6. Vector Addition and Scalar Multiplication
  9. 7. The Dot Product
  10. 8. Distance and Similarity
  11. 9. Matrices as Collections of Vectors
  12. 10. Matrix Multiplication as Transformation
  13. 11. Row Vectors, Column Vectors, and Orientation
  14. 12. Reshaping and Flattening
  15. 13. High-Dimensional Spaces
  16. 14. PyTorch Lab — Becoming Fluent in Vector Shapes
  17. 15. Research Connections: How Papers Enter This Chapter
  18. 16. Where These Vector Operations Reappear
  19. 17. What Chapter 2 Deliberately Does Not Explain
  20. Chapter Summary
  21. What’s Next
  22. Review Questions

Interlude — Can the Whole Sequence Be One Vector?

  1. Bigram: only one position contributes
  2. Flattening works
  3. But flattening is brittle
  4. Shared transformations remove the slot-specific wiring
  5. But the rows still cannot communicate

Chapter 3: Embeddings — From IDs to Learned Representations

  1. What You Will Learn
  2. Setup
  3. 1. IDs Are Indices, Not Representations
  4. 2. nn.Embedding: A Learnable Lookup Table
  5. 3. Why a Lookup Table? The One-Hot Connection
  6. 4. Embedding Lookup Produces Row-Local Gradients
  7. 5. From One Token to a Sequence: (B, T) → (B, T, C)
  8. 6. Building the Minimal Model
  9. 7. The Core Experiment: Training 2D Character Embeddings
  10. 8. Reading the Scatter Plot
  11. 9. Cosine Similarity and Nearest Neighbours
  12. 10. Movement Analysis
  13. 11. Scaling to Higher Dimensions: C=16
  14. 12. Three Types of Embedding
  15. 13. What Token Embeddings Still Do Not Know
  16. Research Connections
  17. What This Chapter Deliberately Does Not Explain
  18. Chapter 3 — Summary
  19. The Road Ahead
  20. Review Questions

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earned over $15 million writing, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub