The Hundred-Page Language Models Course (The Course)
$129.00
Minimum price
$179.00
Suggested price

Course Info

This course includes 5 attempts.

Hi everyone, Andriy here: This Leanpub course is basically the book content, plus 6 quizzes and 27 exercises to help you focus. It's for people who want to learn the material for work, but their company would rather pay for them to do a course than to sit around in their pyjamas and fuzzy slippers reading a book :) Let me know what you think!

Master language models through mathematics, illustrations, and code―and build your own from scratch!

The Hundred-Page Language Models Course by Andriy Burkov, the follow-up to his bestselling The Hundred-Page Machine Learning Book (now in 12 languages), offers a concise yet thorough journey from language modeling fundamentals to the cutting edge of modern Large Language Models (LLMs). Within Andriy's famous "hundred-page" format, readers will master both theoretical concepts and practical implementations, making it an invaluable resource for developers, data scientists, and machine learning engineers.

The Hundred-Page Language Models Course allows you to:

- Master the mathematical foundations of modern machine learning and neural networks

- Build and train three architectures of language models in Python

- Understand and code a Transformer language model from scratch in PyTorch

- Work with LLMs, including instruction finetuning and prompt engineering

Written in a hands-on style with working Python code examples, this book progressively builds your understanding from basic machine learning concepts to advanced language model architectures. All code examples run on Google Colab, making it accessible to anyone with a modern laptop.

About the technology

Language models have evolved from simple n-gram statistics to become one of the most transformative technologies in AI, rivaling only personal computers in their impact. This book spans the complete evolution—from count-based methods to modern Transformer architectures—delivering a thorough understanding of both how these models work and how to implement them.

About the book

The Hundred-Page Language Models Course takes a unique approach by introducing language modeling concepts gradually, starting with foundational methods before advancing to modern architectures. Each chapter builds upon the previous one, making complex concepts accessible through clear explanations, diagrams, and practical implementations.

What's inside

- Essential machine learning and neural network fundamentals

- Text representation techniques and basic language modeling

- Implementation of RNNs and Transformer architectures with PyTorch

- Practical guidance on finetuning language models and prompt engineering

- Important considerations on hallucinations and ways to evaluate models

- Additional resources for advanced topics through the book's wiki

The complete code and additional resources are available through the course's website at thelmbook.com/wiki.

About the reader

Readers should have programming experience in Python. While familiarity with PyTorch and tensors is helpful, it's not required. College-level math knowledge is beneficial, but the book presents mathematical concepts intuitively with clear examples and diagrams.

About the author

Andriy Burkov holds a Ph.D. in Artificial Intelligence and is a recognized expert in machine learning and natural language processing. As a machine learning expert and leader, Andriy has successfully led dozens of production-grade AI projects in different business domains at Fujitsu and Gartner. His previous books have been translated into a dozen languages and are used as textbooks in many universities worldwide. His work has impacted millions of machine learning practitioners and researchers worldwide.

Vint Cerf, Internet pioneer and Turing Award recipient: "This book cleared up a lot of conceptual confusion for me about how Machine Learning actually works - it is a gem of clarity."

Tomáš Mikolov, the author of word2vec and FastText: "The book is a good start for anyone new to language modeling who aspires to improve on state of the art."

Share

Course Material

  • Lesson 1. Machine Learning Basics
  • 1.1. AI and Machine Learning
  • 1.1.1. Early Progress
  • 1.1.2. AI Winters
  • 1.1.3. The Modern Era
  • Artboard 3 Created with Sketch.
    Exercise 1
  • 1.2. Model
  • Artboard 3 Created with Sketch.
    Exercise 2
  • 1.3. Four-Step Machine Learning Process
  • 1.4. Vector
  • Artboard 3 Created with Sketch.
    Exercise 3
  • 1.5. Neural Network
  • Artboard 3 Created with Sketch.
    Exercise 4
  • 1.6. Matrix
  • 1.7. Gradient Descent
  • Artboard 3 Created with Sketch.
    Exercise 5
  • 1.8. Automatic Differentiation
  • Artboard 3 Created with Sketch.
    Exercise 6
  • icon/quiz Created with Sketch.
    Quiz 13 attempts allowed
  • Lesson 2. Language Modeling Basics
  • 2.1. Bag of Words
  • Artboard 3 Created with Sketch.
    Exercise 7
  • 2.2. Word Embeddings
  • Artboard 3 Created with Sketch.
    Exercise 8
  • 2.3. Byte-Pair Encoding
  • Artboard 3 Created with Sketch.
    Exercise 9
  • 2.4. Language Model
  • 2.5. Count-Based Language Model
  • 2.6. Evaluating Language Models
  • 2.6.1. Perplexity
  • 2.6.2. ROUGE
  • 2.6.3. Human Evaluation
  • Artboard 3 Created with Sketch.
    Exercise 10
  • icon/quiz Created with Sketch.
    Quiz 23 attempts allowed
  • Lesson 3. Recurrent Neural Network
  • 3.1. Elman RNN
  • 3.2. Mini-Batch Gradient Descent
  • Artboard 3 Created with Sketch.
    Exercise 11
  • 3.3. Programming an RNN
  • 3.4. RNN as a Language Model
  • Artboard 3 Created with Sketch.
    Exercise 12
  • 3.5. Embedding Layer
  • 3.6. Training an RNN Language Model
  • Artboard 3 Created with Sketch.
    Exercise 13
  • 3.7. Dataset and DataLoader
  • Artboard 3 Created with Sketch.
    Exercise 14
  • 3.8. Training Data and Loss Computation
  • icon/quiz Created with Sketch.
    Quiz 33 attempts allowed
  • Lesson 4. Transformer
  • 4.1. Decoder Block
  • Artboard 3 Created with Sketch.
    Exercise 15
  • 4.2. Self-Attention
  • 4.2.1. Step 1 of Self-Attention
  • 4.2.2. Step 2 of Self-Attention
  • 4.2.3. Step 3 of Self-Attention
  • 4.2.4. Step 4 of Self-Attention
  • 4.2.5. Step 5 of Self-Attention
  • 4.2.6. Step 6 of Self-Attention
  • 4.3. Position-Wise Multilayer Perceptron
  • Artboard 3 Created with Sketch.
    Exercise 16
  • 4.4. Rotary Position Embedding
  • Artboard 3 Created with Sketch.
    Exercise 17
  • 4.5. Multi-Head Attention
  • Artboard 3 Created with Sketch.
    Exercise 18
  • 4.6. Residual Connection
  • 4.7. Root Mean Square Normalization
  • Artboard 3 Created with Sketch.
    Exercise 19
  • 4.8. Key-Value Caching
  • 4.9. Transformer in Python
  • Artboard 3 Created with Sketch.
    Exercise 20
  • icon/quiz Created with Sketch.
    Quiz 43 attempts allowed
  • Lesson 5. Large Language Model
  • 5.1. Why Larger Is Better
  • 5.1.1. Large Parameter Count
  • 5.1.2. Large Context Size
  • 5.1.3. Large Training Dataset
  • 5.1.4. Large Amount of Compute
  • Artboard 3 Created with Sketch.
    Exercise 21
  • 5.2. Supervised Finetuning
  • 5.3. Finetuning a Pretrained Model
  • 5.3.1. Baseline Emotion Classifier
  • 5.3.2. Emotion Generation
  • 5.3.3. Finetuning to Follow Instructions
  • Artboard 3 Created with Sketch.
    Exercise 22
  • 5.4. Sampling From Language Models
  • 5.4.1. Basic Sampling with Temperature
  • 5.4.2. Top-\mathbf{k} Sampling
  • 5.4.3. Nucleus (Top-p) Sampling
  • 5.4.4. Penalties
  • 5.5. Low-Rank Adaptation (LoRA)
  • 5.5.1. The Core Idea
  • 5.5.2. Parameter-Efficient Finetuning (PEFT)
  • 5.6. LLM as a Classifier
  • Artboard 3 Created with Sketch.
    Exercise 23
  • 5.7. Prompt Engineering
  • 5.7.1. Features of a Good Prompt
  • 5.7.2. Follow-up Actions
  • 5.7.3. Code Generation
  • 5.7.4. Documentation Synchronization
  • 5.8. Hallucinations
  • 5.8.1. Reasons for Hallucinations
  • 5.8.2. Preventing Hallucinations
  • 5.9. LLMs, Copyright, and Ethics
  • 5.9.1. Training Data
  • 5.9.2. Generated Content
  • 5.9.3. Open-Weight Models
  • 5.9.4. Broader Ethical Considerations
  • icon/quiz Created with Sketch.
    Quiz 53 attempts allowed
  • Lesson 6. Further Reading
  • 6.1. Mixture of Experts
  • Artboard 3 Created with Sketch.
    Exercise 24
  • 6.2. Model Merging
  • Artboard 3 Created with Sketch.
    Exercise 25
  • 6.3. Model Compression
  • 6.4. Preference-Based Alignment
  • Artboard 3 Created with Sketch.
    Exercise 26
  • 6.5. Advanced Reasoning
  • 6.6. Language Model Security
  • 6.7. Vision Language Model
  • Artboard 3 Created with Sketch.
    Exercise 27
  • 6.8. Preventing Overfitting
  • 6.9. Concluding Remarks
  • 6.10. More From the Author
  • icon/quiz Created with Sketch.
    Quiz 63 attempts allowed

Instructors

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

Now, this is technically risky for us, since you'll have the book or course files either way. But we're so confident in our products and services, and in our authors and readers, that we're happy to offer a full money back guarantee for everything we sell.

You can only find out how good something is by trying it, and because of our 100% money back guarantee there's literally no risk to do so!

So, there's no reason not to click the Add to Cart button, is there?

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earnedover $14 millionwriting, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub