Leanpub Header

Skip to main content

Build Your Own GPT From Scratch

Train a Tiny Language Model on Your Laptop and Chat With It in Your Browser

Build Your Own GPT From Scratch
This book is 100% completeLast updated on 2026-11-18

Build a tiny GPT from scratch, train it on your laptop in minutes, and chat with it in your browser. No API, no pretrained model, no GPU: just clear steps and code you understand line by line.

Minimum price

$12.99

$24.99

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
About

About

About the Book

Ever wondered what is actually going on inside ChatGPT? In this book you build your own tiny GPT, completely from scratch, and watch it learn to write.

There is no API key, no pretrained model and no expensive GPU. You prepare a dataset from three classic novels, write a small decoder-only transformer in PyTorch, train it on an ordinary laptop CPU in about 13 minutes, and then run the very same model in plain JavaScript so you can chat with it right inside your web browser.

What's inside:

- 111 pages: 21 chapters in 7 parts, plus 3 appendices

- The core ideas in plain words, starting with a simple letter-counting model that your GPT has to beat

- Attention worked through by hand with small numbers, step by step

- Every file of the model explained line by line, with all 830,464 parameters accounted for

- A look inside the trained model: real attention maps and what it learned about characters

- Experiments: the model's writing at 8 points during training, and 4 model sizes compared head to head

- "What you learned" summaries and "Try it yourself" exercises in every chapter

- The full source code, a glossary of 39 terms and a one-page command cheat sheet

Who it's for: developers and students who know a little Python and want to truly understand how large language models work. By the last page you will have your own working GPT and the confidence to explain every part of it.

Author

About the Author

Sourabh Somani

Sourabh Somani is a software developer from India who loves building things from scratch to understand how they really work.

His projects include Sourabh OS, an operating system written in Python and designed around AI; KidAI, a simple coding language that lets children write code in plain English and Hindi; and a tiny GPT that trains on an ordinary laptop and runs right inside the web browser. He also builds mobile apps and games.

He writes the way he learns: step by step, in plain language, with code you can run yourself. He shares his builds and demos on GitHub (github.com/sourabhsomani) and on Instagram as @sourabhsomani8 @sourabh.builds. Website: sourabhsomani.com

Contents

Table of Contents

  • Before we start
  • Part I: The ideas
    • 1. What a GPT really does
    • 2. Before transformers: counting letters
    • 3. Neural networks in plain words
    • 4. Setting up your computer
  • Part II: The data
    • 5. Collecting and cleaning the books
    • 6. Tokens, vocabulary and batches
  • Part III: The model
    • 7. Embeddings: giving each character a meaning
    • 8. Self-attention, step by step
    • 9. The transformer block
    • 10. The complete model
  • Part IV: Training
    • 11. Loss: how the model knows it is wrong
    • 12. The training loop, line by line
    • 13. Training for real, and experiments
  • Part V: Generating text
    • 14. From probabilities to sentences
  • Part VI: The browser
    • 15. Exporting the weights
    • 16. The model in plain JavaScript
    • 17. Proving the JavaScript model is correct
    • 18. Building the chat page
  • Part VII: Run, share, extend
    • 19. Running and testing on your computer
    • 20. Run my project yourself
    • 21. Where to go next
  • Appendices
    • A. Full source code
    • B. Glossary
    • C. Command cheat sheet

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub