Leanpub Header

Skip to main content
Go to Leanpub.com

AI Engineering Visualized: Foundations, The Course

Prateek Singh

Build one AI support assistant, module by module: about 3 hours of video, 200+ hand-drawn doodles, 24 exercises, 24 quizzes and a certificate.

Minimum price

$34.99

$49.99

You pay

Author earns

$

Also available for 1 course credit with a Learner Membership

PDF
EPUB
WEB
About

About

About the Course

One assistant, built module by module.

I wrote this course for developers who have to ship features that call a language model. The model is one box. The product is the whole diagram around it, and that's what we draw and build here, one box at a time.

What you'll build

You start with one prompt answering "Can I get a refund for order #1234?" By the end, your assistant:

  • decides refunds with rules in code the model can't argue away
  • counts its tokens and checks its output against a contract
  • calls tools, searches a handbook and answers with citations
  • says "I don't know" when the pages don't cover the question
  • measures itself with evals before every change
  • runs an agent loop with step limits, permissions and human approval
  • guards inputs and outputs, masks personal data and traces every request

What's inside

  • 24 modules, plus a short Start Here module that sets up the code
  • 24 video lessons, about 7 minutes each, so about 3 hours of video in all
  • 200+ hand-drawn doodles, each one a named mental model like the Context Suitcase or the Citation Gate
  • 24 one-page doodle cheat sheets
  • 24 coding exercises, each with a prediction question, the expected output and a worked solution
  • 24 quizzes. Conceptual questions only. Retake as often as you like; your best result counts.
  • A certificate of completion: pass with 70% or more, or 90% or more for a distinction

How each module works

Watch the video. Read the lesson. Keep the cheat sheet. Do the exercise: predict what the code will print, run it, compare. Take the quiz. That's it, at your own pace, with no start date and no deadline.

The code

Everything runs on Python 3.10 or newer, using only the standard library, so there's nothing to install. A built-in stub model means you don't need an API key or a paid account, and every exercise has an expected output you can check against. If you'd like to try a real model, you can point the same code at Ollama on your own machine.

Who it's for

Developers who can read Python and want to build LLM features that hold up in production: prompts, structured output, retrieval, evals, agents and guardrails. You don't need a machine learning background.

Who it's not for

If you want to train models from scratch or study the maths behind transformers, this isn't that course.

Modules

0 Start Here · 1 AI Engineering Is a System · 2 What an LLM Is · 3 Talking to a Model · 4 Tokens · 5 Why Answers Vary · 6 Inside the Context Window · 7 Hallucination · 8 Evals · 9 Choosing a Model · 10 Anatomy of a Prompt · 11 Structured Output and Tool Calling · 12 Embeddings · 13 Search and Retrieval · 14 Chunking · 15 The RAG Pipeline · 16 Citations and "I Don't Know" · 17 Evaluating RAG · 18 Workflows vs Agents · 19 The Agent Loop · 20 Planning · 21 Memory · 22 Agent Safety · 23 Guardrails and Privacy · 24 Ship It

The course and the book

The course follows the book AI Engineering Visualized: Foundations (Vol 1), chapter for chapter. If you already have the book, the course adds the videos in each module, the one-page doodle cheat sheets, the prediction questions, the quizzes and the certificate, with your progress tracked as you go.

Share this course

Instructor

About the Instructor

Prateek Singh

Prateek Singh is an Infrastructure Developer, an avid PowerShell blogger, and a community contributor. In 2017 and 2018, FeedSpot and SQLHack recognized his blog RidiCurious.com as among the “Top 50 PowerShell blogs in the world.” Find his open-sourced at GitHub, and reach out to him on Twitter at @singhprateik.

Prateek has written more than 250 articles on his blog and several other websites such as 4SysOps.com, IPSwitch.com, and TechTarget.com and also runs a YouTube channel. He is a published author of 3 books and contributing author of 2 Book and currently writing the 6th one, following is the complete list:

Material

Course Material

  • Module 0: Start Here

  • Welcome to the Course

  • How each module works

  • Setting Up the Code

  • How to Read the Doodles

  • The visual language

  • The text walkthrough

  • Reuse them

  • A worked example

  • Finishing the Course

  • Module 1: AI Engineering Is a System

  • Watch: AI Engineering Is a System

  • Read: AI Engineering Is a System

  • The Whole Diagram

  • One Request, Traced

  • Sidebar: Your System Map Checklist

  • Further Reading

  • Next

  • Doodle Cheat Sheet: AI Engineering Is a System

  • Cheat Sheet at a Glance

  • Exercise: Finish the trace

  • Check Your Work

  • Quiz: AI Engineering Is a System

    99 attempts allowed

  • Module 2: What an LLM Is: From Text Predictor to Assistant

  • Watch: What an LLM Is: From Text Predictor to Assistant

  • Read: What an LLM Is: From Text Predictor to Assistant

  • Pretrained, Then Tuned to Assist

  • Frozen Knowledge

  • Sidebar: Weights, Nudged by Errors

  • Tuned to Please

  • Sidebar: Base vs Instruct and Chat Models

  • Further Reading

  • Next

  • Doodle Cheat Sheet: What an LLM Is: From Text Predictor to Assistant

  • Cheat Sheet at a Glance

  • Exercise: Does pressure change the decision?

  • Check Your Work

  • Quiz: What an LLM Is

    99 attempts allowed

  • Module 3: Talking to a Model: The Chat API

  • Watch: Talking to a Model: The Chat API

  • Read: Talking to a Model: The Chat API

  • The Message List

  • The Model Has No Memory

  • Images and Files Ride Along

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Talking to a Model: The Chat API

  • Cheat Sheet at a Glance

  • Exercise: Audit the message list

  • Check Your Work

  • Quiz: Talking to a Model

    99 attempts allowed

  • Module 4: Tokens: What the Model Reads and What You Pay

  • Watch: Tokens: What the Model Reads and What You Pay

  • Read: Tokens: What the Model Reads and What You Pay

  • The Token Meter

  • Input vs Output Tokens

  • The Context Suitcase

  • Sidebar: Measure Your Own Language

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Tokens: What the Model Reads and What You Pay

  • Cheat Sheet at a Glance

  • Exercise: Split the meter

  • Check Your Work

  • Quiz: Tokens

    99 attempts allowed

  • Module 5: One Token at a Time: Why Answers Vary

  • Watch: One Token at a Time: Why Answers Vary

  • Read: One Token at a Time: Why Answers Vary

  • Next-Token Dice

  • The Temperature Dial

  • Sidebar: Why the First Token Is Slow

  • Further Reading

  • Next

  • Doodle Cheat Sheet: One Token at a Time: Why Answers Vary

  • Cheat Sheet at a Glance

  • Exercise: Load the dice

  • Check Your Work

  • Quiz: One Token at a Time

    99 attempts allowed

  • Module 6: Inside the Context Window: What the Model Pays Attention To

  • Watch: Inside the Context Window: What the Model Pays Attention To

  • Read: Inside the Context Window: What the Model Pays Attention To

  • The Attention Spotlight

  • Where Am I?

  • Sidebar: Position as an Angle

  • Lost in the Middle

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Inside the Context Window: What the Model Pays Attention To

  • Cheat Sheet at a Glance

  • Exercise: Watch the share shrink

  • Check Your Work

  • Quiz: Inside the Context Window

    99 attempts allowed

  • Module 7: Fluent Is Not True: Hallucination

  • Watch: Fluent Is Not True: Hallucination

  • Read: Fluent Is Not True: Hallucination

  • Fluent Is Not True

  • The Confident Guess

  • The Citation Gate

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Fluent Is Not True: Hallucination

  • Cheat Sheet at a Glance

  • Exercise: Count the invented clauses

  • Check Your Work

  • Quiz: Fluent Is Not True

    99 attempts allowed

  • Module 8: Evals: Measuring a Probabilistic System

  • Watch: Evals: Measuring a Probabilistic System

  • Read: Evals: Measuring a Probabilistic System

  • The Golden Set

  • Sidebar: Slice Your Golden Set

  • The Pass/Fail Rubric

  • Score Before You Change

  • Sidebar: LLM-as-Judge Preview

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Evals: Measuring a Probabilistic System

  • Cheat Sheet at a Glance

  • Exercise: Score the next fix

  • Check Your Work

  • Quiz: Evals

    99 attempts allowed

  • Module 9: Choosing a Model

  • Watch: Choosing a Model

  • Read: Choosing a Model

  • The Model Menu

  • Sidebar: Check the License

  • Thinking Costs Tokens

  • The Bake-Off

  • Sidebar: Big Model, Small Bill

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Choosing a Model

  • Cheat Sheet at a Glance

  • Exercise: Move the bar

  • Check Your Work

  • Quiz: Choosing a Model

    99 attempts allowed

  • Module 10: Anatomy of a Prompt

  • Watch: Anatomy of a Prompt

  • Read: Anatomy of a Prompt

  • The Prompt Stack

  • Examples Beat Adjectives

  • Instructions vs Data

  • Sidebar: Prompt Templates and Variables

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Anatomy of a Prompt

  • Cheat Sheet at a Glance

  • Exercise: Take the stack apart

  • Check Your Work

  • Quiz: Anatomy of a Prompt

    99 attempts allowed

  • Module 11: Structured Output and Tool Calling

  • Watch: Structured Output and Tool Calling

  • Read: Structured Output and Tool Calling

  • The Output Contract

  • The Tool Call Round Trip

  • Parse, Check, Repair

  • Sidebar: JSON Mode vs Schema-Constrained Output

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Structured Output and Tool Calling

  • Cheat Sheet at a Glance

  • Exercise: Try to break the contract

  • Check Your Work

  • Quiz: Structured Output and Tool Calling

    99 attempts allowed

  • Module 12: Embeddings: Meaning as Coordinates

  • Watch: Embeddings: Meaning as Coordinates

  • Read: Embeddings: Meaning as Coordinates

  • The Meaning Map

  • Nearest Neighbours

  • Close Isn’t Correct

  • Sidebar: Cosine vs Dot Product

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Embeddings: Meaning as Coordinates

  • Cheat Sheet at a Glance

  • Exercise: Turn the knobs, then fool the map

  • Check Your Work

  • Quiz: Embeddings

    99 attempts allowed

  • Module 13: Search and Retrieval: Finding the Right Passage

  • Watch: Search and Retrieval: Finding the Right Passage

  • Read: Search and Retrieval: Finding the Right Passage

  • Keywords vs Meaning

  • The Top-k Net

  • The “Nothing Close Enough” Line

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Search and Retrieval: Finding the Right Passage

  • Cheat Sheet at a Glance

  • Exercise: Size the net, then draw the line

  • Check Your Work

  • Quiz: Search and Retrieval

    99 attempts allowed

  • Module 14: Chunking: Cutting Documents So the Right Piece Is Found

  • Watch: Chunking: Cutting Documents So the Right Piece Is Found

  • Read: Chunking: Cutting Documents So the Right Piece Is Found

  • Chunk Size Goldilocks

  • Overlap Keeps the Sentence Whole

  • Labels on Every Chunk

  • Sidebar: Fixed vs Sentence Chunks

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Chunking: Cutting Documents So the Right Piece Is Found

  • Cheat Sheet at a Glance

  • Exercise: Cut the handbook four ways

  • Check Your Work

  • Quiz: Chunking

    99 attempts allowed

  • Module 15: The RAG Pipeline: Retrieve, Then Answer

  • Watch: The RAG Pipeline: Retrieve, Then Answer

  • Read: The RAG Pipeline: Retrieve, Then Answer

  • The Open-Book Exam

  • The RAG Pipeline

  • Prompt, RAG or Fine-Tune?

  • Further Reading

  • Next

  • Doodle Cheat Sheet: The RAG Pipeline: Retrieve, Then Answer

  • Cheat Sheet at a Glance

  • Exercise: Trace the four stages, then break one

  • Check Your Work

  • Quiz: The RAG Pipeline

    99 attempts allowed

  • Module 16: Grounded Answers: Citations and “I Don’t Know”

  • Watch: Grounded Answers: Citations and “I Don’t Know”

  • Read: Grounded Answers: Citations and “I Don’t Know”

  • Answers With Receipts

  • The Honest Abstain

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Grounded Answers: Citations and “I Don’t Know”

  • Cheat Sheet at a Glance

  • Exercise: Try to fool the gate

  • Check Your Work

  • Quiz: Grounded Answers

    99 attempts allowed

  • Module 17: Is the RAG Working? Evaluating Retrieval and Answers

  • Watch: Is the RAG Working? Evaluating Retrieval and Answers

  • Read: Is the RAG Working? Evaluating Retrieval and Answers

  • Two Scorecards

  • Did We Fetch It?

  • Did It Stick to the Source?

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Is the RAG Working? Evaluating Retrieval and Answers

  • Cheat Sheet at a Glance

  • Exercise: Read both scorecards, fix the right half

  • Check Your Work

  • Quiz: Is the RAG Working?

    99 attempts allowed

  • Module 18: Workflows vs Agents: Rails or Steering Wheel

  • Watch: Workflows vs Agents: Rails or Steering Wheel

  • Read: Workflows vs Agents: Rails or Steering Wheel

  • Rails vs Steering Wheel

  • The Autonomy Ladder

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Workflows vs Agents: Rails or Steering Wheel

  • Cheat Sheet at a Glance

  • Exercise: Check the router

  • Check Your Work

  • Quiz: Workflows vs Agents

    99 attempts allowed

  • Module 19: The Agent Loop: Think, Act, Observe

  • Watch: The Agent Loop: Think, Act, Observe

  • Read: The Agent Loop: Think, Act, Observe

  • The Agent Loop

  • The Reason-Act Trace

  • Stop Conditions

  • Further Reading

  • Next

  • Doodle Cheat Sheet: The Agent Loop: Think, Act, Observe

  • Cheat Sheet at a Glance

  • Exercise: Make the loop stop

  • Check Your Work

  • Quiz: The Agent Loop

    99 attempts allowed

  • Module 20: Planning and Multi-Step Tasks

  • Watch: Planning and Multi-Step Tasks

  • Read: Planning and Multi-Step Tasks

  • Plan, Then Execute

  • Replan on Surprise

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Planning and Multi-Step Tasks

  • Cheat Sheet at a Glance

  • Exercise: Break the plan

  • Check Your Work

  • Quiz: Planning and Multi-Step Tasks

    99 attempts allowed

  • Module 21: Memory: What the Agent Remembers

  • Watch: Memory: What the Agent Remembers

  • Read: Memory: What the Agent Remembers

  • Scratchpad vs Long-Term Memory

  • Remembering by Search

  • What to Forget

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Memory: What the Agent Remembers

  • Cheat Sheet at a Glance

  • Exercise: Remember less

  • Check Your Work

  • Quiz: Memory

    99 attempts allowed

  • Module 22: Agent Safety: Permissions, Approvals and the Trojan Ticket

  • Watch: Agent Safety: Permissions, Approvals and the Trojan Ticket

  • Read: Agent Safety: Permissions, Approvals and the Trojan Ticket

  • Least Privilege Tools

  • The Approval Gate

  • The Trojan Ticket

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Agent Safety: Permissions, Approvals and the Trojan Ticket

  • Cheat Sheet at a Glance

  • Exercise: Run the Trojan ticket

  • Check Your Work

  • Quiz: Agent Safety

    99 attempts allowed

  • Module 23: Guardrails and Privacy: Checks In, Checks Out

  • Watch: Guardrails and Privacy: Checks In, Checks Out

  • Read: Guardrails and Privacy: Checks In, Checks Out

  • The Guardrail Sandwich

  • What Leaves Your Building

  • Sidebar: Masking PII: Replace, Then Restore

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Guardrails and Privacy: Checks In, Checks Out

  • Cheat Sheet at a Glance

  • Exercise: Test both slices

  • Check Your Work

  • Quiz: Guardrails and Privacy

    99 attempts allowed

  • Module 24: Ship It: Errors, Tracing, Latency and the Complete System

  • Watch: Ship It: Errors, Tracing, Latency and the Complete System

  • Read: Ship It: Errors, Tracing, Latency and the Complete System

  • Timeout, Retry, Give Up

  • The Debug Replay

  • The Latency Budget

  • The Complete Vol 1 Diagram

  • Further Reading

  • Next

  • Doodle Cheat Sheet: Ship It: Errors, Tracing, Latency and the Complete System

  • Cheat Sheet at a Glance

  • Exercise: Fail on purpose

  • Check Your Work

  • Quiz: Ship It

    99 attempts allowed

  • Wrap-Up

  • You Built the Whole System

  • Get Your Certificate

  • Where to Go Next

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub