Figures v
Preface vi
Part I. Traces of Machine Learning Systems 1
1. The Inversion 2
2. Trace Analysis Patterns in Brief 8
3. Three Strata 14
4. The Training Loop as a Trace 20
5. Loss Curves and Counter Values 30
6. Data Pipelines and Loaders 40
7. Checkpoints, Resumption, and Truncated Traces 48
8. Distributed Training as Threads of Activity 55
9. Inference Serving Traces 65
10. Agentic Runs 71
11. Evaluation and Experiment Traces 82
12. Anomalies in Machine Learning System Traces 88
Part II. Machine Learning as Trace 99
13. The Forward Pass as a Trace 100
14. Tokenization as Text Trace 109
15. Embeddings as Message Embedding 115
16. Attention as Message Context 120
17. The Backward Pass as an Adjoint Thread 130
18. Parameter Updates as a Long Trace 136
19. Dropout, Masks, and Silent Messages 141
20. Batches as Strands of Activity 146
21. Generation as Trace Production 150
22. Generalization, Memorization, and Motifs 157
23. What a Checkpoint Records 162
Part III. Connections, Borrowings, and Limits 174
24. Interpretability as Trace Analysis 175
25. Observability for Machine Learning 182
26. Patterns the Language Borrowed from Machine Learning 191
27. Four Directions, One Table 197
28. Limits of Formalization 201
Appendix A. Pattern Glosses 210
Appendix B. Coverage of the Pattern Language 234
Glossary 253
Notes and Sources 260
References 267
Index 275