Leanpub Header

Skip to main content
Go to Leanpub.com

Data Science Bootstrap Notes

Set up your machine, structure your projects, and work at the speed of thought.

This book is 100% completeLast updated on 2026-07-23
Eric Ma

Most data science programs teach the math. Almost none teach you how to set up your machine, structure a project, or make your work reproducible. This book is the missing half, distilled from a decade of hard-won lessons into a short guide you can read in a weekend.

Minimum price

$7.99

$14.99

You pay

Author earns

$

Also available for 1 book credit with a Reader Membership

PDF
EPUB
About

About

About the Book

Your model is solid. Your statistics are sound. But your environment just broke, your project is a folder of untitled notebooks, and the code that ran fine yesterday won't run today.

Most data science programs teach the math, the models, and the domain. Almost none teach how to set up the machine underneath it all, structure a project so it survives six months, or build a workflow that runs identically on your laptop, your teammate's machine, and in production. That missing half is where data scientists lose days, miss deadlines, and quietly wonder whether everyone else has this figured out.

They don't. They just hit the same walls earlier, or they had a mentor.

Data Science Bootstrap is that mentor. It distills a decade of hard-won infrastructure lessons, battle-tested since I left bench science for computational work in 2013, into a short guide you can read in a weekend. You get the four philosophies that turn chaos into compounding productivity, the exact tools and configs to set up a real machine, a project structure that scales, the core skills that make your work hold up, and the modern AI-augmented workflow that lets you move at the speed of thought.

What's inside:

  • The four philosophies that turn chaos into compounding productivity, so each new project starts further ahead than the last.
  • How to configure a Mac or Linux machine for data science in an afternoon, with a dotfiles repo you can version and replicate anywhere, which means a new machine or a new teammate takes hours, not days.
  • Reproducible environments with pixi that end "it works on my machine," so your code runs identically on your laptop, your colleague's machine, and in CI.
  • A project structure that scales from a throwaway script to production, which means future-you isn't lost in a swamp of untitled notebooks.
  • Shell mastery that turns twenty keystrokes into two, so the boring stuff stops eating your day.
  • How to write tests for data science code and models, so you catch breaks before they reach production and change your code without fear.
  • The modern AI-augmented workflow: repository memory with AGENTS.md, reusable skills, and agentic data science with marimo, so your coding agents compound in leverage instead of starting from zero.
  • CI/CD automation that builds, tests, and ships while you sleep, which means the drudgery disappears and you focus on the work that matters.

Why the eBook, when the knowledge base is free?

The online version will always be free; that's the point of the project. The eBook gives you the same content linearized into a focused read, portable offline in PDF and EPUB, and updated free for as long as I maintain it. It's also the most direct way to support the hundreds of hours behind it.

Who wrote this:

I'm Eric Ma, Senior Principal Data Scientist at Moderna, where I lead the Data Science and AI Research team. MIT ScD, formerly biomedical data science at Novartis, a past core developer on NetworkX and PyMC, and creator of pyjanitor and nxviz. Every practice here was forged in real projects, real collaborations, and real deadlines, not in a classroom.

Share this book

Categories

Data ScienceDevOpsProgramming Cookbooks

Feedback

Email the Author

Author

About the Author

Eric Ma

As Senior Principal Data Scientist at Moderna Eric leads the Data Science and Artificial Intelligence (Research and Development) teams to accelerate science to the speed of thought. Prior to Moderna, he was at the Novartis Institutes for Biomedical Research conducting biomedical data science research with a focus on using Bayesian statistical methods in the service of discovering medicines for patients. Prior to Novartis, he was an Insight Health Data Fellow in the summer of 2017 and defended his doctoral thesis in the Department of Biological Engineering at MIT in the spring of 2017.

Eric is also an open-source software developer and has led the development of pyjanitor, a clean API for cleaning data in Python, and nxviz, a visualization package for NetworkX. He is also on the core developer team of NetworkX and PyMC. In addition, he gives back to the community through code contributions, blogging, teaching, and writing.

His personal life motto is found in the Gospel of Luke 12:48.

Contents

Table of Contents

The Data Science Bootstrap Notes

  1. Where I think you, the reader, are coming from
  2. Things you’ll learn
  3. Apply these ideas just-in-time
  4. Changes from the first edition
  5. Ways to support the project

Philosophies

  1. How these philosophies work together

You should know your computing stack

  1. See this philosophy in action

Automate and standardize everywhere possible

  1. See this philosophy in action

You should always know the source of truth

  1. Repository standards for tools and agents
  2. See this philosophy in action

Categorize everything that you can

  1. See this philosophy in action

Putting it all together

  1. How you’ll see these philosophies in action

Setup your machine

  1. Why this is important
  2. What you’ll learn in this section

Configure your shell

  1. Install Starship
  2. Configure environment variables
  3. Create shell aliases
  4. Troubleshooting
  5. Quick Reference

Install and configure system-wide software

  1. Install package managers
  2. Install software
  3. Configure your PATH
  4. TL;DR: Quick installation commands
  5. Troubleshooting
  6. Quick Reference

Install and configure Git on your machine

  1. Why do we need Git
  2. How to install Git
  3. How to configure Git with basic information
  4. How to configure Git with fancy features
  5. Troubleshooting
  6. Productivity Tip: Shell Aliases
  7. Quick Reference

Install uv to manage and install Python-based command line tools

  1. Further reading

Install Homebrew on your Mac (fallback package manager)

  1. Why install Homebrew?
  2. When to use Homebrew
  3. How to install Homebrew
  4. Using Homebrew on Linux
  5. See also

Install and configure direnv for environment management

  1. Why we need direnv
  2. How to install direnv
  3. How to configure direnv
  4. Loading .env files automatically
  5. Troubleshooting

Leverage dotfiles to get your machine configured quickly

  1. Why create a dotfiles repository
  2. How to structure a dotfiles repository
  3. Examples and resources

Configure VSCode for maximum productivity

  1. How do I access VSCode settings?
  2. What built-in settings have transformed my workflow?
  3. What extensions have actually improved my productivity?
  4. What keyboard shortcuts do I actually use?
  5. How do I handle project-specific settings?
  6. What about collaborative coding?
  7. AI Agent Harnesses
  8. Remember: start simple, grow gradually

Master your shell for data science productivity

  1. Why shell mastery matters for data scientists
  2. What you’ll learn in this section

Take full control of your shell environment variables

  1. Why control your environment variables
  2. How do I control my environment variables

Create shell command aliases for your commonly used commands

  1. Why create shell aliases
  2. How to create aliases
  3. Where to store these aliases
  4. Useful aliases to get started
  5. Git aliases cheat sheet
  6. Port management aliases
  7. Enhancing built-in commands with functions

Shell commands cheat sheet

  1. Basic navigation and file operations
  2. File permissions and ownership
  3. Process management
  4. Network and system info
  5. Archive and compression
  6. Git shortcuts
  7. Text processing
  8. Advanced patterns
  9. Time-saving tips

Shell-based text editors

  1. Why should I care about shell editors?
  2. What do I actually need to learn?
  3. My recommendation: start with nano
  4. What about vim and emacs?
  5. My philosophy on shell editors
  6. Getting started

Manage and configure your projects

  1. Follow the 1:1:1:1… rule
  2. When can we break this rule

Start with a sane repository structure

  1. How to structure a standard repository structure
  2. Automate the scaffolding of new projects

Use pixi for maximally ergonomic and reproducible environments

  1. A practical onboarding path (clone to first command)
  2. Choose one config layout on purpose
  3. Pixi command cheat sheet
  4. Long-term reproducibility through lock files
  5. Composable multi-environment projects

Structure your source code repository sanely

  1. Phase 1: Initial Exploration
  2. Phase 2: Emerging Patterns
  3. Phase 3: One-off Scripts
  4. Phase 4: Production Structure
  5. Leveraging AI Assistants
  6. Core Development Principles

Store your project documentation in your project repository

  1. Introduction to the Diataxis framework
  2. When to add documentation
  3. Code comments as documentation
  4. AI assistance in documentation
  5. Reference documentation
  6. Automating documentation with CI/CD
  7. Auditing and improving documentation

Use CI/CD to automate tasks

  1. Key Concepts of CI/CD
  2. Environment Considerations
  3. Configuration and Environment Variables
  4. Leveraging Pixi Environment
  5. Practical Examples with GitHub Actions

Use data catalogs to manage data

  1. What are data catalogs?
  2. Traditional data catalog examples
  3. Modern ML data storage with xarray and zarr
  4. What are the advantages of data catalogs?
  5. When should you use data catalogs?

Choose your data formats wisely

  1. The binary vs text format decision
  2. The hybrid approach: Binary source, text derivatives
  3. High-dimensional data: The xarray advantage
  4. Format-specific recommendations
  5. How to implement this in practice
  6. The bottom line

Take advantage of uv for one-off projects

  1. Using PEP723 for script dependencies
  2. Self-contained notebooks
  3. Benefits of this approach
  4. Real-world examples
  5. Conclusion

Configuration files guide

  1. Why do we even need all these config files?
  2. What are the core configuration files you should know about?
  3. What about documentation configuration?
  4. How do I handle environment variables?
  5. What’s the quick reference for which tools use which files?
  6. What are my best practices for configuration files?
  7. How do I get started with all this?

Set environment variables in a .env file

  1. Why configure environment variables per project
  2. How to configure environment variables for your project

Name things consistently

  1. What constitutes a “sane” name?

Skills for Effective Data Science

  1. Core Technical Skills
  2. Effective Ways of Working

How to write software tests

  1. Using AI coding agents for tests
  2. What this chapter covers (and what it skips)
  3. Why should I bother writing tests?
  4. How do I actually write tests?
  5. What do real tests look like?
  6. What about testing data assumptions?
  7. Reproducibility habits that tests can enforce
  8. Smoke checks for models and training code
  9. Don’t forget to test error conditions
  10. How do I actually run these tests?
  11. What are some advanced patterns worth knowing?
  12. How do I organize my tests?
  13. How do I know if I’m testing enough?
  14. How does this fit into my development workflow?
  15. What mistakes should I avoid?
  16. How do I get started?

Refactor code

Collaborating on Data Science Projects

  1. The power of pair programming
  2. Leveraging AI in collaborative work
  3. The science in data science projects
  4. Effective work distribution
  5. Pull requests, review, and what CI enforces
  6. Handling merge conflicts without the drama
  7. The art of managing unproductive patches
  8. Scaling tacit knowledge

Use notebooks effectively

  1. Choose Marimo over Jupyter for reactivity
  2. Notebooks as prototyping tools, not production code
  3. Data access best practices
  4. Scratch pad vs. Report-style notebooks
  5. Refactor with the help of AI
  6. Publish notebooks and strip outputs before committing
  7. Jupyter hygiene (if you must use Jupyter)

Working with AI tools

  1. The speed of thought
  2. The right kind of lazy
  3. AI as a mirror of human capital
  4. Effective patterns for AI interaction
  5. Beyond code generation
  6. See this in action
  7. Moving forward

Building repository memory with AGENTS.md

  1. What is AGENTS.md?
  2. The two jobs of repository memory
  3. Training an employee, not programming a bot
  4. Durable norms and corrections
  5. Examples worth spelling out
  6. Bootstrap your AGENTS.md
  7. See this in action

Skills as reusable playbooks

  1. What is a skill?
  2. Recommended installation
  3. Starter pack
  4. Tacit knowledge and the “ESL benefit”
  5. The iteration loop
  6. See this in action

Compounding agent improvement

  1. The maturity model
  2. Decision rule: AGENTS.md vs. Skills
  3. Markdown as an executable language
  4. The metacognition habit
  5. Where this is going

Safe automation with coding agents

  1. Auto-approve safe command line commands
  2. Enable automatic web search
  3. Know your emergency stop shortcuts
  4. Correct agent behavior in real-time
  5. Write prescriptive prompts for complex tasks
  6. Use plan mode for complex tasks
  7. Managing multiple background agents
  8. See this in action

Making surgical changes to large codebases

  1. The process
  2. Why this works
  3. Building your mental model
  4. The constraint of three
  5. See this in action

Advanced workflows with coding agents

  1. Working with specialized tools
  2. Working across repositories
  3. See this in action

Agentic data science with marimo pair

  1. What marimo pair gives you
  2. Get set up
  3. Frame the question before you start typing
  4. Work in small, precise asks
  5. When the loop breaks down
  6. How this connects to the rest of the book

Looking back, moving forward

  1. What we’ve covered together
  2. The journey ahead
  3. A personal note
  4. Where to go from here
  5. Final thoughts

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms...

Earn $8 on a $10 Purchase, and $16 on a $20 Purchase

We pay 80% royalties on purchases of $7.99 or more, and 80% royalties minus a 50 cent flat fee on purchases between $0.99 and $7.98. You earn $8 on a $10 sale, and $16 on a $20 sale. So, if we sell 5000 non-refunded copies of your book for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earned over $15 million writing, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub