Preface: The Rise of Local Coding Agents

OK dear reader, “The Rise of Local Coding Agents” is very optimistic, but that is the future that I want to see. Using local models saves energy and money but we still need to get our work done. The trick is dividing work into what can be efficiently performed using local models and what actually requires frontier models from hyper scalers like Google (providing Gemini), Anthropic (providing Claude), Alibaba Cloud (providing Qwen models), OpenAI (gpt models), DeepSeek (providing DeepSeek v4 flash and pro), etc. I use Gemini, DeepSeek, and Claude sparingly just as needed.

I currently use the setup described in this short (and free!) book for about half of my Python and TypeScript development and experiment with local agentic coding for more niche languages like Common Lisp and Clojure. The Python chapter shows a full skill and example, so Python readers get the same treatment as TypeScript readers.

Note: Dear Reader, running any agentic coding tools bare on your working computer has security risks as discussed in later chapters. This comment applies to all agentic coding harnesses including little-coder, Hermes Agent, Aider, Cline, OpenHands (formerly OpenDevin), Claude Code, gemini-cli, and Swe-agent. In two later chapters we look at two techniques for partially locking down security for agentic coding: running as a non privileged user account, and running inside Apple Containers.

This book reflects my own efforts to maximize the utility of my two home computers, a Mac mini M2-Pro 32G (an old computer) and a MacBook Air M4 16G for local agentic coding. As I write this in June 2026 I am using Itay Inbar’s little-coder project that wraps the pi open source agentic coding harness, optimizing for smaller models and less powerful local computers. The material in this book can also be used with home systems with NVIDIA GPUs, AMD, Intel, etc. because I use Ollama to serve local models and Ollama supports many hardware architectures. pi and little-coder are also portable.

You need to clone the GitHub repository https://github.com/mark-watson/little-coder_Local_Coding_Book that contains example material for this book as well as the complete manuscript.

Note: Dear reader, I have written three other books covering applications of small local models. This book solely covers agentic coding on local computers and does not cover applications using local models.

We are entering a new era of software engineering. The traditional model of writing code entirely by hand is being augmented, and in some cases replaced, by local agentic coding systems. Local coding agents are autonomous or semi-autonomous AI loops that read your codebase, write files, run test suites, and debug compiler errors directly on your machine.

Historically, high-quality coding agents required calls to massive, cloud-hosted proprietary models. However, cloud-hosted agents raise significant concerns:

1. Data Privacy: Sending proprietary, client, or sensitive company code to external APIs is often a security violation.
2. Network Latency: Large models running across WAN connections introduce structural delays that slow down interactive development.
3. Execution Safety: Cloud models do not have direct, low-latency access to run and test code in your local shell unless you run complex, insecure gateway daemons.
4. Environmental Costs: Data centers are expensive, the latest hardware deprecates quickly, and there is protecting the environment, if you care about that.

By moving the coding agent local when possible, you keep your code on your hardware, eliminate network delays, and allow the agent to interface directly with your terminal, compilers, and debuggers.

Dear reader, while you can easily use the content of this book on all systems that support the open source little-coder, pi, and Ollama (based on the wonderful llama.cpp project), all of my testing is on Apple Silicon systems. If you use NVIDIA, Intel, or AMD based systems, this book should support you.

Local execution comes with a cost: memory footprint. While Apple Silicon Macs share unified memory across both CPU and GPU operations, their pools are finite. For developer machines with a strict 32GB unified memory boundary (such as my Mac Mini configuration), choosing the right local model size and architecture is the difference between a fluid, almost instantaneous coding assistant and a freezing system bogged down by heavy memory swapping. We also use a 16GB system for examples in this book, and even more care and effort is required. Dear reader, just to set your expectations: currently I only work on small code bases (several hundred lines of code) and on generating documentation on my 16GB system.

In this book, we center our stack on Google Gemma 4 models. The full comparison of the three variants lives in the Setup chapter, where you act on it:

• gemma4:31b-it-qat (dense 31B): peak reasoning, needs 18GB-20GB, only for 64GB machines.
• gemma4:26b-a4b-it-qat (MoE 26B, 4B active): the daily driver for 32GB systems at roughly 15GB.
• gemma4:12b-it-qat (dense 12B): the sweet spot for 16GB systems at roughly 7-8GB.

Alibaba Cloud Qwen Models: I frequently use qwen models for both experimenting with agentic coding and frequently use qwen models embedded in applications. My choice of Gemma models here is practical: it is easier to pick two good models and concentrate on configuring little-coder.

This book explores Little-Coder, a lightweight agentic tool execution framework designed to orchestrate local LLMs. Instead of relying on complex, bloated agent platforms, Little-Coder utilizes specialized skills files (.pi configuration format) to teach local models how to interact safely with specific languages and build tools without hanging, looping, or corrupting code.

By combining Google Gemma 4 models with Little-Coder’s and pi’s precise execution skills, you can turn a standard local computer into an effective local agentic development environment. Let’s get started.