Setting Up Your Local Agentic Workspace

Tested June 2026 on a Mac mini M2 Pro 32G and a MacBook Air M4 16G. See Troubleshooting for pinned versions.

Before you can run local coding agents, you need to configure your machine to balance the compute needs of local LLMs with the memory needs of your compiler, IDE, and development tools. This chapter guides you through setting up Ollama, installing Little-Coder skills, and optimizing your macOS workspace for a 32GB memory footprint.

Note: Dear reader, the most common configuration problem I have seen people have (and it has hit me also!) is forgetting to manually configure Ollama with a large enough context size. Be warned that I will keep mentioning this!

* * *

0. Prerequisites

Check these before you pull any model:

  • macOS on Apple Silicon (tested), or Linux with NVIDIA, AMD, or Intel plus Ollama support.
  • RAM: 32GB recommended, 16GB workable for small code bases (several hundred lines).
  • Disk: 20GB free per model (weights plus context overhead).
  • Homebrew, Node 22 or later, git, and a terminal editor you know.
  • No admin rights needed except for installing Homebrew packages.
* * *

1. Choosing a Gemma 4 Model

We center this book on Google Gemma 4 models in 4-bit Quantization-Aware Training (QAT) form. Pick by your RAM. Terms are defined in the Glossary chapter.

1. Dense Gemma 4 31B (gemma4:31b-it-qat)

Peak reasoning for the Gemma 4 family. The 4-bit QAT form needs 18GB-20GB.

  • Pros: Peak deep-reasoning, continuous logical chains, robust architectural layout. No Mixture-of-Experts (MoE) routing overhead.
  • Cons: Slower token throughput because each token touches all 31 billion parameters. On 32GB it leaves less than 12GB for macOS, IDE caches, Docker, and agent scratch processes, which causes swap degradation during wide-context queries. Only pick this on 64GB or more.

2. MoE Gemma 4 26B with 4B Active (gemma4:26b-a4b-it-qat)

The Mixture-of-Experts variant. The 4-bit QAT form needs roughly 15GB while only 4 billion parameters activate per token.

  • Pros: Fast inference (only 4B active per pass), headroom for IDEs and build systems, full 256K context windows, strong retention from QAT.
  • Cons: Minor quality loss in open-ended creative prose. Functionally equal to dense for code, syntax checks, and function calling.

3. Gemma 4 12B Dense (gemma4:12b-it-qat)

The lean dense variant at roughly 7-8GB.

  • Pros: Low hardware barrier, runs beside heavy dev environments. Zero routing overhead, uniform latency. QAT preserves logic and instruction following better than standard post-training quantization.
  • Cons: Lower ceiling on complex multi-step algorithms. Past 128K context, retrieval degrades faster.

Rule: gemma4:26b-a4b-it-qat is the daily driver on 32GB. gemma4:12b-it-qat is the sweet spot on 16GB.

* * *

2. Installing Ollama and Downloading Gemma 4

Ollama is the standard runtime for running open LLMs locally on macOS. It uses Apple’s Metal API to run model weights directly on the Mac’s unified GPU. Ollama uses an up to date llama.cpp core that supports the QAT model format we use here.

Step 1: Install Ollama

Download and install the native macOS application from the official site, or use Homebrew:

1 brew install ollama

Step 2: Launch the Ollama Service

Ensure Ollama is not running in the background (if on macOS you see the Ollama icon in your menubar, please use the drop down menu to quit the application). You should use Ollama via a terminal shell:

1 OLLAMA_CONTEXT_LENGTH=32768 ollama serve

You must run with a larger than the default 4K context. Context rule: use 32768 on 32GB systems and 16384 on 16GB systems. On 16GB you can try 32768, and in several experiments file writing errors cleared at 32768 even on 16GB, but watch for swap. If token speed drops, drop back to 16384.

Step 3: Pull the Gemma 4 MoE Model

For a 32GB Mac, pull the 26B Mixture-of-Experts variant with 4-bit QAT:

1 ollama pull gemma4:26b-a4b-it-qat

Note: If you have a Mac with 64GB or more of unified memory, you can optionally pull the dense variant ollama pull gemma4:31b-it-qat for peak reasoning. On a 16GB Mac pull the gemma4:12b-it-qat model.

You might wonder what -it- in model names refers to. These models are instruction trained (not base models).

Step 4: Install little-coder/pi

1 brew install node
2 npm config set ignore-scripts true

The second command stops install scripts from running when installing npm libraries.

Use one of these two installation methods:

1 curl -fsSL https://raw.githubusercontent.com/itayinbarr/little-coder/main/install.sh | bash # method 1
2 npm install -g little-coder # method 2

I also define in my ~/.profile:

1 alias lc='little-coder --model ollama/gemma4:26b-a4b-it-qat'
2 alias lcf='little-coder --model ollama/gemma4:12b-it-qat'
3 # Append functional programming tools to little-coder's bash whitelist
4 export LITTLE_CODER_BASH_ALLOW="sbcl ,clojure ,clj ,npx "
* * *

3. Deploying Little-Coder Skills

Little-Coder configures the agent’s behavior by loading structured instruction templates called skills from the user’s home directory.

To install the skills from the repository https://github.com/mark-watson/little-coder_Local_Coding_Book, copy the .pi/agent/skills folder to your home directory:

1 # Copy the skills to your home folder config path
2 mkdir -p ~/.pi/agent
3 cp -R .pi/agent/skills/ ~/.pi/agent/skills/

After a first time install of pi and little-coder the directory will not contain a ~/.pi/agent/skills folder. The copy command installs the skill files I wrote for my own use.

Verify the structure is set up correctly:

1 ls -R ~/.pi

You should see a directory layout matching this:

1 ~/.pi/
2 └── agent/
3     └── skills/
4         ├── clojure.md
5         ├── common-lisp.md
6         ├── python.md
7         ├── typescript.md
8         └── write-temp-strategy.md

Each markdown file inside ~/.pi/agent/skills/ holds system instructions, CLI tool invariants, and execution rules that the agent loads when working on projects of that language type. The write-temp-strategy.md file defines the Write-and-Notify strategy used through this book: the agent writes full implementations to new tmp_* files and leaves the merge to you instead of editing your files in place.

Verifying That Our New Little-Coder Skills Are Loaded and Available

little-coder is built on top of pi. However little-coder strips the substrate down to a hyper-lean, non-interactive execution setup (purely loading the files directly into the system prompt context at launch), it doesn’t process slash commands like /skills inside the chat terminal.

Since little-coder treats skills purely as filesystem assets, you manage and inspect them entirely using standard shell commands outside the agent interface.

Fortunately we can simply ask what skills are available. I asked what runtime rules were loaded for Lisp and temp file creation, and the agent listed clojure (clojure.md), common-lisp (common-lisp.md), and the Write-and-Notify skill (write-temp-strategy.md), plus the Write refusal invariant. A second prompt for TypeScript surfaced the typescript skill (typescript.md). Both answers matched the files on disk, so setup was good.

As a test of this setup I prompted: “write a library file web-scraper.ts” with four APIs (plain text, Markdown, links, H1/H2/H3 text). Using gemma4:26b-a4b-it-qat this one-shotted working files (package.json, test-scraper.ts, web-scraper.ts, package-lock.json). A second prompt generated a quality README.md documenting npm tsx test-scraper.ts.

* * *

4. Optimizing Your Developer Environment (VRAM and RAM Conservation)

On a 32GB Mac, running a 15GB model leaves 17GB of unified memory. Since macOS, WindowServer, and display buffers consume 4GB-6GB, you have roughly 11GB-13GB of physical RAM left for your developer environment.

To prevent macOS from resorting to disk-swapping (which drastically slows down LLM token generation), adopt a low-memory dev setup:

Terminal-Based Editors (Emacs)

Avoid heavy graphical IDEs when running large local models. A lightweight terminal text editor is ideal. For instance, running Emacs in non-windowed (terminal) mode uses a fraction of the RAM of a GUI instance. For Vi/Vim users, of course use your favorite editor.

[!TIP] Add this alias to your shell configuration file (~/.zshrc or ~/.bashrc) to make terminal editing instant and zero-overhead:

1 alias e="emacs -nw"

Now, typing e filename opens your files directly inside your terminal, bypassing GUI rendering caches.

Other Optimization Rules

  • Limit Browser Tabs: Web browsers are notorious RAM hogs. Keep development tabs focused.
  • Stop Unused Docker Containers: Run docker system prune and stop any database or application containers not actively needed for the current debug loop.
  • Disable Heavy File Indexers: IDEs that constantly re-index the entire disk in the background will compete with Ollama for CPU cycles and memory.

Previous

Preface: The Rise of Local Coding Agents

Up next

Security Analysis