Using OpenCode as a Local-First Agentic Coding Assistant

Where Hermes Agent provides a general-purpose personal agent that grows with you across messaging platforms and scheduled tasks, OpenCode (opencode.ai) focuses specifically on the coding workflow. It is an open-source AI coding agent that operates in your terminal, IDE, or desktop application. With over 150,000 GitHub stars and 6.5 million monthly users, OpenCode has become one of the most widely adopted agentic coding tools in the open-source ecosystem. For our Small AI purposes, its most important feature is that it is completely model-agnostic: it supports over 75 LLM providers, including local models via Ollama, meaning you can run a fully capable agentic coding assistant without sending a single line of your code to the cloud.

What Is OpenCode?

OpenCode is not a simple chat interface wrapped around an API. It is a full agentic system that can read your codebase, write and modify files, execute terminal commands, perform code searches, and integrate with Language Server Protocol (LSP) diagnostics, providing the model with the same code intelligence that your IDE uses. It operates in two distinct modes:

  • Plan Mode: A read-only analysis mode where the agent explores your codebase and proposes implementation strategies without making any changes. Toggle into Plan Mode by pressing the Tab key. This is useful for understanding unfamiliar code or reviewing a proposed approach before committing to modifications.
  • Build Mode: Full read/write access where the agent can modify files, run commands, and perform actual work. Toggle back to Build Mode with Tab when you are ready to let the agent execute.

This dual-mode architecture is a thoughtful design decision. It separates the “thinking” phase from the “doing” phase, giving you explicit control over when the agent is allowed to make changes. For Small AI practitioners, this is especially valuable because smaller models benefit from the structured constraint: you can use Plan Mode to verify the agent’s reasoning before giving it permission to act.

Key Features

  • LSP Integration: OpenCode automatically loads the appropriate Language Server for your project, providing the AI model with real-time diagnostics, type information, and code navigation data. This means the model is not guessing at your code structure; it has access to the same semantic understanding your editor provides.
  • Multi-Session Support: You can run multiple agents in parallel on the same project, each handling a different task without interfering with each other.
  • Session Sharing: Any conversation can be shared via a link for reference or debugging with teammates.
  • Undo/Redo: The /undo command reverts the agent’s most recent changes, restoring your original code and prompt so you can iterate. You can undo multiple times to walk back through a sequence of changes.
  • File References: Use the @ key to fuzzy search for files in your project and include them as context in your prompt.
  • Privacy First: OpenCode does not store any of your code or context data. It can operate in privacy-sensitive environments where sending code to external services is unacceptable.

Installing OpenCode

On macOS, the simplest installation method uses Homebrew:

1 brew install anomalyco/tap/opencode

Alternatively, you can use the install script:

1 curl -fsSL https://opencode.ai/install | bash

Or install via npm:

1 npm install -g opencode-ai

Configuring OpenCode for Local Models

Dear reader, the configuration step is where OpenCode becomes relevant to our Small AI discussion. While many users connect it to cloud providers like Anthropic or OpenAI, we can configure it to use local models via Ollama, keeping our code and context entirely on our own machine.

First, ensure Ollama is running with a capable model pulled:

1 ollama pull qwen3.5:9b

Then configure OpenCode to use Ollama’s OpenAI-compatible endpoint. Create or edit the configuration file at ~/.config/opencode/config.json:

 1 {
 2   "provider": {
 3     "ollama": {
 4       "api_key": "ollama",
 5       "base_url": "http://localhost:11434/v1",
 6       "models": {
 7         "qwen3:4b": {
 8           "name": "qwen3.5:9b",
 9           "max_tokens": 4096,
10           "can_reason": true
11         }
12       }
13     }
14   }
15 }

Note that OpenCode connects to Ollama’s /v1 endpoint (the OpenAI-compatible API), not the native Ollama /api endpoint. The api_key field can be set to any non-empty string since Ollama does not require authentication.

Now navigate to your project directory and launch OpenCode:

1 cd /path/to/your/project
2 opencode

Letting Ollama Configure OpenCode For You

On a 16G laptop, run the Olla service with:

1 OLLAMA_CONTEXT_LENGTH=16384 ollama serve

On a 32G laptop I use:

1 OLLAMA_CONTEXT_LENGTH=32768 ollama serve

1 ollama launch opencode --model qwen3.5:9b

This runs on a 16G memory system. If you have less memory, you can use a smaller model like qwen3.5:1.7b or qwen3.5:4b. If you have 32G or more memory, try qwen3.6:27b-coding-nvfp4.

As sample prompt might be:

1 Look at the Python source code and uv config and write the README.md file

A Practical Workflow With OpenCode

Initializing a Project

On first launch in a new project, run the /init command. This tells OpenCode to analyze your project structure and create an AGENTS.md file in the project root that helps it understand your codebase conventions:

1 /init

You should commit this AGENTS.md file to Git. It serves as a persistent context document that improves the agent’s understanding of your project across sessions.

Asking Questions About Your Code

In Plan Mode, you can explore unfamiliar code:

1 <TAB>  (switch to Plan Mode)
2 
3 How is the RAG pipeline structured in @source-code/RAG_zvec/zvec_RAG_app_gemini.py?

OpenCode reads the referenced file, analyzes its structure using LSP data, and provides a detailed explanation without modifying anything. The @ prefix triggers a fuzzy file search so you do not need to type the full path.

Building a Feature

Once you understand the codebase, switch to Build Mode and describe what you want:

1 <TAB>  (switch to Build Mode)
2 
3 Add a --topk command line argument to the RAG app that
4 controls how many chunks are retrieved. Default should
5 be 5. Update the search function to use this value.

OpenCode will modify the relevant files, add the argument parsing logic, and thread the value through to the search function. You can review the changes in your editor and use /undo if anything looks wrong.

Iterating on Changes

If the agent’s output is not quite right, you can undo and refine:

1 /undo
2 
3 Add a --topk command line argument using argparse. Make sure
4 the help text explains that higher values retrieve more
5 context but increase latency. Default to 5.

The /undo command restores your files and re-presents your original prompt so you can modify it. This rapid iteration loop is where OpenCode excels: you are having a conversation with the agent about your code, with the ability to rewind at any point.

Small AI Considerations for OpenCode

Using OpenCode with local models involves the same trade-offs we have discussed throughout this book:

When Small Models Work Well

  • Code navigation and explanation: Asking the agent to explain existing code, trace data flow, or summarize function signatures. These tasks rely on pattern matching and the model’s training on code, where even small models perform well.
  • Structured modifications: Adding command line arguments, refactoring function signatures, updating configuration files. These tasks have clear patterns and constrained output.
  • Boilerplate generation: Creating test files, README templates, configuration scaffolding. The output format is well-defined and mechanical.

When to Escalate to a Larger Model

  • Complex architectural decisions: Designing new systems from scratch, choosing between competing design patterns, or refactoring large interconnected modules.
  • Cross-file reasoning: Tasks that require understanding the relationships between many files simultaneously, especially in large codebases where context window limitations of smaller models become a bottleneck.
  • Subtle bug diagnosis: When the bug involves complex state interactions or race conditions that require deep reasoning chains.

The practical approach is to start with a local model for day-to-day work and switch providers when you hit a wall. OpenCode makes this straightforward: you can change providers through the configuration without restarting your session.

OpenCode Zen: Curated Models for Coding

OpenCode also offers a service called Zen which provides access to a curated set of AI models that have been specifically tested and benchmarked for coding agent performance. This removes the guesswork of choosing between the hundreds of available models and ensures consistent quality. Zen is a paid service but eliminates the need to manage separate API keys and subscriptions for multiple providers.

For our Small AI philosophy, the important takeaway is that not all models perform equally well as coding agents, even models with similar benchmark scores on general tasks. The Zen service validates this principle: model selection for agentic coding is a specialized evaluation problem, and the models that excel at it are not always the largest or most expensive.

Wrap Up for OpenCode

OpenCode brings the agentic coding paradigm to your terminal with a focus on privacy, model flexibility, and developer control. Its dual Plan/Build mode architecture provides a safety net that is especially valuable when working with smaller models, allowing you to verify the agent’s reasoning before granting it write access to your codebase. The LSP integration gives even small models access to semantic code understanding that partially compensates for their smaller training sets. For Small AI practitioners, OpenCode demonstrates that a world-class agentic coding experience does not require sending your proprietary code to a cloud API. You can run it entirely locally with Ollama, accepting the trade-offs in model capability while gaining complete privacy and zero marginal cost per query. Explore the project at opencode.ai and on GitHub at github.com/anomalyco/opencode.

Up next

Wrap Up for Winning Big With Small AI