An agent is a model and a harness. The harness runs tools, holds state, manages permissions, and feeds context back to the model. With a local model, the harness matters more. Small context windows and weaker tool calling expose every design flaw. This guide ranks 11 open-source harnesses by how well they document local inference. All repo facts were read from GitHub on September 18, 2026. The ranking weighs 4 things: OSI-approved license, documented local runtimes, maintenance status, and safety controls. The 3 rules that apply to every harness 1. Raise the context window first : Per Ollama’s context length docs , defaults depend on VRAM: 4k under 24 GiB, 32k from 24 to 48 GiB, and 256k at 48 GiB or more. The same page says agents and coding tools should get at least 64,000 tokens. The fix is one line: OLLAMA_CONTEXT_LENGTH=64000 ollama serve . 2. Pick a model that supports tool calling : Goose’s provider docs state that models without tool calling can only do chat completion. With llama.cpp, Pi’s docs note the --jinja flag enables compatible chat templates and tool calling. 3. Budget memory honestly : Cline’s local guide maps 16 to 32GB RAM to small quantized models, 32 to 64GB to mid-size coding models, and 64GB or more to larger models. Ollama’s Hermes page lists gemma4 at about 16 GB VRAM and qwen3.6 at about 24 GB VRAM. 1. OpenCode OpenCode documents 3 local paths in its own provider docs : Ollama, LM Studio, and llama.cpp’s llama-server . Each uses the @ai-sdk/openai-compatible package with a local baseURL . The docs claim support for 75+ providers overall. Setup can
Source: MarkTechPost