Agent Harness

An agent harness is the software shell around a language model that turns it into an agent — "everything in an AI agent except the model itself," as Birgitta Böckeler puts it in her martinfowler.com analysis (Harness engineering for coding agent users, 2026-04-02). It spans four surfaces: the loop that feeds the model context and executes its tool calls, the tool interfaces the agent acts through, the machinery that manages the context window across turns, and the permission boundary that decides what the agent may touch. Anthropic condenses the runtime behavior into one cycle — gather context, take action, verify work, repeat (Building agents with the Claude Agent SDK — Anthropic, 2025-09-29). The model supplies the reasoning; the harness determines what the model sees, what it can do, and how its output gets checked.

Why it matters

Harness changes move outcomes without touching the model. Anthropic states that "Claude Sonnet 3.5 achieved state-of-the-art performance on the SWE-bench Verified evaluation after we made precise refinements to tool descriptions" — wording, not weights (Writing effective tools for agents — Anthropic). The SWE-bench figure is historical, but the lever it demonstrates is current: a mediocre harness quietly caps a strong model, and the harness is the part of the system a team fully owns.

The practice has consolidated into a named discipline. Mitchell Hashimoto's post of 2026-02-05 defines harness engineering as "the idea that anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again," and places it as step 5 — "Engineer the Harness" — of his AI adoption journey (My AI Adoption Journey — Mitchell Hashimoto). OpenAI published its own harness-engineering post in February 2026; according to InfoQ's coverage, it describes a three-person team that produced roughly 1,000,000 lines of production code and about 1,500 merged pull requests in roughly five months, with the code generated by Codex agents rather than written by hand (reported figures as of the post's 2026-02 publication) (OpenAI Introduces Harness Engineering — InfoQ).

How it works

Surface	Role	Representative levers
Agent loop	Drives gather → act → verify cycles until the task resolves	turn limits, hooks, stop conditions
Tools	Defines which actions exist and what their results look like	consolidation, namespacing, response budgets
Context management	Decides what occupies the context-window each turn	context-compaction, memory files, subagent delegation
Permissions and isolation	Bounds what the agent may read, write, and reach	permission modes, allow/deny rules, sandboxing

The loop. In the Claude Agent SDK, a session starts from the prompt, system prompt, tool definitions, and conversation history; the model replies with text and/or tool calls; the harness executes each call and feeds results back; the cycle repeats until a response contains no tool calls. One "turn" is one such round trip. Hooks such as PreToolUse and PostToolUse run in the host application process — not inside the model's context window — and a PreToolUse rejection prevents the tool call from executing at all (How the agent loop works — Claude Docs).

Tools. More tools do not produce better outcomes. Anthropic's tool-design guidance: consolidate operations into fewer high-leverage tools (instead of list_users, list_events, and create_event, one schedule_event that finds availability and books it), namespace related tools under common prefixes, and return high-signal, token-lean results — its Slack-tool example carries the same information in 72 tokens instead of 206 (as of 2026-07-12) (Writing effective tools for agents — Anthropic). Claude Code caps a single tool response at 25,000 tokens by default and expects tools to paginate, filter, or truncate beyond that (as of 2026-07-12, same source) — the pressure behind techniques like tool-result-clearing. External integrations commonly enter the tool surface through MCP servers (Building agents with the Claude Agent SDK — Anthropic).

Context management. The system prompt, memory files (CLAUDE.md), tool definitions, and the full conversation history accumulate in context and do not reset between turns; only the stable prefix (system prompt, tool definitions, memory files) is served from prompt cache. Near the limit the SDK compacts automatically — summarizing older history while keeping recent exchanges — which can drop instructions given early in the conversation; persistent rules therefore belong in memory files, which are re-injected on every request (How the agent loop works — Claude Docs). Subagents are the isolation lever: each starts a fresh conversation with no parent history, and only its final response returns to the parent as a tool result (How the agent loop works — Claude Docs). The cost side is steep: Anthropic reports agents use about 4x more tokens than chat and multi-agent systems about 15x, with token usage alone explaining 80% of performance variance on the BrowseComp evaluation (as of the post's 2025-06-13 publication) (How we built our multi-agent research system — Anthropic).

Permissions and isolation. The Claude Agent SDK evaluates every tool call through a fixed pipeline — hooks → deny rules → ask rules → permission mode → allow rules → canUseTool callback; a hook that allows a call does not skip the deny and ask rules, and scoped deny rules hold even in bypassPermissions mode (Configure permissions — Claude Docs). Six permission modes span asking about everything (default) to never asking (bypassPermissions); when a parent session runs in bypassPermissions, acceptEdits, or auto mode, all subagents inherit that mode with no per-subagent override — auditing what an agent can touch means auditing the whole subagent tree (Configure permissions — Claude Docs). Sandboxing adds OS-level enforcement: Claude Code's sandbox confines writes to the working directory and routes network traffic through a domain-allowlisting proxy, built on Linux bubblewrap and macOS Seatbelt; Anthropic reports that "sandboxing safely reduces permission prompts by 84%" in internal usage (as of the feature's 2025-10-20 launch), attacking the approval fatigue that makes humans rubber-stamp prompts (Making Claude Code more secure and autonomous with sandboxing — Anthropic).

Böckeler's taxonomy is the most systematic public decomposition of these parts: "guides" steer the agent before it acts (docs, rules, reference material), "sensors" observe after it acts (linters, tests, type checkers, llm-as-judge evaluators), and each splits into fast deterministic computational checks and slower, non-deterministic inferential ones (Harness engineering for coding agent users).

Boundaries and misconceptions

A harness is not a workflow. Anthropic's distinction: "workflows are systems where LLMs and tools are orchestrated through predefined code paths," while agents "dynamically direct their own processes and tool usage" — and the standing advice is "finding the simplest solution possible, and only increasing complexity when needed," reserving agents for open-ended problems where the number of steps cannot be predicted (Building effective agents — Anthropic, 2024-12-19). If the steps are knowable in advance, a predefined workflow is cheaper and more predictable than any harness.

A harness is not model-portable. In Anthropic's long-running-apps case study (2026-03-24), Claude Sonnet 4.5 exhibited "context anxiety" (a context-rot-family failure mode) strongly enough that compaction alone was insufficient and explicit context resets became essential; with Opus 4.5 the author dropped context resets from the harness entirely, and for Opus 4.6 removed the sprint-decomposition construct, moving the evaluator to a single end-of-run pass (Harness design for long-running application development — Anthropic). Harness re-tuning is a recurring cost of every model upgrade, not a one-time build.

A harness does not establish functional correctness. Böckeler calls the behavioral harness largely unsolved: mechanical sensors catch maintainability and architecture drift, but checking whether the agent solved the right problem currently "puts a lot of faith into the AI-generated tests, that's not good enough yet" — and "the harness is most needed where it is hardest to build," in legacy codebases with accumulated technical debt (Harness engineering for coding agent users).

Self-evaluation is not verification. Even inside a tuned harness, "agents tend to respond by confidently praising the work—even when, to a human observer, the quality is obviously mediocre" (Harness design for long-running application development — Anthropic). Verification needs an evaluator independent of the producer — the motivation for llm-as-judge and evaluator-optimizer arrangements.

A harness is broader than context-engineering. Anthropic's SDK framing treats the loop, tool design, context management, and verification strategy as distinct engineering surfaces (Building agents with the Claude Agent SDK — Anthropic); context management is one subsystem of the harness, with tools, permissions, and verification as peers rather than details.

In practice

The original harness-engineering mechanism is plain text. In Hashimoto's Ghostty project, agent mistakes became permanent lines in an AGENTS.md file — "each line in that file is based on a bad agent behavior, and it almost completely resolved them all" — supplemented by small purpose-built scripts (screenshots, filtered test runs) documented in the same file (My AI Adoption Journey — Mitchell Hashimoto). Claude Code packages the full surface set as a product: the loop and memory files (How the agent loop works — Claude Docs), the permission pipeline (Configure permissions — Claude Docs), and the sandbox (Making Claude Code more secure and autonomous with sandboxing — Anthropic).

At the heavier end, Anthropic's multi-agent research system runs a lead agent (Claude Opus 4) that plans and delegates to parallel Claude Sonnet 4 subagents, saving the research plan to external memory because context beyond 200,000 tokens gets truncated; that configuration outperformed single-agent Claude Opus 4 by 90.2% on Anthropic's internal research eval (as of the post's 2025-06-13 publication), and its early failure modes read as a catalog of harness bugs — 50 subagents spawned for simple queries, duplicated work from ambiguous task division, endless searching for nonexistent sources (How we built our multi-agent research system — Anthropic).

Harness depth is a cost dial, not free quality. In Anthropic's long-running-apps case study, a solo-agent run on a game-building task cost $9 and took 20 minutes but produced broken gameplay, while the same task through a full planner/generator/evaluator harness cost about $200 over 6 hours and produced a playable game (as of 2026-03-24) (Harness design for long-running application development — Anthropic). The practical reading matches Anthropic's simplicity principle: start from the bare loop plus a tool interface and a permission boundary, and add layers — subagents, evaluators, sandboxing, compaction tuning — only against observed failure modes (Building effective agents — Anthropic).

Related

• claude-agent-sdk — the harness primitives above (loop, hooks, permissions, subagents) exposed programmatically.
• computer-use — GUI-driving tools; a tool category with its own isolation demands on the harness.
• context-compaction — the usual first lever when the context side of the harness saturates.
• evaluator-optimizer — the generator/evaluator split as a reusable verification structure.