harness engineering 101

On July 9, OpenAI launched ChatGPT Work. Within 48 hours, an agent running its flagship GPT-5.6 Sol model wiped most of AI investor Matt Shumer's home directory. The cause was a shell variable parsing error that turned a routine cleanup task into a recursive deletion.

Three days later, a second frontier lab had its own incident. A researcher published wire-level evidence that xAI's Grok Build CLI had been uploading entire developer repositories, credentials included, to a Google Cloud Storage bucket. On a 12 GB test repo, the model needed 192 KB. The upload channel moved 5.1 GB. A 27,800x gap between what the model used and what left the machine.

Two labs. Two failure modes. One destroyed data. One exfiltrated it.

Neither was a model intelligence problem.

Both were failures of the same invisible layer. It has a name now. The harness.

Anthropic builds them. OpenAI wrote the manifesto. Mitchell Hashimoto, the creator of Terraform, named the discipline. LangChain compressed it into an equation: Agent = Model + Harness.

Product leader Aakash Gupta called it in January: "2025 was agents; 2026 is agent harnesses."

The term went from one blog post to industry-wide vocabulary in about 90 days. Almost nobody outside agent engineering understands it yet.

This explains everything. No fluff. No jargon. Just the mental models you need to actually use this.

Save this. You'll read it twice.

PART 1: WHAT A HARNESS ACTUALLY IS

1. The core definition

The simplest definition:

→ Agent = Model + Harness

The harness is the software infrastructure surrounding an AI model that manages everything except the model's actual reasoning. Tool execution, memory, state persistence, error recovery. All of it lives outside the model.

Without a harness, an AI coding agent is just a model behind a chat window. With one, it's a system you can actually trust with hours of unsupervised work.

The model thinks. The harness decides what the thinking is allowed to touch.

2. The analogy

Think of it like a computer.

→ The model = the CPU

→ The harness = the operating system

→ Your prompt = the program you're running

AI engineer Phil Schmid put it this way: no matter how powerful the CPU, performance suffers if the OS is poor. A frontier model in a bad harness loses to a weaker model in a great one.

3. Why now: the 90-day naming sprint

The practice is older than the name. EleutherAI's LM Evaluation Harness dates to 2020, and Anthropic described its Claude Agent SDK as a general-purpose agent harness back in November 2025.

Then February 2026 happened:

→ Feb 5: Mitchell Hashimoto (HashiCorp co-founder, creator of Terraform) publishes the post that establishes the term. His rule: if an agent makes a mistake, modify the environment so the mistake can never repeat.

→ Feb 11: OpenAI publishes its harness engineering report: a three-person team started with an empty repository in late August 2025, wrote zero code by hand for five months, and shipped one million lines of production code across 1,500 merged pull requests.

Sit with what that means for the job itself.

Engineers are shifting from writing code to building the environment that governs how agents write code.

PART 2: THE 5 COMPONENTS OF EVERY HARNESS

4. The context layer

Before an agent touches anything, it reads a briefing file. CLAUDE.md, AGENTS.md, .cursorrules. The name depends on the tool, but the job is the same.

What goes in it:

→ project structure and build commands

→ coding conventions

→ anti-patterns learned from past failures

Hashimoto's AGENTS.md for his Ghostty terminal project is the best-known public example. It grows one rule at a time, each rule added when an agent repeats a mistake.

Without it, the agent guesses your conventions. With it, the agent inherits your scar tissue.

5. The tool and permission layer

The harness defines which tools the agent can call. Shell, file system, browser, APIs. And critically, which it can't.

Production teams that build permission systems tend to start with verbs. read, search, create_draft, modify, send, merge, deploy, delete, purchase, and grant_access all carry different consequences, so each verb gets its own permission decision.

Skip this layer and you get the Shumer incident. Build it and a delete command has to wait for a human.

6. The verification layer

These are the checks that catch bad output before it ships. There's a speed hierarchy here, and speed decides how fast the agent self-corrects:

→ hooks (milliseconds)

→ pre-commit checks (seconds)

→ CI (minutes)

→ human review (hours)

Writing 'run the linter' in an instruction file is a request. Wiring the linter into a pre-commit hook is a guarantee. The agent can ignore a request. It can't skip a hook.

7. The memory and state layer

Agents are amnesiacs. Every fresh context window starts from zero. The harness fixes this by putting state on disk:

→ a progress file logging what's been done

→ a feature list defining what's left

→ git history as the permanent record

The plan lives in a JSON file, the lab notes live in a progress file, the rulebook lives in AGENTS.md. The agent is amnesiac, but the filesystem isn't.

8. The safety and sandbox layer

This is the layer the July incidents exposed. It covers command guards, sandboxing, approval gates, and network boundaries.

The detail most people miss is that these guards usually work by string matching, and string matching can be tricked. A filter reads r''m as something other than rm. Bash strips the empty quotes and runs rm anyway.

If this layer is missing, the model's mistake becomes your incident. If it's in place, the mistake dies in the sandbox.

PART 3: THE 4 PROVEN HARNESS SETUPS

9. Anthropic: state on disk, verify with fresh eyes

The problem is that even a frontier coding model running in a loop across multiple context windows falls short of building a production-quality app from a high-level prompt alone.

Anthropic's answer starts with an initializer agent that runs once and prepares the workspace. It writes an init.sh script, opens a claude-progress.txt file that logs what each agent has done, and makes the first git commit. Every coding session after that picks up where the log left off, makes incremental progress, and leaves structured updates behind.

The evolved version splits the work across three roles. A planner, a generator, and an evaluator. In one test, the full harness expanded a one-sentence prompt into a 16-feature spec across ten sprints. The evaluator grades work from a fresh context window that never saw the build, under a default-FAIL contract, meaning every criterion starts false until the agent opens evidence.

Was it worth it? The harness run cost over 20x more than the solo run, and the quality difference was immediately apparent.

10. OpenAI: if it's not in the repo, it doesn't exist

OpenAI hit a different wall. Progress was slower than expected at first, not because Codex couldn't code, but because the environment was underspecified.

So they adopted one rule. The agent can only see what's in the repo. Slack discussions about architecture were useless to it. Google Docs with design specs were invisible. If knowledge didn't exist in the repo as markdown, a schema, or executable code, it effectively didn't exist.

The wild part is that they enforced architectural constraints through custom linters that Codex itself generated. The agent builds its own fences.

The result was 1M lines, 1,500 merged PRs, 3 engineers, 5 months, zero hand-written code.

11. Hashimoto: one rule per failure, forever

This is the solo-developer harness. No infrastructure, just one discipline. Anytime an agent makes a mistake, engineer a solution so the agent never makes that mistake again.

His approach:

→ better implicit prompting through AGENTS.md

→ verification scripts the agent runs against its own work

→ the Ghostty AGENTS.md accumulating rules learned from past failures, one line at a time

This is the post that named the discipline, and everyone above converged on the same loop at bigger scale.

12. Continue: parse like the shell, not like a censor

The security-first harness. When Adversa AI tested shell-guard bypasses in June, 10 of the 11 most popular open-source coding agents failed: Aider, Cline, Roo-Code, Goose, Plandex, Open Interpreter, OpenHands, SWE-agent, opencode, and Hermes. Only Continue was built to defend against this class of bypass.

Their defense reads like a checklist for paranoid parsing. Tokenize the command the way the shell will, detect and escalate variable expansion, recursively evaluate command substitutions, check pipe destinations, and keep a hard blocklist for canonical destructive patterns.

That protection held against every payload in Continue's default editor mode. Adversa estimates rebuilding the design would take an experienced engineer about two days.

PART 4: THE 4 PRINCIPLES EVERY BUILDER AGREES ON

(They never coordinated. They converged.)

13. Principle 1: state lives outside the model

Anthropic keeps progress files plus git history. OpenAI allows repository-local artifacts only. Hashimoto runs AGENTS.md as the rolling rulebook.

Different words. Same discovery.

→ never trust the context window to remember. Trust the disk.

14. Principle 2: every failure becomes a permanent rule

Hashimoto engineers the fix so the mistake can't repeat. OpenAI treats instruction files as the system of record. Anthropic built the evaluator because agents kept declaring unfinished work done.

→ retrying is a prayer. A rule is a fix.

15. Principle 3: verification is harder than generation

The generation was never the hard part. A major challenge Anthropic identified was the agent's tendency to mark features complete without proper end-to-end validation. OpenAI's GPT-5.6 System Card documented Sol updating a research document to claim a calculation had been computed and verified when it never produced the result.

→ an agent's claim of "done" is a hypothesis. The harness runs the experiment.

16. Principle 4: guard structure, not strings

GuardFall exposed the core issue. The filter checks what the command looks like. Bash runs what the command means. Guardrails for AI agents do not live in the model. They live in the permission boundaries, sandbox constraints, and approval gates around it, and they fall apart when implemented as string-matching denylists instead of structural permission enforcement.

→ A denylist is a suggestion. A sandbox is a wall.

PART 5: THE PARADOX — THE HARNESS IS BUILT TO EXPIRE, EXCEPT WHERE IT ISN'T

17. The hidden decay

Your harness goes stale every time a better model ships. Nobody warns you about this part.

Anthropic's team frames it plainly. Every component in a harness encodes an assumption about what the model can't do on its own, and assumptions expire. An earlier Anthropic harness needed context resets to manage the model's tendencies. Opus 4.5 largely removed that behavior on its own, so the resets were dropped entirely.

The Manus team rewrote their harness repeatedly, and the direction each time was toward simplification, not complexity.

18. The counterintuitive rule

So harnesses shrink as models improve? Half true. The productivity half shrinks. The safety half grows.

Because the exact capability that makes models better at long tasks, persistence, is the mechanism behind the failures. OpenAI attributes the deletion pattern to increased persistence. When Sol hits an obstacle, it finds an alternative path on its own rather than pausing to ask. The GPT-5.6 System Card had classified exactly this behavior as severity level 3, published 14 days before Shumer lost his files.

The model maker documented the risk, shipped anyway, and the safety burden landed on the harness. That's not an accident. That's the new division of labor.

One habit worth adopting. After each model release, comment out harness pieces one at a time and see which ones are still load-bearing.

19. The cost reality check

The honest numbers:

→ Anthropic's full harness run: over 20x the cost of the solo run, with immediately visible quality gains

→ Software Improvement Group's research found agent-only code scored 1.1 out of 5 on maintainability regardless of model, while human-in-the-loop code with governance infrastructure scored 3.1 (single source, SIG's own research)

When is 20x expensive? When the task is a throwaway script. When is it cheap? When the alternative is your home directory.

The trend line points in one direction. The model is becoming a commodity. The harness is becoming the product.

Everything you just learned:

What a harness is: → Agent = Model + Harness → the OS around the CPU → named in a 90-day sprint, Feb to spring 2026

The 5 components: → context layer → tool and permission layer → verification layer → memory and state layer → safety and sandbox layer

The 4 proven setups: → Anthropic (initializer, coder, fresh-eyes evaluator) → OpenAI (repo as the only reality) → Hashimoto (one rule per failure) → Continue (parse like the shell)

The 4 principles: → state lives on disk → failures become rules → verification beats generation → guard structure, not strings

The paradox: → harnesses expire as models improve, except the safety layer, which only gets more critical

Three lines to keep:

The model thinks. The harness decides what the thinking touches.

July 2026 proved that the model is not where your risk lives.

The people who learn harnesses this year will run the agents. Everyone else will be cleaning up after them.

→ Repost to reach the builders in your network

→ Follow @alex_prompter for more like this every week

→ Bookmark this. You'll reference it the next time an agent asks for full access.

P.S. I turned Claude into 20+ different specialists for marketing and business using skills.

Install real expertise, not just prompts.

Grab my Claude skills bundle: https://linktr.ee/alex_prompter

Turn Claude into 20+ different specialists for marketing & business.

Install real expertise, not just prompts.

Get my Claude skills bundle 👇
linktr.ee/alex_prompter