cat >> /home/claude/harness-engineering.md << 'HARNESS2EOF'
---
---
PART II — THE AGENT HARNESS
Now we build the thing. This part walks the anatomy of a production agent harness, component by component, loop-first.
Chapter 3: The component model — what's actually in a harness
A production harness isn't one thing; it's a set of cooperating components. The 2026 practitioner consensus decomposes it into roughly these modules (names vary, roles don't):
- **Instruction manager** — holds the system prompt, role, standing rules, and repo-/domain-local instructions (the "constitution" of your agent).
- **Context builder** — assembles what goes into each model call: instructions + relevant history + retrieved data + tool results, within the token budget. This is where your context-engineering and RAG work plugs in.
- **Model adapter** — the provider-neutral shim that talks to whichever LLM you're using, so the rest of the harness doesn't care if it's Claude, Falcon, or Jais.
- **Tool registry** — the catalog of tools the model can call, their schemas and descriptions (this is where MCP servers plug in — your MCP knowledge is the tool layer of the harness).
- **Permission resolver** — decides whether a given tool call is allowed, for this user, in this context. The enforcement point for least privilege.
- **Budget tracker** — counts tokens, dollars, wall-clock time, and steps, and cuts things off before they run away.
- **Memory / state store** — persists what must survive across calls and sessions (often externalized to a filesystem or database — Chapter 6).
- **Observability layer** — logs every step, tool call, and decision as a **trajectory** you can replay and audit (this is the bridge to the eval harness — Part III).
The mental model: **the loop (next chapter) is the engine; these components are the systems around the engine that make it drivable and safe.** A toy agent is a `while` loop and an API call. A production harness is all of the above, and the gap between them *is* the 88% failure rate.
For your legal engine, you already have pieces of this: the context builder (your RAG pipeline), the tool registry (if you expose it over MCP), and the permission resolver (Supabase RLS). Harness engineering is assembling them into one coherent, observable runtime with a loop at its center.
---
Chapter 4: The agentic loop — the beating heart
The loop is what produces autonomy. Strip everything else away and an agent is this cycle:
1. **Think** — the model, given the goal and current context, decides the next action.
2. **Act** — the harness executes that action (calls a tool, runs code, queries a DB).
3. **Observe** — the result is captured and fed back into context.
4. **Repeat** — until a stop condition is met (goal reached, budget exhausted, human needed).
A <cite index="42-1">systematic analysis of 70 open-source LLM agent projects showed 60% adopt this Agent Loop pattern</cite> — it's the dominant shape of agentic AI. A concrete minimal example from a real research harness: <cite index="51-1">the LLM generates a response, the harness extracts the first fenced code block, executes it in a bash shell, and returns stdout/stderr to the model as the next observation, capped at 100 action steps</cite>. That's the whole engine — everything else is refinement.
**The critical design decision nobody defends but evernyone makes: who's in charge of the loop?** There's a genuine spectrum here, and it's the heart of loop engineering:
- **LLM-as-orchestrator (maximum autonomy).** The model runs the loop, choosing every tool and deciding when it's done. Maximally flexible, minimally predictable. Most frameworks default here, and it's <cite index="39-1">a design decision they seldom defend — the LLM is put in charge of execution</cite>.
- **Hardcoded pipeline (maximum control).** Your code fixes the sequence of steps; the model just fills in each step. Predictable, but rigid — it can't handle tasks whose shape isn't known in advance.
- **The middle path (the mature choice).** Your *program* fixes the discipline — the loop's structure, its budgets, its stop conditions, its verification gates — while the *model's judgment* shapes the decisions within that structure. As one framing puts it: <cite index="39-1">an orchestrating function can ask "which stage next?" and branch on the answer, while an open-ended task runs as a bounded loop whose max_steps and stopping checks are the program's</cite>.
For a law-firm legal engine, you want to sit firmly toward the controlled end. "Explore this contract corpus for risks" can be a bounded, autonomous loop; "file this document" or "email the client" cannot — those need fixed pipelines with human approval. The loop's autonomy should be proportional to the reversibility of what it can do. This is the security guide's blast-radius principle expressed as loop design.
---
Chapter 5: Context engineering — managing the model's most precious resource
Recall from the fundamentals guide: the context window is finite, expensive, and the model's *entire* view of the world on any given call. In a multi-step agent, the loop *fills the window with its own history* — every tool result, every observation piles up. This creates the defining failure mode of agent harnesses, and it has a name:
**Context rot (also "context overflow").** As <cite index="41-1">interactions progress, massive tool outputs lead to context rot, which degrades the LLM's attention</cite>. The window fills with accumulated cruft, the model's attention smears across too much text, and quality silently degrades — the answer gets worse without any error firing. This is the agent-harness version of "models answer worse when the key fact is buried in noise," which you met in RAG.
The harness's defenses, which are the core craft of context engineering:
- **Compaction.** Periodically summarize the accumulated history into a compact form, discarding the raw verbose version. The loop continues with a tight summary instead of a bloated transcript.
- **Pruning.** Drop tool results and messages that are no longer relevant. Not everything from step 2 matters at step 20.
- **Sub-agent isolation.** Spin off a sub-task into a *separate* agent with its own fresh context, and return only its result to the main loop. The main context never sees the sub-task's mess. This is why multi-agent designs exist — not because "more agents = smarter," but because isolation contains context rot and blast radius.
- **Externalized memory.** Rather than keeping everything in the window, write state to a **virtualized filesystem** using standardized artifacts — <cite index="41-1">files like AGENTS.md or todo.md logs</cite>. The window holds a pointer ("see todo.md"); the filesystem holds the detail. The model reads state back in only when needed.
The mental model shift: **context is not a transcript to accumulate; it's a working set to actively curate.** A good harness is constantly deciding what the model needs to see *right now* and ruthlessly excluding the rest. For your legal engine as an agent, this means a multi-document analysis doesn't cram all 40 contracts into one window — it processes them with isolation and externalized notes, keeping each model call focused.
---
Chapter 6: Long-horizon work — surviving beyond one context window
The hardest thing in agent engineering, and the frontier of the field: tasks too long to fit in any single context window — analyze a hundred documents, refactor a large codebase, run for hours. As one practitioner puts it plainly: <cite index="44-1">autonomous long-horizon work is the holy grail and the hardest thing to get right. Today's models suffer from early stopping, poor decomposition of complex problems, and incoherence as work stretches across multiple context windows.</cite>
Two named failure modes to know:
- **Early stopping / "context anxiety."** The model, sensing its context filling up, tries to wrap up and exit prematurely — declaring done when it isn't. A behavioral quirk the harness must actively counter.
- **Incoherence across windows.** When work spans multiple context windows (because no single window holds it all), the agent loses the thread between them.
The signature harness pattern that addresses this — worth knowing by name because it's become a reference primitive — is the **Ralph Loop**: <cite index="44-1">a hook intercepts the model's attempt to exit and re-injects the original prompt into a fresh context window, forcing the agent to continue against a completion goal. Each iteration starts clean but reads state from the previous one through the filesystem.</cite> More fully: it <cite index="41-1">reinjects the original intent into a clean, compacted context window, ensuring long-horizon continuity, allowing agents to survive network crashes or session suspensions</cite>.
Why this is elegant and worth internalizing: it's <cite index="44-1">a surprisingly simple trick for turning a single-session agent into a multi-session one — the kind of primitive you'd never derive from "just use a smarter model."</cite> That last clause is the entire thesis of harness engineering in one line. The capability comes from the *scaffolding*, not the model.
The general principle beneath the Ralph Loop: **for long-horizon work, the filesystem is the memory and each context window is disposable.** State lives durably outside; each window is a fresh, focused work session that reads state in, does a chunk, writes state out, and is discarded. Coherence comes from the persistent external state, not from one heroic mega-context.
One more research-backed subtlety: **temporal awareness must be engineered into the harness, not assumed from the model.** A 2026 finding showed that <cite index="42-1">temporal awareness — handling deadlines and time constraints — appears orthogonal to reasoning capability; explicit temporal feedback in the agent loop significantly improves performance on deadline-constrained tasks</cite>. Translation: if your legal agent needs to respect a filing deadline, the *harness* must inject current time and time budgets into context — a smarter model won't infer them on its own.
---
Chapter 7: Permissions, budgets, and stop conditions — the safety rails
This is where the security guide becomes concrete harness code. Three rails, each non-negotiable for a production agent, especially one touching legal data.
**Permissions (the permission resolver in action).** Every tool call passes through a check: is *this* action allowed for *this* user in *this* context? This is least privilege (security guide's master control) implemented as a harness component. Crucially — and this echoes the MCP guide — enforcement must be real, not advisory: the harness decides <cite index="50-1">which actions are exposed, who may invoke them, and when execution terminates</cite>, and for your legal engine the deeper authorization (which matters/documents this user may touch) still lives in the database via RLS. The harness's permission resolver and the database's RLS are defense-in-depth layers, not substitutes.
**Budgets (the budget tracker in action).** An unbounded agent loop is a runaway cost and a runaway risk. Without a harness, <cite index="40-1">agents become black boxes that leak tokens, run uncontrolled loops, and execute dangerous commands</cite>. Every loop needs hard caps: max steps, max tokens, max dollars, max wall-clock time. When a budget is hit, the loop stops — full stop. Budgets are also a security control: prompt-injection attacks often try to <cite index="40-1">exhaust resources</cite>, and a budget cap turns "denial of wallet" into a bounded, survivable event.
**Stop conditions (loop termination logic).** The loop must know when to end: goal achieved (verified, not just claimed — Part III), budget exhausted, an error it can't recover from, or a point where a human must decide. "Human-in-the-loop for anything consequential" (from the security and MCP guides) is implemented here, as a stop condition that hands control back to a person before an irreversible action.
The unifying idea: **these rails are what convert an interesting demo into a system you can point at real legal work without losing sleep.** They're not features you add later; they're the load-bearing structure. An agent harness without budgets and permissions isn't a minimal harness — it's an incident waiting for a trigger.
HARNESS2EOF
echo "Harness Part 2 written"