Agent Harnessing: The Part of Building AI Agents Nobody Talks About

Agent = Model + Harness

Here's the simplest way I can put it: the model is the brain, the harness is the body and the rules it has to follow.

A raw model — GPT, Claude, Gemini, whatever — only does one thing. You give it text, it predicts the next chunk of text. That's it. It doesn't remember your last message on its own. It doesn't know which tools it's allowed to call. It doesn't know when to stop. All of that — the loop, the memory, the tool list, the permissions, the "okay you're done now" logic — that's the harness.

The simplest harness that ever existed is almost embarrassingly basic: back with GPT-2 in 2019, it was just "sample tokens until you hit a stop marker, then stop." That's a harness. It's just a very dumb one.

A modern production harness is the same idea, way more built out. It's usually made of six pieces:

• The loop — calls the model, reads what it wants to do, does it, feeds the result back, repeats
• Tools — the actual list of things the model is allowed to call (search the web, run code, query a database)
• Memory — what it remembers right now (context window), what it's scratch-noting mid-task, and what it pulls from long-term storage
• Permissions & sandbox — what it can do without asking you first, and where it's allowed to actually execute anything
• Sub-agents — handing a piece of the task to a fresh agent and getting back a short summary instead of drowning the main agent in detail
• Tracing — a log of every single step, so when something breaks, you can actually see why

None of that lives in the model. All of it lives in your code.

Building the dumbest possible harness, just to see it

I think the fastest way to actually get this is to build one from scratch — no framework, just raw API calls — so you can see exactly what a harness is doing when a framework normally hides it from you.

import json
from openai import OpenAI

client = OpenAI()

# the "tool" our agent is allowed to call
def get_weather(city: str) -> str:
    return f"The weather in {city} is 72°F and sunny."

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What's the weather in Jaipur?"}]

# this while loop IS the harness. this is the whole thing.
while True:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools,
    )
    msg = response.choices[0].message
    messages.append(msg)

    if not msg.tool_calls:
        print(msg.content)   # model is done, no more tools needed
        break

    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(**args)   # the harness actually runs the tool
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": result,
        })
    # loop again — the harness feeds the tool result back to the model

That while True loop is a harness. It's a genuinely tiny one — no memory beyond the message list, no sandbox, no sub-agents, no tracing — but it has the one non-negotiable piece every harness needs: it calls the model, checks what the model wants to do, does it, and feeds the result back. Everything else in this article is that same loop, just with more rooms built onto it.

Framework vs. harness — these are not the same word

This tripped me up at first, so let me save you the confusion.

A framework — LangGraph, Pydantic-AI, the OpenAI Agents SDK, Strands — gives you the building blocks to construct a harness. It's the Lego set.

A harness is the actual assembled, tuned thing that runs. Claude Code, Codex CLI, OpenCode — when you open one of those and start typing, you are not talking to a raw model. You're talking to a harness, one that somebody already built, tuned, and shipped, using building blocks like the ones above (or fully custom code).

This is exactly the same distinction I ran into comparing the Python agent SDKs a while back — every one of those is a framework, a set of primitives. None of them is a finished harness on its own. You still have to decide what goes in the loop, what the model's allowed to touch, and when it stops.

Why this matters more than picking a "better" model

Go back to that ARC-AGI number for a second. 30% to 95%, same model. If you're building agents and all your energy is going into "which model should I use," you're probably optimizing the smaller lever.

I've felt this directly building our own agent stack. The scrapers and lead-gen agents I've written about before don't get dramatically better because I swap the model — they get better when I fix what's actually in the harness: better memory of what's already been scraped so it doesn't repeat work, tighter permissions so it doesn't touch data it shouldn't, and proper sandboxing so a bad tool call can't take down something real. The model swap is the easy change. The harness is where the actual engineering is.

A quick checklist before you call something "done"

If you're building your own agent right now, here's roughly what I'd check the harness has, in the order I'd actually check it:

1. Does the loop know when to stop? Not just "ran out of turns" — an actual signal that the task is genuinely complete.
2. Is every tool call logged? If you can't see what happened after the fact, you can't fix it when it breaks — this is the same audit-trail problem I went deep on in our piece on agent compliance.
3. What's the blast radius if a tool call goes wrong? If the honest answer is "it could do real damage," it needs a sandbox, full stop.
4. Does memory actually persist where it needs to? Between turns is one thing; between sessions is a much harder, much more valuable thing to get right.
5. Could a smaller model run this same harness and still mostly work? If the answer's no, you're probably leaning on the model to cover for gaps in the harness — which is expensive and fragile.

Where this is actually headed

The part I find genuinely interesting: right now, almost every harness out there is hand-built. A developer sits down and decides the loop, the memory rules, the tool list, by hand.

There's real research right now — with names like Self-Harness, Agentic Harness Engineering, and JIT-Agent — on agents that improve their own harness automatically. The agent runs, logs what worked and what failed, and a separate process proposes a small change to its own scaffolding — tighten a permission, change how it retrieves memory, adjust the order it tries things in — then checks whether that change actually helped before keeping it. One early result worth remembering before you get too excited: researchers found that a model's ability to propose a good harness change and its ability to actually benefit from that change turned out to be two separate skills that don't reliably go together. A model can write a very convincing case for a change that doesn't help at all.

We're not at "your agent rewrites its own rules safely" yet. But the direction is clear: the harness used to be something only a human designed. That's stopping being universally true, and the gap between "I picked a good model" and "I built a good harness" is only going to get more important to actually understand, not less.