Harness² Explained: Self-Improving Agents Without Training

Author: Prompt Engineering · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Harness² Explained: Self-Improving Agents Without Training · Channel: Prompt Engineering · 11:25 · Watch on YouTube

Topics: AI agents, Task and workflow automation, AI evaluation and benchmarks, Prompt and context engineering

Summary

Harness² is a system that lets agents adapt their tools, skills, sub-agents, and control loops to the task at hand instead of relying on a generic pre-built harness. It uses a propose, probe, and compose process to test changes and keep only those that address concrete failures, then saves reusable improvements for future tasks. The video reports that evolving a harness can improve performance and even help a smaller model outperform a stronger one using its default harness. The approach has costs and risks, including extra compute, faulty self-verification, and agents exploiting access to evaluation materials.

Key points

Generic pre-built harnesses are not equally effective across tasks; task-specific harnesses can perform better.
Adding every learned rule to the system prompt can consume attention and token budget, especially on long tasks.
Harness² can create on-demand skills, custom tools, sub-agents, and code hooks that modify the agent’s control loop.
The propose, probe, and compose process compares baseline and modified behavior, retaining edits only when they fix a concrete failure.
The system separates task-specific harness changes from reusable lessons stored in an experience library.
The video reports accuracy gains of 35 to 50 points on some complex tasks and a 17-point coding-score increase when an evolved harness was transferred to a smaller model.
Testing candidate harnesses can require 4 to 20 agent runs per task, making the approach costly for quick prompts.
Self-written verification can reinforce incorrect assumptions, so a separate verifier is recommended.
Agents with internet access may seek out public benchmark grading materials, making evaluation guardrails important.

Chapters

Start	Chapter	Summary
00:00	Why agents need adaptable harnesses	The video introduces the limits of generic harnesses and presents Harness² as a way for agents to adapt their setup to each task.
04:11	Testing harness changes	Harness² uses propose, probe, and compose to compare edits against a baseline and keep only changes that fix observed failures.
06:46	Learning across tasks	Two levels of adaptation separate task-specific changes from reusable tools and skills saved for future tasks.
09:23	Limitations and practical lessons	The video covers compute costs, unreliable self-checking, benchmark-access risks, and recommendations for building safer, more effective harnesses.

Quotes

A smaller, weaker model running inside a self-evolving harness flat-out beat a much stronger frontier model running on its default pre-built harness. — 01:03

harness two uses a three-step loop called propose, probe, and compose. — 04:43

And only proven edits gets merged to the final harness. — 06:16

How you engineer the harness now matters just as much as which model you pick. — 10:54

Concepts and entities

Harness², Harness, control loop, skill, subagent, Claude Code, Codex, DeepSeek, GLM 5, Gemini 3.5 Flash, Google Research, Northwestern University, Hugging Face

Full transcript (original language)

00:00 Every agent that we use today comes with a pre-built harness. The problem is that a pre-built harness has to be one-size-fits-all. And in most cases, a generic harness is simply not optimized for the specific tasks that you are trying to solve. In fact, when researchers tested the exact same AI model inside four popular harnesses, including Claude, Code, and Codex, no single harness won everywhere. Claude Code was better at coding and spreadsheets, while Codex was better at

00:32 web tasks. And research keeps proving that a purpose-built harness tailored to your task always beats a generic pre-built one. So, what if the moment you hand an agent a task, the agent could build and evolve its own harness on the fly? What if it could rewrite its own instructions, create its own tools, and even edit its own control loop while it works? This is the exact idea behind a new

01:03 project from Google Research and Northwestern University called Harness Squared or Harness 2. And their biggest finding is wild. A smaller, weaker model running inside a self-evolving harness flat-out beat a much stronger frontier model running on its default pre-built harness. So, in this video, let’s break it down what the core ideas of this Harness 2 are, what happens when an agent can edit its own loop, and where this approach breaks

01:33 in the real world. When most people try to build a self-evolving agent, the obvious move is to let the model keep adding new rules to its system prompt whenever it makes a mistake. The researchers tested that, and it actually makes the agent worse. Why? Because your system prompt is always on. So, every word that you add to the prompt gets sent at every single step of every task, which ends up eating up the model’s attention and token budget.

02:04 On long tasks, the number one reason agent fail is that they spend too much time reading instructions and reasoning uh that they run out of budget before even trying to finish the job. So, instead of loading the system prompt, this new harness design lets the agent edits uh four much smaller layers on the fly. First is on-demand skills. When the agent learns a specialized workflow, like how to uh redline a legal contract or clean a spreadsheet, it writes that playbook as

02:36 a separate skill that costs zero tokens until a task actually triggers it. Second is custom tools. LLMs are bad at doing repetitive math or checking hundreds of cells in their head. So, when a task needs data verification, the agent writes small Python script to check the data deterministically. And this approach has shown to work really well for AI agents. Third is sub-agents. If a task requires

03:08 reading 20 huge documents just to pull out one small table, the agent spawns a separate sub-agent with its own clean memory, so all the raw text never clutters the main agent’s context. This way, they’re able to avoid context rot and are able to remove distractors. And fourth, editing its own control loop. This is the biggest shift. A rule in the prompt is just a polite suggestion that uh distracted model can ignore.

03:41 By injecting uh code hooks directly into its own loop, the agent can enforce hard rules on itself. On one task where the agent kept reasoning until uh time ran out, it wrote a watchdog hook into its own loop that counted its steps and forced it to stop researching and write the final file. On another task, it blocked itself from finishing until a script responded in Excel file and verified that none of the

04:11 formulas were blank. Now, we have seen this pattern elsewhere as well. You can completely modify the DeepSeek harness agentic loop or the Claude mods now enables you to modify the control loop of Claude code. Okay. Uh so, this actually raises the hardest question in the paper because most of the experiments are done under Evals. Uh in a lab, you can optimize a harness because you have a training set with an answer key. In the real world, when you

04:43 give an agent a brand new task, there is no answer key. And when the researchers let the AI write its own harness blindly without testing those edits, performance dropped by 10%. To solve this without an answer key, harness two uses a three-step loop called propose, probe, and compose. Now, here’s how it works. First, in propose, the agent runs the task once with its default harness. So, even without a grade, uh the log shows where the agent struggled.

05:15 Then, two specialized editors inspects that log. One editor only fixes workflow mechanics like tool errors or uh loop hookups, while the second editor acts like a domain expert, uh like an accountant or a lawyer, uh which keeps adding domain rules. The second step is uh probe. So, in the probe step, the system runs uh those upgraded harnesses uh on the exact same task.

05:45 So, now we have the baseline with a default harness and we have results with the upgraded harness. And the third step is compose. In compose, the system compares the before and after behavior side by side. So, instead of guessing which answer sounds nicer, it checks whether an edit fixed a concrete failure from the first run, like catching 12 blank spreadsheet cells that the first run missed.

06:16 And only proven edits gets merged to the final harness. If the edit didn’t help, the system simply keeps the original harness. So, this is a very nice way of adding the capabilities that are actually helping compute to the baseline, rather than going with attempts that seems to be helping. Okay. Now, the thing is that this fixes the current task that the harness for the agent is working on. But, what happens on the next task?

06:46 Let’s say uh we have task one and task two. If you carry task one’s heavily customized harness straight into task two, it potentially is going to break because half of those rules were specific to task one. That is why the project is called harness squared. It uses two levels of recursions. So, level one evolves the harness for the task right in front of you. Level two evolves how the system builds harnesses for future tasks. After each task, the system reflects

07:17 on which edits actually helped, stripes out the task-specific noise, and saves the reusable tools, skills, and lessons into an experience library. So, when the next uh task arrives, the agent starts fresh with a clean base harness and pulls only the proven tools and skills it needs from that library. So, over time, it’s going to build a library of tools and skills that can potentially be used for subsequent tasks.

07:48 Okay, now let’s talk about what actually works, what breaks, and what are the real takeaways from this paper. First, let’s talk about what actually works. So, across coding, web, office, and legal work, evolving the harness on the fly beats static harnesses, including untouched cloud code codex in every domain. On complex tasks like legal redlining or budget reconciliation, evolving the harness boosted task accuracy by 35 to 50 points in a

08:19 single step, which is pretty significant. Most importantly, it proves the harness or model effect. Not only did a smaller model GLM 5 uh with an evolved harness beats a stronger model, uh in this case, Gemini 3.5 flash, on its default harness, but when the researchers took the harness evolved for Gemini and handed it unchanged to a smaller model, the smaller model code score jumped by 17 points instantly.

08:50 Now, this is a very important point to keep in mind. This means you can customize a harness with a larger, more stronger model, and then in production, you can potentially use that evolved harness with a relatively smaller model. Now, second, and we need to think about where this um whole concept breaks. The first one is the compute multiplier. Testing candidate harnesses on the fly takes um anywhere from 4 to 20 full agent runs for a single task. Uh so, you

09:23 would never want to run this live on a quick prompt. Instead, you use it on your own uh first 10 or 20 tasks to build up your domain library, and then reuse the upgraded harness cheaply on our task in production. Second is the echo chamber trap. Without a real answer key, when the agent writes its own verification scripts, uh it can bake its own bad assumptions into the checker. In one test, the agent’s self-written

09:54 checker reported pass three times while failing in the real test. So, this is something to keep in mind. And in most of the cases, you want to have a separate uh verifier or checker uh that is different from the model that you are uh evolving the harness for. And third is the funniest moment in the repo. During a validation run with uh open internet access, the agent realized it was being tested on public benchmark,

10:24 went on to hugging face, and downloaded the benchmark’s grading rubric all by itself. This is the behavior that we have started seeing more and more. These agents have a lot more autonomy and they’re going to use any resource possible or available to them in order to accomplish a task. So, you want to make sure uh if you’re evaluating your uh evolving harness or harnesses in general, uh you need to have really good guardrails around what type of access these are

10:54 going to have. But, the big takeaway uh from harness two is simple. How you engineer the harness now matters just as much as which model you pick. If you’re building agents, keep your always-on prompts tiny, move domain playbooks into on-demand skills, replace repetitive LLM reasonings with small script, and enforce your most important rules inside the control loop itself. I’ll put uh the paper and the GitHub

11:25 repo in the description below. And if you’re finding these deep dives helpful, please consider subscribing to the channel. That helps a lot. Anyways, I hope you found this video useful. Thanks for watching and as always see you in the next one.