Google's AI Rewrites Its Own REVOLUTIONARY Harness (Harness²)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Google’s AI Rewrites Its Own REVOLUTIONARY Harness (Harness²) · Channel: Prompt Engineer 48 · 06:40 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Open source and software ecosystems, AI safety

Summary

Harness Squared is a Google Research approach in which an AI agent improves the harness around a frozen model, rather than retraining the model itself. Its inner loop proposes and tests harness edits by comparing runs, then composes a final version using evidence; an outer loop stores useful lessons in a library for future tasks. In the described benchmarks, one improvement step raised scores across all tested setups, including a case where a weaker model with an improved harness outperformed a stronger model with its original harness. The video also notes limitations, including cost and unreliable signals without ground truth, and describes the project as open source under Apache 2.0.

Key points

The agent is described as a model plus a harness; the model stays frozen while the harness can change.
Harness Squared addresses tasks without answer keys or verifiers by comparing runs for evidence of whether an edit fixes a targeted flaw.
The harness is divided into eight editable files covering prompts, guardrails, memory, skills, tool guidance, scripts, sub-agents, and a plugin.
The inner loop uses three steps: propose candidate harnesses, probe them on the same task, and compose a final harness.
Parallel recursion explores multiple candidates, while sequential recursion carries refinements and evidence forward.
An outer loop records supported lessons in a library, which can inform later tasks without being automatically inherited.
In the jobbench example, a harness edit improved the score from 43.1 to 77.8 by adding process checks and domain-specific reconciliation guidance.
The reported benchmarks showed gains across all tested setups, and an improved GLM 5 score exceeded Gemini’s untouched baseline in one comparison.
Ablations found lower performance without the library, scope editors, or composition; the video also reports that more runs add cost and that signals can be fooled by randomness or shared model bias.
The project is presented as open source under Apache 2.0, with a local demo that does not require model credentials.

Chapters

Start	Chapter	Summary
00:00	The harness as a target for improvement	The video introduces Harness Squared and explains why improving the harness can change performance without changing the model.
01:33	The inner loop: propose, probe, compose	The harness’s eight files are introduced, followed by the evidence-based process for testing and composing edits.
03:04	The outer loop and a jobbench example	The system stores supported lessons in a library and demonstrates a task-specific improvement from 43.1 to 77.8.
04:37	Benchmark results and ablations	Reported benchmark gains, harness scaling results, and ablations illustrate both the benefits and the importance of the system’s components.
06:09	Limitations and how to try it	The video discusses cost and evaluation limitations, outlines possible next steps, and points viewers to the open-source project.

Quotes

The only thing that changed was the harness around it. — 00:00

Did this edit fix the targeted flaw, change nothing, or break something? — 02:34

43.1 became 77.8. — 04:06

More compute alone isn’t the answer, smarter use of it is. — 05:07

The harness learned to improve itself and then learned to improve how it improves. — 06:40

Concepts and entities

Harness Squared, Harness, recursão em nível de tarefa, recursão em nível de harness, Gemini 3.5 Flash, GLM 5, Google Research, OpenCode, Codex, WorkBuddyBench, JobBench, Apache License 2.0

Full transcript (original language)

00:00 Imagine an AI agent that fails a job, then without a human touching it, rewrites its own instruction manual and tries again. On one real task, the first attempt scored 43.1, the second attempt scored 77.8. Same model, same weights, nothing retrained. The only thing that changed was the harness around it. This is Harness Squared Recursive Agent Harnessing from Google Research. To understand it, start with a simple equation. An agent is a model plus a harness.

00:31 The model is the brain, frozen, trained, finished. The harness is everything wrapped around it, the prompts, the tools, the memory, the skills, and the rules. And here’s the uncomfortable truth. The same model can perform wildly differently depending only on the harness it wears. Today, humans build harnesses by hand, tweak a prompt, run it, read the logs, tweak again. It’s slow and it doesn’t scale to new kinds of work. So, researchers ask a bold question, can an agent improve its own harness? That idea is called recursive

01:02 self-improvement. But in the open world, there’s a catch. New tasks arrive with no answer key, no unit test, no verifier to say whether you got it right. How do you improve something when you can’t measure success? That is the first problem Harness Squared solves. The second problem is memory. Fixing a harness for one task usually teaches the system nothing about the next one. The paper’s answer is two loops, one inside the other, task-level recursion and harnessing-level recursion. Before the loops, meet the thing being edited.

01:33 Harness Squared splits a harness into eight files that the model can read. A system prompt, guardrails, memory, and skills. These four shape what enters the context. Tool guidance and scripts define how the model touches the world. Sub-agents handle delegation and a plugin hooks every tool call outside the model’s control. Eight plain files. Anything the system learns has to be written into one of them. Now, the inner loop task level recursion works in three moves: propose, probe, compose.

02:03 First, the agent simply does the task once with the original harness. That’s the reference run. It shows how the harness behaves, not whether the answer is right. Propose, an editor reads that run and drafts two new harnesses, one invariant aimed at general failures like planning and tool use, one domain specific aimed at the knowledge of this exact field. Probe, both candidates then run the same task under identical conditions. Now, there are three runs to compare side by side. That comparison is the trick. Nobody needs to know the right answer. The

02:34 system only asks, “Did this edit fix the targeted flaw, change nothing, or break something?” Compose, a final editor reads every run and every diff. It can merge edits, resolve conflicts, adopt one candidate, or keep the original. Each decision must point to evidence. The composed harness runs the task one last time. That run is the deliverable you actually get. Want more? You spend a bigger budget called K, and there are two ways to spend it. Parallel recursion probes K general and K domain candidates

03:04 at once, then writes K compositions from the shared evidence. It explores wider. Sequential recursion chains the steps, carrying each refined harness and all its evidence into the next one. It refines deeper. When K is above one, an anonymous lector compares the finished outputs without knowing which harness made which and picks the best. Now, the outer loop, after every task, a reflector reviews the whole improvement record, what failed, what was tried, what the probes revealed. The winning harness is filed in a library beside an experience note,

03:35 the situation, the lesson, and the evidence behind it. If the evidence is thin, nothing is written. Next task, the agent starts again from the plain base harness, but the improver now has a library of tried ideas to borrow from, borrow, not inherit. Let’s see a case study, a job bench task, Gemini 3.5 flash running in open code, the baseline scored 43.1. The trajectory showed process failures, not missing knowledge. The agent never reconciled numbers across source files, and it never reopened its own outputs to

04:06 check them. The invariant edit added a guardrail, inventory every input, trace each figure to its source, then run a script to reopen and verify every file you produce. The domain edit added a reconciliation skill for schedule, venue, labor, and budget files, plus tool guidance for tricky number formats. The composer kept all four edits. The rerun cross-checked the figures and restored the missing sections. 43.1 became 77.8. Now the scoreboard, three professional benchmarks, work buddy bench for office,

04:37 web, and code work, jobbench for end-to-end jobs, and LAB Harvey’s legal benchmark. Two frozen models, Gemini 3.5 flash and GLM 5, two starting harnesses, open code and codex four setups in all. One single improvement step lifted every setup on every benchmark. The biggest jump, GLM 5 on work buddy, up 11.5 points. On jobbench, gains reach 4.5 points. On legal work, up to 11.8 points of partial pass rate. Notice something, a weaker model with a better harness

05:07 beat a stronger model with its original one. GLM 5 once improved scored 75.7. Gemini’s untouched baseline was 72.4. The baselines tell a story, too. Reasoning bank fell as much as 6.6 points below base on coding. Harness scaling lost to the base in seven of 12 cases. Turn up the budget and the gains keep coming on. Code, Gemini climbs from 76.5 to 80.8 on office work, from 79.5 to 83.4.

05:38 Even k equals 2 beats every baseline at k equals 5 on code and office. More compute alone isn’t the answer, smarter use of it is. Remove any piece and it hurts. No library minus 3.1 no scope editors minus 2.9 no composition minus 1.6. And the library really does transfer experience from job bench improved results on work buddy with no extra probing. Does it need a stronger editor? No. Using the same model as both worker and editor, all five models tested improved on every benchmark.

06:09 Now the honest limits more runs mean more cost and with no ground truth the signals can be fooled by randomness and shared model bias. Next steps are a library that prunes itself and a system that decides how much to explore based on how hard the task is. Want to try it? It’s open source under Apache 2.0 install with UV and run the local demo which needs no model credentials. Then set up a benchmark and run a single task to start. Model runs cost money so keep it small. Harness squared, the model never changed. The harness learned

06:40 to improve itself and then learned to improve how it improves.