11.5-Point Boost: Google's Harness2 Rewrites Agent Files at Test Time Without Model Updates

Google Research has open-sourced Harness2, a framework that allows developers to improve AI agent performance by modifying the surrounding code and files rather than retraining the underlying model. According to MarkTechPost, the release provides a method for agents to get better results without requiring a better or larger model. The tool operates by refining the agent's harness at test time, offering a plug-and-play solution for optimizing system behavior.

The risk

The primary operational shift involves moving optimization from model training to deployment-time harness editing. Harness2 modifies eight editable components, such as prompts, tools, memory, and orchestration logic, which are mounted as files. Because the model weights never change, the risk of model drift is eliminated, but the complexity of the harness logic increases. The framework runs on Linux with Python 3.12 or 3.13 and requires the uv package manager. Model calls are routed through Vertex AI, though specific GPU requirements were not disclosed by the researchers.

Who is exposed

Developers using frozen models for agentic tasks are the primary audience for this framework. The system is designed for open-world settings where tasks arrive unseen and no verifier confirms if an answer is correct. According to MarkTechPost, most self-improvement methods assume the presence of labels or test cases, but Harness2 is built to work without them. It targets scenarios where agents must handle novel problems without pre-defined verification steps, making it relevant for teams deploying agents in dynamic, unstructured environments.

Cost and compliance

The framework is released under the Apache-2.0 license, allowing for open use and modification. Performance gains vary significantly by model and benchmark. MarkTechPost reported that GLM-5 under OpenCode rose from 64.2 to 75.7 on WorkBuddy-Bench, an 11.5-point increase. In contrast, GLM-5 under Codex gained only 1.2 points on JobBench, moving from 23.6 to 24.8. The system supports both parallel and sequential refinement modes, with a budget parameter k scaling the search depth for invariant and domain-specific candidates.

What should you do?

Teams should evaluate whether their current agent failures stem from model limitations or harness configuration issues. Developers can test the framework on non-critical tasks to observe how recursive editing affects trajectory stability. Monitor the EXPERIENCE.md file generated by the reflector to understand which harness components are being modified and why. Ensure that the eight editable components align with your existing security and compliance policies before deployment.

The background in plain language

An agent harness includes all the code, prompts, and tools surrounding a language model, not the model itself. Just as a driver can improve by adjusting their car’s mirrors or route planning without changing the engine, an agent can improve by optimizing its harness. This approach allows teams to refine how an agent uses tools and remembers context without the high cost of retraining. It shifts the focus from model size to system design, enabling faster iteration and adaptation to new tasks without altering the core AI weights.

Background: AI agents

AI agents are software systems that perceive their environment, make decisions, and execute actions to achieve specific goals. Unlike standard models, they maintain internal state and interact with external tools, creating unique security risks around tool execution, memory retention, and autonomous decision loops that require specialized controls.

Read the full guide: AI Agents Explained: Answers to Security and Architecture Questions

What to do and how to stay safe: Google Research

• Review your agent’s current harness components to identify areas where prompts or tool definitions may be causing errors.
• Test the framework in a sandbox environment to observe how recursive editing impacts agent behavior on unseen tasks.
• Monitor the experience logs to ensure that automated harness changes do not introduce unintended security or compliance risks.
• Verify that your deployment pipeline supports dynamic file updates for the eight harness components without requiring full redeployment.

Step-by-step guide: Mobile Security Architecture: 8 Practices for Device Integrity

General security guidance from the Vector Tribune newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Does Harness2 require retraining the AI model?

No, the framework optimizes the agent’s harness components at test time without updating any model weights.

What are the system requirements for running Harness2?

The framework runs on Linux with Python 3.12 or 3.13 and the uv package manager, using Vertex AI for model calls.

How much performance improvement can be expected?

Results vary; GLM-5 saw an 11.5-point gain on WorkBuddy-Bench, while another setup gained only 1.2 points on JobBench.