Decoded
Feed

AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces

arXiv:2608.23041AICLLG24 aug 20261 min readHN ×4PDF

An AI coding agent debugs another agent's failures and patches its wrapper code, raising success rates.

9.0

pp

Pass@1 gain on GAIA2

Similar gains on coding and terminal benchmarks, without manual harness tuning

How it works

  1. Harnesses are hand-tuned

    Agents fail on long tasks. The prompts, tools and control code around them are tuned slowly by hand.

  2. Debug traces, patch code

    An agent reads failed runs and the harness source. It writes typed patches and keeps those that help unseen tasks.

  3. Better agents, fewer runs

    Test success rose 9 to 10 points on three benchmarks. It beat GEPA and Meta-Harness using fewer task executions.

failed tracespatchif improvedlessonsnext iterationRunmini-batchDiagnoseand patchVerifyon batchDev-setcheckReflectinto EvoDAGEvolve nextharnessfailed tracespatchif improvedlessonsnext iterationRunmini-batchDiagnoseand patchVerifyon batchDev-setcheckReflectinto EvoDAGEvolve nextharness
From the paper