Fancy harnesses make coding agents more expensive but not better, study finds

A study by Liu and Han ran five models through Claude Code, OpenCode and mini-SWE-agent and found that swapping harnesses shifts SWE-bench Verified results about as much as simply rerunning the same setup, with differences of two to five solved tasks per model. The only measurable harness effect was a deficit for OpenCode, while Claude Code's roughly 16,600-token system prompt and tool descriptions made a single task cost up to three times more than the leanest harness. The authors conclude a harness buys fewer ways to lose a task and a bill, not pass rate, and recommend publishing a rerun of the same setup alongside every harness comparison.