James E. Murphy

Restart Economics for AI Agents

Cutoffs, Retries, and Cost per Resolved Task
James E. Murphy · ORCID 0009-0001-2217-4306 · Working paper for discussion · v0.1 · 28 September 2026

Repeated runs of the same AI coding agent on the same task differ widely in what they cost and whether they succeed, so an operator runs a policy rather than a single attempt: cap each attempt, retry, and hand the task to an engineer when the attempts are spent. Most work on stopping agents early asks how much compute a stopping rule saves for a given loss in success. This paper asks the operator’s question instead: once every lost success has to be bought from a fallback that costs money, does stopping still save anything? It replays four logged OpenHands runs on each of the 500 SWE-bench Verified tasks for seven model configurations, 13,681 attempts in all, and scores every policy on expected cost per task, which charges agent spend, verification of what the agent submits, and a task-specific outside option priced from the benchmark’s annotated engineer time at a swept wage. Because the outside option resolves every task, no policy can look cheap by abandoning hard ones. Policies are chosen on training folds and scored on held-out folds, under an analysis plan frozen and registered before any result existed.

Three things follow. When verification is automated and the fallback is an engineer, retrying is the lever: at $100 to $300 an hour it cuts expected cost by 13 to 21 percent for six configurations and 4 to 8 percent for the seventh, and raises the share of tasks the agent resolves itself from 58 to 69 percent to 69 to 78 percent. Choosing a cap on attempt length from these data adds nothing at any human wage, because attempts that have run long still succeed a third to a half of the time, and each success cut off costs tens of dollars against the few dollars of tokens a cutoff saves; a cap pays only while the fallback is worth fewer than two to four attempts, which is a few dollars an hour here, and in an unregistered check even the best cap chosen with hindsight saves at most three cents per task at $100 an hour. When each submitted patch must be reviewed at a large share of the fallback’s cost, the economics reverse: retrying stops paying, the cap saves in point estimate at every wage, distinctly from zero below about $10 to $50 an hour, by keeping long attempts out of review, and, in an unregistered comparison, the best capped policy costs about what sending every task straight to the engineer costs, so the question becomes whether to run the agent at all. The registered low-complexity state rule adds little beyond a fixed cap, and switching configurations after a failure saves about 9 percent in point estimate, with an interval that includes zero.

Download the working paper · PDF · 49 pages · v0.1 Code, results, and manuscript sources · GitHub · the v0.1 release this draft was built from Preregistration · OSF · the analysis plan, registered before any policy was evaluated Also on SSRN · abstract and mirror of this draft

Status

This is a living draft. Versions are numbered and dated on the title line. Comments and criticism are welcome: contact@jamesemurphy.com.

Cite as

James E. Murphy, “Restart Economics for AI Agents: Cutoffs, Retries, and Cost per Resolved Task,” working paper v0.1, 28 September 2026. Available at https://jamesemurphy.com/ai-economics/restart-economics/ and https://ssrn.com/abstract=7538000, doi:10.2139/ssrn.7538000.