Most of what's written about agents is about getting them to work. This is about watching them fail — slowly, unattended, in ways that don't show up in a five-minute demo — and what we changed in response. If the long-horizon design piece is the theory, this is the field notebook.
What we expected to break vs what did
Going in, we assumed the failures would be about intelligence: the model would hit a problem too hard and get it wrong. That happened far less than expected. The model was usually capable enough. What broke was everything around the model.
The real failure modes were mundane and systemic: an external call that hung forever with no human to notice; a lock the agent couldn't acquire, so it waited politely into the night; a run that technically succeeded but had wandered off the original goal somewhere around hour three. None of these get fixed by a smarter model. They get fixed by better scaffolding.
Drift is the silent killer
The most dangerous failure isn't a crash — a crash is loud and you deal with it. It's drift. A small wrong assumption at minute ten doesn't announce itself. It quietly shapes every decision after it, and the run looks perfectly healthy right up until the output is subtly, expensively useless.
Drift is hard precisely because nothing alarms. Our defence was to make the agent commit to an explicit plan up front and check its work against that plan as it goes. When new information contradicts the plan, the agent has to revise it deliberately — which turns silent drift into a visible decision you can inspect later.
Checkpoints turn disasters into hiccups
Early on, a failure at minute 200 meant starting over at minute zero. That's not just slow; it makes you timid — you don't dare run long jobs if any stumble throws away hours.
Durable checkpointing changed the economics. Every meaningful step writes state, so a failure resumes from minute 195, not zero. Once "the run failed" stopped being a catastrophe and became a hiccup, we could be ambitious about how long and how hard a single job could push. The willingness to attempt six-hour work came directly from making six-hour work cheap to recover.
Knowing when to stop and ask
The last lesson was about humility. An autonomous agent that never asks for help is not impressive — it's reckless. The hard skill is calibrating when to stop and surface a question versus push on.
We'd rather an agent pause and say "this looks ambiguous, here are the two readings" than confidently produce the wrong one for six hours. Long-horizon autonomy isn't the absence of human oversight; it's earning the right to run unattended by being honest about the edges of what it knows. That honesty, more than raw capability, is what makes it safe to walk away and let the work happen.