The pitch arriving in consultancies, agencies and accounting practices this month is the overnight shift. Hand the agent the brief at six, read the finished work at nine the next morning. For a firm that sells its people’s time, that is the most attractive proposition in software: output that does not consume a timesheet.
The capability is real and it has moved fast. The number being quoted alongside it is not the number a firm should be making decisions with.
01 — What happened
On 19 October, Notion published an update to its agents. The headline, in the company’s own words: agents “can now work on a single task for more than 24 hours, up from five minutes.” The same post claims a model router matching frontier-model results at roughly 90% lower average cost across more than twenty common use cases. Those are the vendor’s figures, not independent ones.
Set that beside how agent capability is actually measured.
METR, an independent evaluation organisation, publishes a metric called the task-completion time horizon. It is the length of task — measured by how long a human expert would take — that an agent is predicted to finish at a given reliability. It is a measure of task difficulty, not of how long the AI spends working. In its original 2025 study, METR found that the length of task frontier agents could complete with 50% reliability had been doubling roughly every seven months.
The critical detail is in that phrase: 50% reliability. In METR’s Frontier Risk Report covering February to March 2026, the publicly available frontier models had a 50% time horizon of about 12 hours. Their 80% time horizon was about 1.5 hours. The best models METR had access to, including unreleased ones, sat at 16 hours or more at 50%, and between three and four hours at 80%.
Two further findings matter more than either figure. METR notes its task suite cannot reliably measure horizons above 16 hours, so the long end of the curve is bounded by the ruler, not the model. And on tasks of eight hours or longer, METR found that at least 16% of apparently successful runs were illegitimate on review — the task had been gamed rather than completed.
Then there is the shape of the failure. Toby Ord’s 2025 paper modelled agent failure as a constant risk per minute of human task time, which produces an exponential decline in success as tasks get longer, and gives every agent something like a half-life. He revised it in February 2026: hazard rates do not stay constant, they “systematically decline as the task goes on.” That cuts both ways. Success on tasks longer than the 50% horizon is better than the simple model predicts. Success on tasks shorter than it is worse.
02 — Why it matters
Three numbers are now circulating as if they described the same thing. They do not.
Twenty-four hours is a run-time ceiling: how long the software is permitted to keep working. Twelve hours is a 50% horizon: a coin flip. Ninety minutes is an 80% horizon: still a one-in-five failure rate, which no firm would accept from a graduate.
A professional services firm does not buy run time. It buys finished work it can put its name on. On that measure, the gap between the marketed number and the usable one is roughly eightfold.
The reward-hacking finding is the one that should change behaviour. It means a long unattended run can return something with the shape of a deliverable — sections filled, citations present, tables populated — that did not come from doing the work. On a five-minute task you notice. On an overnight research brief, you notice in the client meeting.
And because hazard is front-loaded, you cannot reason your way down the curve. “It completes twelve-hour tasks half the time, so a twenty-minute task is nearly certain” is exactly the inference Ord’s revision rules out. Short tasks are less reliable than the clean exponential suggests. The riskiest minutes are the first ones, where the agent is working out what it has been asked, what it can reach and what the output should look like.
03 — The economics
Reliability compounds against you.
Take an illustrative research-and-draft job broken into ten steps — scope, source, read, extract, check, outline, draft, cite, format, summarise. Assume each step independently succeeds 95% of the time. The chain finishes correctly 60% of the time. At 99% per step it finishes 90% of the time. This is why long manual processes have checkpoints.
Now price the failure modes. A short step that fails costs a few minutes and is obvious. A long unattended run that fails costs the metered compute for the whole run, the overnight delay, and the time someone spends working out which part went wrong — usually by redoing it. A run that appears to succeed costs all of that plus the credibility of the output — discovered later, by someone else.
Against that, count the saving honestly. If an overnight agent produces a brief that takes a senior four hours to verify properly, the firm has not saved a day of work. It has bought a four-hour review, and the review is unbillable. We have argued before that hours saved is the wrong way to value an automation; long unattended runs are the clearest case. The variable that decides whether the economics work is not how long the agent can run. It is how quickly a human can tell whether it was right.
That is why cheaper tokens, welcome as they are, do not resolve the question. Lower cost per run changes the price of attempts. It does not change the completion rate, and the completion rate is what you are actually buying.
04 — What businesses should do
For a firm of, say, fourteen people producing twenty client research briefs a month, five rules follow directly from the evidence.
Define the checkable unit before the automation. Break the job into steps whose output a person can verify in under two minutes: a source list with links, a table of extracted figures against their origin, a drafted section with citations attached. If a step’s output cannot be checked at a glance, it is too long.
Do not let a run outlast the review cycle. If nobody will look until nine the next morning, do not start work at six that can be silently wrong by eleven. The length of your unattended run should match the length of your attention, not the vendor’s ceiling.
Log three outcomes, not one. Completed, completed-but-wrong, failed. Most firms track the first and the third. METR’s 16% figure says the middle column exists and is large, and it is the only one that tells you whether to extend the leash.
Spend on the first five minutes, not the next twenty hours. Front-loaded hazard means the returns are in clean inputs, correct access to the real client record, and a specified output format. That is the same conclusion we reached about agent access being the bottleneck, and it is consistent with what exceptions do to automation economics.
Ask the vendor the right question. Not “how long can it run?” but “what proportion of runs on work like ours finish correctly without intervention?” If the answer is a run-time number, you have been given a capacity figure in place of a quality one.
05 — Where technology fits
For most professional services firms, the honest answer is that the overnight agent is not the thing to buy yet.
What usually pays first is duller and sits underneath it: an integration so the agent reads the actual matter, client or engagement record instead of a pasted summary; a defined output format so a result can be checked mechanically; and a verification step somebody owns. That is integration and automation work of the kind described in how we think about systems, and it is what makes any later agent useful.
Narrow, repetitive, high-volume extraction with a checkable output — pulling dates, obligations and figures out of inbound documents into a record — is where this technology is already worth the money, because the output is verifiable in seconds and the volume justifies building the check. Open-ended overnight research is the opposite: unverifiable in seconds, low volume, high consequence.
And sometimes the answer is nothing. If the task is twenty minutes of a senior’s time twice a week, designing a verifiable automation around it costs more than the time it returns. That judgement, not the capability, is the work — which is why our process starts with the operating model rather than the tool, and why the economics differ so much across professional services firms depending on how standardised their output already is.
06 — RevenueStack’s take
Our view: the important development is not that agents can now run for a day. It is that run length has become the number vendors market while reliability has improved far more slowly — and the two are measured on completely different scales, so nobody notices the substitution.
Run time is a vendor metric. The business metric is the longest piece of work you can hand over and verify at a glance. For most firms that is still measured in minutes, and it will stretch on its own as the underlying models improve. Design around the verifiable unit and every capability gain arrives as a free upgrade. Design around the headline and you have bought something that can be confidently wrong while everyone is asleep.
Sources
- Agents, rebuilt to do more for less — Notion, 2026-10-19
- Frontier Risk Report (February to March 2026) — METR, 2026-05-19
- Task-Completion Time Horizons of Frontier AI Models — METR, 2026-09-08
- Measuring AI Ability to Complete Long Tasks — METR — Kwa, West, Becker et al., 2025-03-19
- Is there a half-life for the success rates of AI agents? — Toby Ord — arXiv:2505.05115, 2025-05-08
- Hazard Rates for AI Agents Decline as a Task Goes On — Toby Ord, 2026-02-04
