A model can understand the task and still be unable to finish it
Ask an AI a question and it can often tell you whether it knows the answer. Ask it to complete a real task and the situation changes.
A task may require a browser session, a file that is not present, a permission the model does not have, a fresh source of truth, an external dependency, or authority to make a change. The model can understand the instructions and still be unable to reach the outcome.
Answer confidence and task feasibility therefore need to be evaluated separately.
Before the work starts
Before meaningful work begins, a system should be able to check more than whether the prompt makes sense.
Capability:
can it perform this type of work at all?
Environment:
are the required tools and systems available now?
State:
is the source information current, and does the system know which state is authoritative?
Authority:
is it actually allowed to take the actions the task requires?
Dependencies:
is it waiting on another person, service, file, approval, or event?
Verification:
is there a credible way to determine whether the outcome really happened?
If the system cannot answer those questions, a confident “yes, I can do that” is weak evidence.
What the evidence does and does not tell us
Research on model uncertainty is promising but incomplete. Larger language models can express useful confidence signals, and some uncertainty interfaces improve how people interpret model knowledge. But benchmark confidence is still far from perfect.
More importantly, most calibration work asks whether a model knows an answer. It does not ask whether the model can complete an end-to-end task under live tools, permissions, dependencies, time limits, and effect boundaries.
Agent benchmarks show why that difference matters. In REAL, a 2025 benchmark covering 112 practical multi-step tasks across deterministic replicas of 11 websites, the best evaluated frontier models achieved at most 41% success. A system can be impressive on short tasks and still fail often enough on longer workflows that delegation becomes expensive. Source: REAL
A five-part preflight
Before delegating meaningful work, we think a useful preflight should make five things explicit:
- What outcome are we trying to create?
- What tools, information, permissions, and dependencies are required?
- What could block completion?
- How confident is the system that it can finish under the current conditions?
- What evidence will prove completion?
That preflight gives the user something concrete to inspect instead of a generic confidence score.
The open question
We do not yet know whether an AI can reliably forecast its own end-to-end task success across varied real workflows.
We also do not know how much preflight detail helps a user before it becomes another form of overhead.
GO Beyond proposes testing that directly: compare generic confidence with a structured readiness check that inspects tools, state, authority, dependencies, predicted success, and the evidence plan.
Before you delegate important work
If the consequence is low, a rough attempt may be perfectly reasonable.
If the work matters, ask a better question than “Can you do this?” Ask: “What do you need in order to finish this, what could stop you, and how will we know when it is actually done?”
For important work, the system should be able to explain what it needs, what could block completion, and how the outcome will be checked.
Sources
- Can LLMs Express Their Uncertainty? — Calibration and failure-prediction methods across multiple models and datasets.
- What large language models know and what people think they know — Human calibration around model explanations and uncertainty.
- METR — Measuring AI Ability to Complete Long Software Tasks — Task length and success-rate horizons; living benchmark.
- REAL benchmark — State-checked multi-step website tasks.