The AI Said It Was Done. What Counts as Evidence?

“Done” can mean four different things

AI systems are fluent enough that a completion statement can feel like evidence. It is not.

There are at least four different states that often get collapsed into one word: done.

1. The process ran. A command, workflow, or generation finished.
2. An output exists. A file, message, answer, or artifact was produced.
3. The expected state changed. The thing that was supposed to become true actually became true.
4. The outcome was independently verified. A source outside the system's own assertion confirms it.

Those states can all be useful. They are not interchangeable.

Why a false completion claim is costly

A false completion claim changes human behavior. The person stops checking, moves to the next task, tells somebody else the work is finished, or makes a decision based on a state that does not exist.

The damage is rarely limited to the first mistake. It creates cleanup, duplicated work, lost time, and distrust in everything that follows.

As workflows become more autonomous, the cost of confusing those states grows.

Evidence should come from the state that matters

Evidence should fit the task. A document task may need an exact file and a review result. A website change may need the expected content in the correct repository, a deployed build, and a rendered check. A scheduled action may need provider confirmation. A data task may need the final dataset and validation against the source.

Useful evidence can include:

• state checks from the system that owns the truth;
• exact artifact identity, version, or hash;
• confirmation that a requested external effect actually occurred;
• an independent read-only verifier;
• negative checks showing that a partial or wrong-target result did not occur.

Use the lightest evidence level that matches the consequence

Not every task needs Level 4. The appropriate level depends on consequence, reversibility, and the cost of being wrong.

Why state checks matter

NIST's AI risk-management guidance emphasizes measurement, monitoring, documented limits, and incident response rather than self-asserted trustworthiness.

Controlled agent benchmarks such as REAL use deterministic state checks precisely because narrative claims are not enough to establish success. Across 112 practical multi-step tasks on deterministic replicas of 11 websites, the best evaluated frontier models achieved at most 41% success. That is not a universal production failure rate; it is strong evidence that state-based verification exposes gaps that fluent completion language can hide. Source: REAL

That does not prove one universal receipt format. It does support a broader design principle: when the outcome matters, verification should come from the state that matters.

The first experiment we want to run

Our first proposed original experiment compares several completion presentations: a simple “done” statement, a narrative checklist, a machine-readable state receipt, and a state receipt plus an independent verifier.

We want to know whether stronger completion evidence can reduce false-completion acceptance without creating another layer of management work.

A practical rule

For low-stakes work, a plausible answer can be sufficient.

For important work, ask: “What changed, where can I see it, and what would prove you are wrong?”

That gives the user something concrete to inspect instead of relying on the system's own completion claim.