VII · Verification & ReflectionEmerging★

Physical-Evidence Progress Gate

also known as Achievement-Grounded Memory, Sense-Confirmed Subgoal Advance, Progress Pointer Verification Gate

Advance an embodied agent's task-state pointer only after re-sensing the world confirms the subgoal actually happened, never on the fact that the action was attempted.

This pattern helps complete certain larger patterns —

used-byReplan on Failure★★— Trigger a fresh planning step when execution evidence contradicts the current plan.

Context

An embodied agent works through a task written as a sequence of subgoals — approach the mug, close the gripper, lift, place — and keeps a pointer to the subgoal it believes it has reached. Later planning, retrieval and reporting all read that pointer. The acting policy is often frozen: a pretrained vision-language-action model that emits the next command and moves on, with no channel for saying that the command did not land. Between the command and the pointer update there is a gap, and something has to decide what fills it.

Problem

The physical world returns no status code. A grasp that closed on the object and one that closed a centimetre short produce the same emitted command and the same absence of an error, so treating an attempted action as completed progress is the cheapest available rule and the one most systems fall into. That rule converts a recoverable local slip into a persistent belief that the subgoal is done. Every subsequent step is then planned against a world state that does not exist, and the error compounds instead of being corrected at the point where it was still one missed grasp.

Forces

The physical environment returns no status code: a missed grasp and a successful one look identical to the policy that emitted the command, and only going back and looking separates them.
A verification look costs a real observation and sometimes a repositioning move, so checking after every command is unaffordable, while checking after none is what turns a local execution error into a persistent task-state error.
A frozen policy cannot be retrained to report its own failures, so the check has to sit outside it — AGM adds it as a single 2.43M-parameter verification head over frozen foundation models rather than touching the policy at all.
The slips that matter are ordinary ones, a missed grasp, a dropped object, an unexpected collision, but demonstration data collected from successful runs contains no example of noticing or recovering from any of them.
Evidence from sensors is often partial, because occlusion, sensor drift and state-estimation error can hide an achievement that really occurred, so a strict gate stalls the task while a lenient one readmits the silent failure it was added to catch.

When to use

An action's success cannot be read back from any authoritative record and has to be re-observed through sensors.
The task is represented as a sequence of subgoals with a pointer or memory that later steps plan against.
The acting policy is frozen or otherwise cannot be changed to report its own execution failures.
Ordinary slips such as a missed grasp or a dropped object are frequent enough that compounding them is the dominant failure mode.

When not to use

The effect of an action is readable from an authoritative system of record, where a direct read-back is cheaper and more reliable than re-sensing.
Every subgoal is cheap to redo from scratch, so a wrong pointer costs one retry instead of a compounding error.
No available sensor can observe the subgoal's effect, in which case the gate would produce false negatives without catching anything.
The control loop is tight enough that no post-actuation observation fits inside the step budget, and the check belongs at a slower supervisory layer instead.

Example

A home robot is told to load four mugs into the dishwasher. On the second mug the gripper closes a centimetre short and comes up empty, but the policy has already moved on to the next command, so the robot records the mug as loaded. It then places nothing on the rack and continues with the third and fourth, and at the end it reports four mugs loaded while one is still sitting on the counter.

Diagram

flowchart TD A["Policy emits action for current subgoal"] --> B["Actuate"] B --> C{"Proprioceptive cue suggests an interaction?"} C -- "no" --> D["Keep acting, pointer unchanged"] C -- "yes" --> E["Re-observe scene, compare against pre-action view"] E --> F{"Does the named subgoal effect hold?"} F -- "positive evidence" --> G["Advance progress pointer, write subgoal achieved"] F -- "negative or unclear" --> H["Hold pointer, raise retry or replan"] H --> A

Solution

Therefore:

Treat the progress pointer as a write that has to be earned. After the policy acts, a cheap signal from the body — gripper closure, contact force, a joint-torque spike — decides whether an interaction plausibly occurred and therefore whether a verification look is worth its cost. When it fires, the agent re-observes the scene and compares the new observation against the state before the action, tying the observed change to the language of the subgoal: point tracking establishes that the named object moved, a cross-view comparison establishes that it ended where the subgoal said it should. Only a positive verdict advances the pointer and writes the subgoal as achieved. A negative verdict, or evidence too partial to judge, leaves the pointer where it is and hands the verdict to retry, replanning or escalation. The default on silence is to hold, not to advance, so the gate degrades toward stalling rather than toward a confidently wrong world model.

What it gives you

A missed grasp stays a missed grasp instead of becoming a wrong world state that every later subgoal is planned against.
The check works with a frozen acting policy, so an off-the-shelf vision-language-action model gains failure awareness without retraining or fine-tuning.
The proprioceptive trigger keeps the perceptual cost proportional to how often an interaction actually occurred rather than to how many commands were emitted.
A held pointer is an explicit, inspectable signal that something went wrong, which is what a replanner or a human operator needs in order to act.

What it costs you

Each verification look costs a real observation and on some embodiments a repositioning move, so a trigger that fires too readily slows the task measurably.
A false negative stalls a subgoal that in fact succeeded; under occlusion or sensor drift the gate can refuse to advance on a completed step and retry it indefinitely.
The verifier is another learned component reading the same cameras as the policy, so the scenes that fool one can fool both.
A review of runtime action authorization for physical systems finds that no surveyed research stream supplies a complete boundary between a black-box model and physical execution, so a gate built today covers the subgoal effects someone thought to name and is silent about the rest.

What this pattern forbids. The task-state pointer cannot advance because an action was attempted or a policy emitted it; the memory write is permitted only after a post-actuation observation returns positive evidence that the subgoal's effect holds, and evidence that is absent or ambiguous must never be scored as success — the pointer stays where it is.

And the patterns that stand alongside it, or against it —

complementsAffordance Grounding Before Action·— Have a vision-language model ground each candidate action against the current scene and predict its affordance, so that actions the environment cannot physically support are discarded before any reach the controller.
complementsPhysical Hallucination✕— Anti-pattern: an embodied or process-control agent issues a confidently-phrased command that is syntactically valid but physically infeasible or unsafe, because nothing checks it against geometry, dynamics, or actual plant state before actuation.
complementsPhantom Action Completion✕— Anti-pattern: the agent reports a side-effecting action as complete from its own narration, when the tool call silently failed or never ran and nothing checked that the effect occurred.
complementsSimulate Before Actuate★— Before issuing an irreversible action, run a deterministic simulation that computes pre-conditions, invariants, and expected deltas; require a verifier — automated or human — to green-light the simulated outcome before the real command is sent.
complementsBehavior Tree Back Chaining·— Construct an agent's behavior tree starting from the desired goal condition and recursively adding child nodes whose post-conditions satisfy each parent's pre-conditions.
complementsAction-Admissibility Tiering★— Gate a proposed physical action through an ordered ladder of independent, non-compensatory admissibility tests, and rank actions by usefulness only among those that passed every tier.
complementsModality-Conflict Arbitration★— Have each observation channel emit its own recorded claim, treat disagreement between channels as a detected event, and resolve it against a pre-declared per-claim-type modality authority rather than an implicit fused guess.
complementsFoveated Perception Escalation★— Treat observation fidelity as a budget the agent allocates: perceive the whole surface coarsely, let the reasoning name the region that decides the answer, then re-observe only that region at full resolution.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance

Source: patterns/physical-evidence-progress-gate.md on GitHub · commit c389d0e · view history
Added to catalog: 2026-09-10
Last updated: 2026-09-10
Contribute: open an issue or PR at github.com/agentpatternscatalog/patterns.