XIV · Anti-PatternsAnti-pattern✕

Physical Hallucination

also known as Embodied Hallucination, Infeasible-Command Hallucination

Anti-pattern: an embodied or process-control agent issues a confidently-phrased command that is syntactically valid but physically infeasible or unsafe, because nothing checks it against geometry, dynamics, or actual plant state before actuation.

Context

An agent drives an embodied or physical system — a robot arm, a mobile robot, a process-control loop — by emitting commands to actuators or controllers. The model produces those commands from a language or vision-language policy, the same way it produces text, choosing the most plausible next action given the goal. The physical world, unlike a text channel, has hard constraints: reachable poses, dynamic limits, collision geometry, and a current plant state the command must be consistent with.

Problem

The model has no faithful internal model of physics, so it can emit a command that reads as correct and is phrased with full confidence yet cannot be executed. Examples are an unreachable arm pose, a dynamically infeasible motion, a path through an obstacle, or a setpoint inconsistent with the plant's current state. Because the command is syntactically valid and confident, downstream systems that trust the policy pass it through to actuation, where it fails, damages equipment, or creates a hazard. The failure is grounded in the physical world the model cannot perceive faithfully, not in missing facts or tools.

Forces

  • A language or vision-language policy generates the most plausible command, and plausibility is not feasibility — a fluent command can be physically impossible.
  • Physical constraints (reach, dynamics, collision, plant state) are not legible in the token stream, so the policy is not penalised for violating them at generation time.
  • A confidently-phrased command invites trust from a controller that has no independent feasibility check.
  • Adding a grounding or simulation gate before actuation costs latency and engineering that a direct policy-to-actuator path avoids.

Example

A warehouse robot is told to place a box on the top shelf. The vision-language policy confidently emits an arm pose to reach it, but the pose is outside the arm's workspace; with no feasibility check the controller accepts it, the arm jams against its limit, and the motion faults. A grounding gate that predicted the pose was unreachable would have rejected the command before the arm ever moved.

Diagram

Solution

Therefore:

Treat every command from the policy as a proposal to be checked against physics, not an instruction to execute. Insert a feasibility stage between policy and actuator that grounds the candidate action in the current scene and the system's kinematic, dynamic, and state limits. That stage can predict the action's affordance from perception, or roll it out in a simulator or world model, and it rejects or repairs any action the environment cannot support. Only feasible, in-state commands reach the controller; infeasible ones are discarded, re-planned, or escalated. The check is independent of the policy's own confidence, so a fluent but impossible command cannot pass merely by being well-phrased.

What this pattern forbids. A policy-generated command must not reach an actuator on its confidence alone; it has to pass an independent feasibility check against the current scene and the system's physical limits before execution, and an infeasible command is discarded rather than issued.

The patterns that counter or replace it —

  • alternative-toAffordance Grounding Before Action·— Have a vision-language model ground each candidate action against the current scene and predict its affordance, so that actions the environment cannot physically support are discarded before any reach the controller.
  • complementsSimulate Before Actuate★— Before issuing an irreversible action, run a deterministic simulation that computes pre-conditions, invariants, and expected deltas; require a verifier — automated or human — to green-light the simulated outcome before the real command is sent.
  • complementsHallucinated Tools✕— Anti-pattern: trust the model to invoke only the tools it has been given, then debug calls to functions that do not exist.
  • complementsPhantom Action Completion✕— Anti-pattern: the agent reports a side-effecting action as complete from its own narration, when the tool call silently failed or never ran and nothing checked that the effect occurred.
  • complementsMental-Model-In-The-Loop Simulator·— Run candidate multi-step strategies inside an internal simulator of the environment before committing in the real world — broader than simulate-before-actuate (single action) by simulating multi-step strategies.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance