Deliberation Over Observation

Problem

The model spends that budget reconstructing the environment's state by inference rather than reading it: reasoning about what a file probably contains instead of opening it, about why a test probably fails instead of running it, about what a directory probably holds instead of listing it. Because the chain stays self-consistent it reads like progress, and nothing in the transcript marks the moment an assumption went unchecked. Past a point more thinking lowers task success rather than raising it. Across 4018 SWE-bench Verified trajectories, an overthinking score validated against human expert assessment correlated negatively with resolution rate, and reasoning-tuned models scored higher on that measure than non-reasoning ones.

Solution

This entry names the anti-pattern; the corrective is to make observation the default and deliberation the budgeted exception. Bound the thinking a step may spend while a cheap, decisive observation is available, so an assumption about a file, a test result or a directory is resolved by reading rather than by inference. Score each trajectory by the ratio of internal reasoning tokens to environment interactions and treat a high ratio as a defect signal, not as thoroughness; where several samples exist, ship the low-ratio one. Allocate the budget by difficulty rather than flat, so hard cases keep their reasoning while simple ones stop paying for it, and log the step at which the agent last touched the environment so a long chain built on stale observations is visible in review.

When to use

• A reasoning-trained model runs in an interactive environment where ground truth is one cheap tool call away.
• The model decides for itself, at each step, whether to extend its chain or call a tool, and the thinking budget is large.
• Task success has stopped improving, or has fallen, as the thinking budget was raised.
• Trajectories show long reasoning between few environment interactions, or actions taken on state that was never read.