XIV · Anti-PatternsAnti-pattern✕

False Confidence Syndrome

also known as Uniform-Confidence Failure, Calibration Failure

Anti-pattern: the model produces incorrect answers with the same high confidence as correct ones, failing to vary its expressed certainty with its actual reliability — Oxford-documented for constraint-heavy prompts.

Context

An agent produces analytical outputs across a workload with mixed difficulty. Some answers it should be confident about; others it should hedge. The model's expressed confidence (in prose tone, in any numeric confidence it provides) doesn't track its actual reliability — it sounds certain on confident-but-wrong answers just like on confident-and-right ones.

Problem

The user has no signal to weight outputs differently. Sycophancy adjacency: the user pushes back, the model doubles down with the same confident tone, rationalizing rather than reconsidering. The downstream cost is decisions made on outputs that should have been flagged as uncertain.

Forces

  • Confidence calibration requires the model to know what it doesn't know — hard.
  • User experience favors confident tone; hedged outputs feel weak.
  • Forcing per-output confidence annotations adds output complexity.

Example

A medical-triage agent gives confidence-sounding diagnoses across cases. Audit shows: when the agent was wrong, it expressed the same confidence as when it was right. A clinician noted: 'I couldn't tell when to push back.' Fix: confidence-checking-workflow with per-diagnosis calibration, plus calibration-monitoring eval that flags uniform-high-confidence batches.

Diagram

Solution

Therefore:

Pair with: confidence-checking-workflow (force per-part annotation), reflexive-metacognitive-agent (explicit self-model), eval-harness (measure calibration). Treat uniform-confidence outputs as a calibration alarm. Cite Pawitan & Holmes 2024 (arXiv 2412.15296) for the Oxford findings.

What this pattern forbids. No useful constraint; the missing constraint is per-output / per-part calibrated confidence.

And the patterns that stand alongside it, or against it —

  • alternative-toConfidence-Checking Workflow★— Always ask the agent, for each part of its output, to state its confidence and identify which parts need human verification, like triaging a junior analyst's work.
  • alternative-toReflexive Metacognitive Agent·— Agent maintains an explicit self-model of its own capabilities, confidence and limitations, and reasons over that model when accepting / refusing / handing off tasks.
  • complementsSycophancy✕— Anti-pattern: train or tune an agent on user-preference feedback without a counter-balancing truth signal.
  • alternative-toConfidence Reporting★— Surface the agent's uncertainty about its answer alongside the answer itself.
  • complementsPremature Closure✕— The LLM commits to a confident answer before processing all constraints, characteristic of constraint-heavy tasks where it fills in plausible answers fast and gets cross-constraint interactions wrong.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance