XIV · Anti-PatternsAnti-pattern✕

Same-Model Self-Critique

also known as Echo-Chamber Reflection, Single-Model Reflexion

Anti-pattern: have the same model both produce an answer and critique it, expecting independence.

Context

A team builds a reflective agent — Reflexion, self-refine, or an evaluator-optimizer loop — where one call produces a candidate answer and a second call critiques and revises it. To keep cost and integration simple, both calls use the same model, often with prompts that share their wording about what 'good' looks like. The critique step is then presented internally or to users as an independent check on the answer.

Problem

Because producer and critic come from the same weights and read overlapping prompts, the critic shares the producer's blind spots; whatever the model is confidently wrong about, it is also confidently wrong about when wearing the critic hat. Wrong answers come back from the loop endorsed and slightly polished, and the team reports higher confidence on what is, statistically, the same error rate. Replication studies through 2025 have repeatedly confirmed that single-model self-critique catches surface mistakes but does not act as independent verification.

Forces

  • Two models cost twice.
  • Cross-model judges have their own biases.
  • Self-critique feels free.

Example

A team ships an agent where the same model writes an answer and then 'self-critiques' it before returning, and treats the critique as independent verification. Replication studies and their own evals show the critic confidently endorses confidently-wrong answers because it shares the producer's blind spots. They stop pretending independence: they either accept that self-critique catches surface errors only, or they swap the critic to a different model family.

Diagram

Solution

Therefore:

Don't pretend it is independent. Either accept that self-critique catches surface errors only, or use a different model family for the critic. See reflection, evaluator-optimizer, llm-as-judge.

What this pattern forbids. By definition, this anti-pattern imposes no useful constraint; the missing constraint is the failure mode.

And the patterns that stand alongside it, or against it —

  • alternative-toReflection★★— Have the model review its own output and produce a revised version in one or more passes.
  • conflicts-withEvaluator-Optimizer★★— One LLM generates; another evaluates and feeds back; loop until criteria are met.
  • conflicts-withSelf-Refine★★— Iterate generate → feedback (same model) → refine until a stop criterion fires, with no separate critic model.
  • alternative-toDegenerate-Output Detection★— Detect when the agent is about to emit a near-duplicate of its own recent output and either drop, replace, or escalate to a stronger model rather than ship the loop.
  • alternative-toBlind Grader with Isolated Context★— Run an evaluator in a separately-allocated context window with access only to the artifact and the rubric, never the producing agent's reasoning trace, so the grader cannot be primed by the producer's framing.
  • complementsSycophancy✕— Anti-pattern: train or tune an agent on user-preference feedback without a counter-balancing truth signal.
  • alternative-toCross-Reflection★— Reflection step performed by a *different* agent or foundation model from the original generator, so critique error is decorrelated from generation error.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance