XIV · Anti-PatternsAnti-pattern✕

Context Fragmentation

also known as Working-Memory Limit Failure, Simultaneous-Constraint Holding Failure

Anti-pattern: the LLM cannot hold multiple interconnected constraints in mind simultaneously the way human working memory can; it processes each constraint locally and loses the cross-constraint view.

Context

An agent task requires reasoning over a constraint web — a crossword where each cell intersects two clues, a schedule where each slot constrains and is constrained by others. Humans hold the web in working memory; LLMs process tokens through attention which is capable but architecturally distinct from working memory.

Problem

The model's attention mechanism, though it accesses all input tokens, does not replicate the human ability to hold a small number of interconnected variables in immediate joint focus. Each constraint gets attended to locally; the joint constraint structure is not represented. The agent satisfies each constraint individually and violates them jointly. Differs from lost-in-the-middle (positional bias) by being about simultaneous holding of constraints, not about position.

Forces

  • Attention mechanism is the architecture; rewriting it is research-level work.
  • Some constraint webs are too large to enumerate explicitly.
  • Forcing the model to write out each constraint explicitly adds latency.

Example

A scheduling agent given 12 meetings with overlapping participant and resource constraints. The model satisfies each meeting's constraints individually but produces a schedule with three double-bookings — each individually plausible but jointly violating participant constraints. Fixing requires externalizing the participant-by-time matrix, not asking the model to 'try harder'.

Diagram

Solution

Therefore:

Pair with: strategic-preparation-phase (enumerate constraints explicitly), generate-and-test-strategy (verify against explicit list), large-reasoning-model-paradigm (LRMs handle this better via deliberation). For severe cases, decompose into sub-problems whose constraint sub-webs are small enough to hold.

What this pattern forbids. No useful constraint; the missing constraint is explicit constraint-web externalization for tasks beyond a working-memory threshold.

And the patterns that stand alongside it, or against it —

  • alternative-toStrategic Preparation Phase★— Mandate an explicit problem-space representation step before the agent attempts solutions, mirroring how expert humans build a mental model of constraints and dependencies before solving.
  • alternative-toGenerate-and-Test Strategy★— Generate multiple candidate solutions in parallel, then systematically test each against declared constraints rather than committing to the first plausible one — adapted from Langley & Simon's cognitive-science research on human expert problem-solving.
  • alternative-toLarge Reasoning Model (LRM) Paradigm★— Route reasoning-heavy tasks to a reasoning-tuned model that trades inference time for deliberation, rather than to a fast LLM that exhibits premature-closure.
  • complementsLost in the Middle (Positional Bias)✕— LLM accuracy on retrieving information from long contexts drops sharply when relevant content sits in the middle of the prompt rather than at the start or end.
  • complementsPremature Closure✕— The LLM commits to a confident answer before processing all constraints, characteristic of constraint-heavy tasks where it fills in plausible answers fast and gets cross-constraint interactions wrong.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance