XIV · Anti-PatternsAnti-pattern✕

Context Gap (Security)

also known as Security-Rule-Following Without Implication-Understanding

Agents faithfully follow explicit security rules but miss the broader implications — they log access correctly without flagging the unusual pattern a human expert would catch immediately.

Context

A security-aware agent is told to log file access, verify permissions, encrypt storage, etc. The agent does all of this correctly. But it doesn't think like a security professional — it executes the rules without grasping the security-implication landscape they're meant to address.

Problem

Rule-following without implication-understanding misses the security signals that the rules were designed to surface. The agent logs the file access; it doesn't flag that the access happened at 3am from a new IP. The agent verifies permissions; it doesn't notice that the same user requested unusually many sensitive files this week. Rule-following without context is compliance-theater, not security.

Forces

  • Encoding all security implications as explicit rules is infinitely-many edge cases.
  • Asking the agent to 'think like a security expert' produces hallucinated security reasoning.
  • Security context drift means yesterday's rules don't catch tomorrow's threats.

Example

A file-access agent is configured with logging rules: log filename, timestamp, user ID. It does so faithfully. A breach attempt: an attacker uses a compromised credential to access 47 sensitive files in 8 minutes from an unusual IP. Rule compliance: 100% — every access logged. Security detection: 0% — the agent never flagged the unusual pattern. A human security analyst would have seen the pattern in seconds. Fix: separate anomaly-detection pipeline; agent does compliance, not security judgment.

Diagram

Solution

Therefore:

Acknowledge the gap. Pair with: policy-as-code-gate (deterministic rule enforcement), policy-gated-agent-action (audit-trail tagging), human-in-the-loop (review for novel patterns), eval-harness (anomaly-detection metrics independent of rule compliance). Cite Paredes et al. 2021 (arXiv 2108.02006).

What this pattern forbids. No useful constraint; the missing constraint is separating compliance (the agent can do) from security judgment (the agent cannot).

And the patterns that stand alongside it, or against it —

  • alternative-toPolicy-as-Code Gate★— Evaluate every proposed agent action against externally-managed machine-readable policies before dispatch, so compliance authorship lives outside the prompt and outside the agent code.
  • complementsPolicy-Gated Agent Action (KRITIS)★— Each agent action passes through a policy gate (NIS2, EU the agent Act, BSI rules) and is tagged with Run ID + Model Digest + Policy Hash for WORM-audit reconstruction.
  • complementsHuman-in-the-Loop★★— Require explicit human approval at defined points before the agent performs an action.
  • complementsShadow Canary★★— Run a candidate agent version in shadow alongside the champion, comparing outputs without affecting users.
  • complementsContext Window Dumb-Zone Cap★— Hold context-window utilization below a working threshold (~40%) to keep the model out of the 'dumb zone' where it begins ignoring earlier instructions and hallucinating.
  • complementsFalse Resolution✕— The agent proposes a compromise that addresses each constraint individually but subtly violates one in joint interpretation, shipping as success but discovered as failure at audit.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance