XIV · Anti-PatternsAnti-pattern✕

Black-Box Opaqueness

also known as Opaque Agent, No-Trace Agent

Anti-pattern: ship an agent without traces, decision logs, or provenance, then debug from user reports.

Context

A team is shipping an LLM-based agent under schedule pressure, often using a framework that emits no traces by default. Observability — recording each model call, each tool invocation, and the decision that led to it — is treated as something to add later once the product proves itself. The agent goes to production with no run logs, no decision log, and no record of which inputs led to which outputs.

Problem

When the agent eventually does something wrong, and it will, the team has no record of what the agent saw, what it decided, or which tool it called with which arguments. Debugging collapses into trying to reproduce a user's vague timeline from memory, and most incidents are never explained at all. The team ends up retrofitting traces during an outage, which is the most expensive moment to add them.

Forces

  • Observability has a cost (storage, dev time).
  • Frameworks differ in trace quality.
  • Privacy and trace coverage tension.

Example

A startup ships a customer-facing agent in a hurry with no traces, no decision logs, and no tool-call provenance. A week later a user complains the agent issued a duplicate refund. The team has nothing to look at — they spend two days trying to reproduce the bug from the user's vague timeline and never definitively explain it. This is the Black-Box Opaqueness anti-pattern: the absence of observability is itself the failure, and recovery requires retrofitting traces to every step before the next incident.

Diagram

Solution

Therefore:

Don't. Add traces, decision logs, and provenance from day one. See provenance-ledger, decision-log, lineage-tracking.

What this pattern forbids. Avoiding it means observability is not optional: an agent must not ship without traces, decision logs, and provenance attached to every action, and debugging must never depend on user reports alone.

The patterns that counter or replace it —

  • alternative-toProvenance Ledger★★— Log every agent decision and state change with enough metadata to explain or reverse it later.
  • alternative-toDecision Log★★— Persist the agent's reasoning trace alongside its actions so post-hoc review can explain why.
  • alternative-toLineage Tracking★★— Track which prompt version, model version, and data sources produced each agent output.
  • complementsAgent Confession as Forensics✕— Anti-pattern: after an agent-caused incident, the team treats the agent's confabulated self-narrative as the forensic record and root cause, even though the self-report is generated rather than remembered and can be flatly wrong.
  • complementsAccountability Laundering via Algorithm✕— Anti-pattern: route a hard decision through an agent so no person owns the outcome, treating the recommendation as the decision while the firm's legal liability stays unchanged.
  • complementsAgent Tool-Invocation Data Black-Box✕— Anti-pattern: behind a single chat interface an agent silently invokes third-party tools that route the user's personal data to undisclosed destinations, so the user cannot see which tools or data services handle it.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance