XIV · Anti-PatternsAnti-pattern✕

Hallucinated Tools

also known as Phantom Tool Calls, Imagined Functions

Anti-pattern: trust the model to invoke only the tools it has been given, then debug calls to functions that do not exist.

Context

An agent is configured with a registered set of tools — a tool palette — that it is supposed to choose from on each turn. The host code that receives the model's tool call accepts whatever name and arguments the model emits and dispatches them without first checking that the name actually exists in the registered palette. The team assumes that because the model was shown the palette in the prompt, the model will only call tools from it.

Problem

Models routinely invent tool names that look reasonable but are not registered — a slight rename, a pluralised version, an imagined helper that should logically exist. The unvalidated host then either crashes with an unhelpful error, silently drops the call, or, in the worst case, fuzzy-matches the invented name to a similar real tool and executes the wrong action with side effects. Without strict validation at the dispatch boundary, phantom calls become indistinguishable from legitimate ones in the logs.

Forces

  • Validation feels redundant when providers offer typed tool calls.
  • Provider-side validation is not always strict.
  • Logging fails to surface 'tool does not exist' as a first-class event.

Example

A coding agent in production starts logging mysterious errors: 'unknown function: search_repo_v2'. The model invented a tool name that almost matches a real one and the host quietly dispatched to the closest match, deleting a file. The team recognises hallucinated-tools as the underlying anti-pattern and adds a strict allowlist: every tool call is validated against the registered palette, unknown names return a typed error the agent reads on the next turn, and fuzzy matching is forbidden. The phantom calls disappear within a day.

Diagram

Solution

Therefore:

Don't trust. Validate every tool call against the registered palette before dispatch. Reject unknown names with a typed error the agent can react to. See tool-use, structured-output.

What this pattern forbids. By definition, this anti-pattern imposes no useful constraint; the missing constraint is the failure mode.

And the patterns that stand alongside it, or against it —

  • alternative-toTool Use★★— Let the LLM produce typed calls against an external toolkit instead of producing free-form text the surrounding system has to parse.
  • alternative-toStructured Output★★— Constrain the model's output to conform to a JSON Schema (or similar typed shape).

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance