XIV · Anti-PatternsAnti-pattern✕

Schema-Free Output

also known as Free-Form Tool Call, String-Parsing the Model

Anti-pattern: parse free-form model output for downstream code instead of using structured output.

Context

A team uses an LLM to produce values that downstream code consumes — a JSON-looking blob, a yes/no decision, a list of records — but the model is asked for free-form text and the consumer parses it with regular expressions, string splits, or substring checks like 'does the word yes appear here'. The provider offers structured output (a JSON Schema or function-calling contract that constrains the model's output), but the team has not adopted it, often because the integration looked like extra setup at the time. The model's text is treated as essentially typed even though nothing enforces that.

Problem

The model varies its punctuation, capitalisation, field names, and ordering in ways the parser was not written for: smart quotes instead of straight quotes, a missing comma, a 'sure, here is the answer' preamble the parser tried to skip but did not. The downstream code fails in non-obvious ways, corrupts state, or silently misinterprets the result. Post-mortems then blame the model for being flaky when the real bug is in the parser, and the team chases evals that were never going to fix a parsing problem.

Forces

  • Structured output adds setup cost and provider lock-in.
  • Some providers offered structured output later than tool use.
  • Free-form feels flexible until it breaks.

Example

A team ships an agent whose downstream consumes free-form model output by regex-parsing 'looks like JSON.' Edge cases (smart quotes, missing commas, surprise prose preamble) fail in non-obvious ways, and post-mortems blame the model when most failures are parser bugs. They stop doing this and switch to structured output with a JSON Schema, validating against it and retrying on parse failure. The 'flaky model' framing dissolves into a parser-bug fix.

Diagram

Solution

Therefore:

Don't. Use structured-output (JSON Schema, Pydantic, function calling). See structured-output, tool-use.

What this pattern forbids. By definition, this anti-pattern imposes no useful constraint; the missing constraint is the failure mode.

And the patterns that stand alongside it, or against it —

  • alternative-toStructured Output★★— Constrain the model's output to conform to a JSON Schema (or similar typed shape).
  • alternative-toTool Use★★— Let the LLM produce typed calls against an external toolkit instead of producing free-form text the surrounding system has to parse.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance