III · Tool Use & EnvironmentEmerging★

Machine-Checkable Tool Contract

also known as Schema-Enforced Tool Constraint, Typed Tool Failure Result

Express every constraint a tool places on its caller in the machine-readable schema rather than the prose description, and return failures as typed fields the caller can branch on.

Context

A tool is exposed to an agent through an interface description such as an OpenAPI document or an MCP tool definition. That description has two channels: a machine-readable schema of parameters and types, and a prose field meant for a human reader. Constraints that matter for a call to succeed — which values an enumeration accepts, which date ranges are servable, which combinations of arguments are legal — can be written into either channel. The prose channel is cheap to write, so it accumulates most of them.

Problem

A constraint stated only in prose is enforced by nobody. The schema validator does not see it, the server often does not check it, and the model reads it as a suggestion among many. A call that violates such a constraint is not rejected: it runs, matches nothing, and returns a well-formed empty result with a success status. The caller has no field to branch on, cannot distinguish a query that legitimately matched nothing from one the server never understood, and so reports the empty result as a finding.

Forces

  • Prose is the cheapest place to record a constraint and the only place that can express nuance, but it is the one channel no validator enforces.
  • A schema constraint costs authoring effort up front and rejects some calls that would have worked, while a prose constraint costs nothing and silently admits calls that cannot.
  • An empty result and an unsatisfiable query are indistinguishable at the transport layer, so the caller cannot recover a distinction the provider never encoded.
  • Typed failure fields are useful only if the provider commits to them; a boolean error flag tells the caller that something went wrong but not what to do next.

Example

A research agent calls a company's public filings API for documents from 1998. The API only serves records back to 2001, a limit stated in the endpoint's description text but absent from the parameter schema. The call returns HTTP 200 with an empty list, and the agent tells the user the company filed nothing that year. Had the year range been declared in the schema, the call would have been rejected with a message the agent could act on.

Diagram

Solution

Therefore:

Treat the tool description as a contract with two obligations. On the way in, every constraint that determines whether a call can succeed is declared in the machine-readable schema — enumerations for closed value sets, patterns for formats, ranges for servable bounds — so an unsatisfiable call is rejected at validation time instead of running and returning nothing. On the way out, a failure is not a boolean flag but a structured result: a specific cause, the parameter or resource at fault, an executable repair where one exists, and whether replaying the call is safe. The caller then branches mechanically on the failure kind, fixing an argument, authenticating, backing off, or choosing another tool, rather than inferring intent from a prose message. Prose keeps the nuance a schema cannot hold, but it stops being the only record of anything load-bearing.

What this pattern forbids. No constraint that determines whether a call can succeed may live only in the prose description; it must be declared in the machine-readable schema, and a failure result must not be a bare boolean flag.

And the patterns that stand alongside it, or against it —

  • complementsTool Output Trusted Verbatim✕— Anti-pattern: trust whatever tools return without validation, schema enforcement, or trust labels.
  • complementsTyped Refusal Codes★— Define a single source of truth for machine-readable refusal codes across all guard surfaces, so refusals can be triaged mechanically rather than by string-grepping ad-hoc human-readable messages.
  • complementsAgent-Computer Interface★— Design the tool surface for an LLM agent specifically, with affordances different from human-facing CLIs.
  • complementsSilent External-Source Rot✕— Anti-pattern: an agent keeps reporting success while a wrapped external source has silently changed structure, so its tool returns valid-but-empty or degraded output that nothing watches.
  • complementsException Handling and Recovery★★— Catch and react to predictable failure modes (tool errors, rate limits, validation failures) with structured recovery paths.
  • complementsStructured Output★★— Constrain the model's output to conform to a JSON Schema (or similar typed shape).

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance