On-Demand Tool Synthesis

Problem

A fixed toolset cannot cover an open task space, yet shipping every conceivable tool is impossible and bloats tool selection. When the agent hits a capability gap mid-task it has no clean way forward: hallucinating a tool fails, and forcing the wrong tool produces wrong results. The agent needs a way to manufacture the missing capability as a proper, callable tool, and to do so without blindly trusting code it just wrote.

Solution

Split the work into a tool-creator and a tool-user. When the user role finds no tool fits the subtask, it hands the specification to the creator role, which writes the tool's code and interface, generates or runs a test to confirm it behaves, and registers it in the tool registry. The user role then calls the new tool exactly as it would a built-in one, and the tool persists for later reuse. Execution of synthesized code runs in a sandbox, and a tool that fails its validation check is discarded rather than registered.

When to use

• The task space is open-ended and a fixed toolset will inevitably hit capability gaps.
• The agent can run code in a sandbox and write a check that validates a new tool before use.
• Synthesized tools are worth keeping because similar gaps recur.