Mirrored Tool Environment

also known as Simulated Tool Environment, Virtual API Server, Mirror API, LLM-as-Search-Engine Simulator

Replace the live external tool during agent training and benchmarking with a fine-tuned model that emits responses in that tool's shape, so runs stay cheap, reproducible and controllable in difficulty.

This pattern helps complete certain larger patterns —

used-byEval Harness★★— Run a held-out dataset against agent versions to detect regressions and measure improvement.

Context

A team trains or benchmarks an agent that calls an external service: a web search engine, a public REST API, a commercial data provider. Every rollout of a reinforcement-learning run and every replay of a benchmark issues real calls against that service. A single training run can require hundreds of thousands of requests. The service charges per call, returns different content between runs, goes down, rate-limits bulk traffic and deprecates endpoints over the life of the benchmark.

Problem

The live service is the wrong thing to train or measure against, for two independent reasons. Per-call pricing on frequent rollouts bounds how much training is affordable at all. And the content that comes back is outside anyone's control: response quality varies unpredictably between runs, so the same agent scores differently on the same task, and a misaligned response corrupts the reward signal by penalising correct reasoning or rewarding a fabricated answer. Freezing a static snapshot of the service removes the cost and the variance but introduces its own misalignment, because the snapshot stops answering the queries the agent learns to ask.

Forces

Reinforcement-learning rollouts can run to hundreds of thousands of requests per training run, so per-call pricing decides how much training is affordable rather than how much is useful.
Live responses vary between runs in ways nobody controls, so the same task returns a different reward and the signal becomes noisy rather than merely hard.
A static snapshot removes cost and variance but drifts out of alignment with the queries the agent learns to ask, and that misalignment corrupts the reward by penalising correct reasoning or rewarding hallucination.
A generated response is useful only if behaviour learned against it transfers, so fidelity traded away for cost reappears later as a gap between measured and live performance.
Difficulty under local control is a training asset because noise can be scheduled, but the same control makes it easy to train against an environment that is systematically gentler than production.

When to use

Reinforcement-learning or benchmark runs would issue enough calls to an external service that per-call cost or rate limits decide the size of the experiment.
Benchmark scores must stay comparable over time, but the live service changes its results, goes down or deprecates endpoints between runs.
Training benefits from controlling how good the tool's answers are, for example scheduling retrieval quality from clean to noisy.
Enough real request and response pairs exist, or can be harvested once, to fit a responder that is faithful in shape and content.

When not to use

The run serves an end user, where a generated response would be presented as a real result.
No corpus of real request and response pairs exists, so the mirror would be fitted to guesses about the interface.
The measurement is specifically about live-service behaviour such as latency, uptime, freshness or rate-limit handling.
The tool is already cheap, stable and deterministic, in which case calling it directly is simpler and more faithful.

Example

A small team wants to train an agent to search well, but each training run would send several hundred thousand queries to a paid search API, and the results come back different every day. They fine-tune a model on real query-and-result pairs, drop it into the search slot, and start each run with clean results before mixing in more and more junk. Training now costs nothing per query and gives the same numbers twice. Once a week they run a short check against the real search engine, to make sure the agent has not learned to trust a mirror that flatters it.

Diagram

flowchart TD A["Harvested request/response pairs"] --> B["Fine-tune mirror model"] C["Agent issues tool call"] --> D{"Observed response cached?"} D -- "yes" --> E["Replay cached real response"] D -- "no" --> F["Mirror generates response"] B --> F G["Curriculum controller: noise ratio"] --> F E --> H["Agent continues rollout"] F --> H H --> I["Training or benchmark score"] I --> J{"Holdout run on live service"} J -- "gap small" --> K["Report as claim about production"] J -- "gap large" --> L["Refit mirror, do not report"]

Solution

Therefore:

Harvest real request and response pairs from the tool being mirrored, then fine-tune a model to produce a same-shaped response for a request it has not seen. During training or benchmark runs the mirror occupies the tool slot: the agent issues an ordinary call and receives an ordinary-looking result, with no signal that the responder generated it rather than fetched it. Because the responder is a local model, response quality becomes a scheduled variable instead of an accident, so a curriculum can begin with clean, useful results and mix in a rising proportion of noise as training proceeds, forcing the agent to reason through bad retrieval rather than memorise one retrieval distribution. A cache of genuinely observed responses sits in front of the mirror and serves whatever it can, handing the mirror only the requests real data cannot cover, which keeps fidelity high where evidence exists. The mirror stands in for the tool's response channel only; it never substitutes for the agent's own reasoning, and any number measured against it is checked against a smaller holdout run on the live service before it is reported as a claim about production.

What it gives you

Training and benchmark runs stop paying per-call fees to a commercial provider, so run cost no longer scales with the provider's price list; TRUSTEE reports task generation, user simulation, tool simulation and trajectory evaluation all covered by a free open model as small as 8B.
Runs are reproducible, because the responder does not change between them the way a live service does.
Response quality becomes a schedulable curriculum, so an agent can be trained deliberately against degrading retrieval instead of whatever noise the live service happened to emit that day.
A tool whose live endpoint is down, deprecated or rate-limited still has a benchmark presence, so scores stay comparable across time.
Behaviour learned in the mirror can transfer: SearchGym reports a Qwen2.5-7B agent trained purely in simulation beating a web-enhanced baseline by 10.6% relative across nine benchmarks.

What it costs you

The mirror is a model, so it can emit a plausible response the real service would never return, and an agent that learns against that fabrication carries the error into production.
Fidelity has to be measured rather than assumed; without a periodic holdout against the live service the gap between mirrored and real performance stays invisible.
Building the mirror needs a corpus of real request and response pairs, so the live service must first be called enough to harvest one — MirrorAPI's corpus spans more than 7,000 APIs.
A curriculum degrades responses only in the ways the mirror knows how to degrade them, which trains resilience against a narrow slice of the real failure distribution.
The mirror inherits the licensing and terms of service of the provider whose responses it was fitted to, and redistributing it can redistribute that provider's data.

What this pattern forbids. A mirrored responder may occupy the tool slot only during training and benchmark runs; it must never serve a production request, and no claim about live behaviour may be made from mirror-only results without a holdout run against the real service.

And the patterns that stand alongside it, or against it —

complementsWorld Model as Tool·— Let a planning agent invoke a generative world model as a tool to roll out hypothetical futures before committing to an action, treating the world model as a callable simulator rather than a training target.
complementsMental-Model-In-The-Loop Simulator·— Run candidate multi-step strategies inside an internal simulator of the environment before committing in the real world — broader than simulate-before-actuate (single action) by simulating multi-step strategies.
alternative-toSemantic Response Cache★— Embed each query and, when its nearest cached neighbour is within a similarity threshold, return the stored answer instead of re-running the model so near-duplicate questions are answered cheaply.
complementsDimensional Synthetic Eval Set★— Generate evaluation inputs not by free-form LLM prompting (which mode-collapses) but by enumerating tuples over explicitly named dimensions and seeding generation from each tuple.
complementsDual Evaluation (Offline + Online)★— Run two parallel evaluation tracks — offline benchmark gates before deploy AND online production-traffic monitoring after — so drift is caught even when pre-deploy benchmarks pass.
complementsDemo-to-Production Cliff✕— Anti-pattern: ship a demo-validated agent straight into production without a frozen eval, cost ceiling, loop-detector, or named oncall, then act surprised when accuracy drops and cost runs away.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance

Source: patterns/mirrored-tool-environment.md on GitHub · commit 4eec3f8 · view history
Added to catalog: 2026-09-09
Last updated: 2026-09-09
Contribute: open an issue or PR at github.com/agentpatternscatalog/patterns.