Speculative Agentic Actions

Problem

Many of the tool calls an agent will make are highly predictable from the current state — after listing a directory it will read the obvious file, after a failing test it will open the named stack-frame, after a search hit it will fetch the top result. Forcing each of these through its own confirm-then-act turn spends a full model round-trip and a transcript replay on a decision the agent had already implicitly made. The agent needs a way to run ahead of itself on the predictable stretches without committing to a wrong branch when the prediction misses.

Solution

Add a speculation step to the agent loop. From the current observation a lightweight predictor proposes the tool call (or short chain of calls) the agent is most likely to issue next, and the harness dispatches those calls speculatively while the main model reasons about the same turn. When the model commits to its actual next action, the harness checks it against the speculation: on a hit it splices in the already-computed result and skips the round-trip, collapsing two or more turns into one; on a miss it drops the speculative result and falls back to the normal act-observe step. Speculation is confined to read-only, idempotent, side-effect-free calls so a discarded prediction costs only wasted compute, never corrupted state. The prediction horizon is bounded by confidence, so the loop speculates aggressively where the next step is near-certain and conservatively where it is not.

When to use

• The next tool call is frequently predictable from the current observation, so a cheap predictor can hit often enough to pay for its misses.
• Per-turn cost (model round-trip plus transcript replay) dominates the run and the task spans many turns.
• The predictable calls are read-only and idempotent, so a discarded misprediction wastes only compute and not external state.