Context Folding

Problem

A single linear context cannot hold a hundred-step trajectory, yet most of the intermediate detail produced while exploring one subtask stops mattering once that subtask returns its result. Keeping every step wastes the window and slows the model, while discarding steps blindly loses the thread. The agent needs a way to spend a large working context on a subtask and then reclaim almost all of it, retaining only the outcome.

Solution

Expose two control actions to the agent. The branch action opens a new sub-context seeded with just the subtask goal; the return action closes it and writes back only a short outcome summary to the parent trajectory. The agent reasons and calls tools freely inside the branch, and when it returns, the intermediate steps are folded away so the parent sees one compact result. The decision of when to branch and what to keep is learned during training (FoldGRPO assigns credit through the fold) rather than hard-coded by the harness, so the agent folds where it pays off.

When to use

• Tasks routinely exceed the context window because they decompose into many self-contained subtasks.
• The detail generated inside a subtask is not needed once the subtask returns a result.
• The folding policy can be trained or fine-tuned rather than relying on prompt-only self-summarisation.