III · Tool Use & EnvironmentEmerging★

Hierarchical Tool Selection

also known as Tool Tree, Categorised Tool Catalog, Two-Stage Tool Routing

Organise tools into a tree of categories so the agent first picks a branch and then a specific tool within it.

Context

An agent has access to dozens or hundreds of tools — every public API the company exposes, every micro-action across many domains (billing, identity, scheduling, search, code, files). Presenting them all in the system prompt blows up the context window and overloads the model's selection step.

Problem

A flat tool list collapses in two ways past roughly 30 tools. Token cost grows linearly in description length × tool count. Selection error rises non-linearly as the model confuses similar tools or misses the right one entirely. Worse, permissions and ownership are flat too — there is no scope at which a team can say 'these are the billing tools, this team owns them'. The agent ends up either under-tooled (some tools dropped) or unreliable (the model picks wrong).

Forces

  • Token cost of tool descriptions scales with catalog size.
  • Model selection accuracy degrades past a few dozen choices.
  • Permissions, ownership, and audit naturally group by domain.
  • The first-stage choice (category) must be cheap enough not to cost what was saved.

Example

An ops agent has 180 tools spanning eight domains. The system prompt presents only the eight category names. After picking 'billing' the agent receives the 24 billing tools. End-to-end latency is one extra decode at the top; total tokens drop ~80% on average and selection accuracy on a 200-trace eval rises from 64% to 91%.

Diagram

Solution

Therefore:

Group tools into named categories (billing, identity, scheduling, search, code, files). At the top level the agent sees only the category names with one-line descriptions. After it picks a category, it sees the tools in that branch. Permissions can scope per branch (this user can read but not write billing tools). For very large catalogs nest the tree further. The cost is one extra decoding step at the top; the saving is paying full tool descriptions only for the chosen branch.

What this pattern forbids. A large tool catalog must not be presented as a flat list to the model; tools are organised into named categories and the agent first picks a category before seeing tool-level descriptions.

And the patterns that stand alongside it, or against it —

  • complementsTool Use★★— Let the LLM produce typed calls against an external toolkit instead of producing free-form text the surrounding system has to parse.
  • complementsAgent Skills★— Package author-time procedures (markdown + optional resources) the agent loads on demand for specific task types.
  • complementsModel Context Protocol★★— Standardise how agents discover and call tools so that a tool written once is usable by any conformant agent.
  • complementsMCP Bidirectional Bridge★— Run a framework as both MCP client (consuming external MCP servers as tools) and MCP server (publishing its own agents, tools, and workflows back over MCP) so capabilities flow both directions across the protocol boundary.
  • complementsAgent-Computer Interface★— Design the tool surface for an LLM agent specifically, with affordances different from human-facing CLIs.
  • composes-withTool Transition Fusion·— Mine tool-call telemetry for high-probability X-then-Y transitions and fuse those pairs into a single composite tool, shrinking the planner's step count.
  • alternative-toOne Tool, One Agent★— Design agent systems as a team of narrow single-purpose agents, each owning one tool or one capability, rather than a single super-agent that handles every tool — the agent analogue of microservices over monolith.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance