IX · Routing & CompositionEmerging★

Trust and Reputation Routing

also known as Reputation-Based Agent Selection, Trust-Weighted Routing

Maintain a per-agent reputation score updated from outcome quality and peer feedback, and route new tasks preferentially to high-reputation agents.

Context

A platform hosts many agents (third-party plug-ins, model variants, internal specialists). Tasks arrive that any of several agents could plausibly handle. The routing decision is currently 'pick the first capable' or 'round-robin' or 'pick by static rank'.

Problem

Static routing wastes the platform's most valuable signal: track record. Agents that have historically produced good outcomes get the same allocation as agents that have repeatedly failed. New tasks are routed to the wrong agents because routing ignores past evidence. Without a reputation layer, the platform cannot learn from outcomes; bad agents stay in rotation and good agents are under-used.

Forces

  • Reputation must be updated from outcome signal (success rate, user rating, peer review).
  • Reputation must be slow to gain and fast to lose, or attacker agents game it.
  • Cold-start agents need exploration weight or they never get a chance.
  • Reputation must be auditable to be legitimate.

Example

A code-agent marketplace hosts 40 plug-in agents claiming various capabilities. After tasks complete, the user rates and a quality LLM-judge scores the result. Each agent's reputation updates. A new refactoring task is routed with weight proportional to reputation across the agents that claim refactoring capability; a small fraction goes to a newly-registered agent (cold-start exploration). Repeatedly-bad agents fade out of rotation without manual deprovisioning.

Diagram

Solution

Therefore:

For each agent maintain a reputation score updated after each task from outcome signals (deterministic success, user rating, peer review by another agent). Route new tasks by sampling weighted by reputation, with a small exploration term for newcomers (cold-start). Decay reputation over time so stale records don't dominate. Surface reputation scores in operator dashboards. Distinct from a router LLM (which picks once per request based on intent): reputation routing is statistical and longitudinal.

What this pattern forbids. Candidate agents must not be treated as equally trustworthy after track records diverge; routing is weighted by reputation with an explicit cold-start exploration term.

The smaller patterns that complete this one —

  • usesAgent-as-a-Judge★— Evaluate an agent's full trajectory (steps, tool calls, intermediate states) by another agent rather than scoring only the final output.

And the patterns that stand alongside it, or against it —

  • complementsRouting★★— Classify an incoming request and dispatch it to the specialist (lane / agent / model) best suited to handle it.
  • complementsCoalition Formation·— Agents form temporary subgroups around a task because the coalition can achieve more value than the sum of its members acting alone, with explicit rules for who joins and how payoff or credit is shared.
  • complementsContract Net Protocol★★— Classical bid-based multi-agent task allocation: a manager broadcasts a task announcement, contractors submit bids, and the manager awards the contract to the best bid.
  • complementsShadow Canary★★— Run a candidate agent version in shadow alongside the champion, comparing outputs without affecting users.
  • alternative-toBayesian Bandit Experimentation★— Replace fixed-split A/B tests between agent variants with a bandit that dynamically reallocates traffic toward better-performing variants based on observed reward, bounding regret from bad variants.
  • complementsMulti-Principal Welfare Aggregation·— When an agent serves multiple humans with conflicting preferences, declare the aggregation rule explicitly rather than letting it be implicit in the prompt or fine-tune.
  • complementsVickrey Auction Allocation★★— Allocate a task to the lowest sealed bidder but pay them the second-lowest bid, making truthful cost reporting a dominant strategy.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

Used in recipes

References

Provenance