XIV · Anti-PatternsAnti-pattern✕

Consensus-Averaging Over Expertise

also known as Integrative-Compromise Dilution, Expert-Dilution-by-Consensus

Anti-pattern: a self-organising LLM team pursues integrative compromise, averaging expert and non-expert views instead of weighting the known expert, so team output falls below the best member and degrades further as the team grows.

Context

A task is handed to a team of LLM agents rather than a single model, on the assumption that more perspectives produce a better answer. One agent is, or is even explicitly designated as, the domain expert for the task. The team deliberates and converges on a joint answer through discussion and mutual adjustment.

Problem

LLM teams tend toward integrative compromise: they average the expert's view with the non-experts' rather than deferring to the member most likely to be right, so the joint answer lands below what the expert would have produced alone. Telling the team who the expert is does not fix it — the failure is not identifying expertise but using it against the pull of consensus. And because every extra member adds more views to average in, the dilution grows with team size, so scaling the team makes the answer worse, not better.

Forces

  • Consensus-seeking is a reasonable default for combining views, but it dilutes a member who is genuinely more likely to be right.
  • Identifying the expert is easy; deferring to them against the social pull toward the average is not.
  • Each added member contributes another view to compromise over, so the dilution scales with team size.
  • A single expert acting alone avoids the dilution but loses the genuine value other members add on the parts they do know.

Example

A medical-coding task is given to a five-agent team that includes one agent fine-tuned for clinical coding. The team discusses each case and settles on a consensus code. The specialist is right far more often than the others, but its answers get blended with theirs, so the team's accuracy lands below the specialist's solo score — and adding two more generalist agents to help makes it worse.

Diagram

Solution

Therefore:

Treat aggregation as the design problem, not team size. Weight each member's contribution by demonstrated competence on the specific task rather than averaging all views equally, and give the expert's judgement decisive weight on the parts of the task it covers, while still drawing on others where they are stronger. Verify that adding a member actually raises the team's expected answer before adding it, since more voices that get averaged in can lower it. Where a single member is clearly most reliable, let the team defer rather than negotiate the answer toward the mean. The goal is to draw on the best member, not to blend everyone.

What this pattern forbids. A team's answer must not be formed by averaging all members' views equally; contributions are weighted by demonstrated competence, the expert's judgement is not diluted by less-competent members, and a member is added only when it raises the team's expected answer.

The patterns that counter or replace it —

  • complementsVoting-Based Cooperation★— Finalise a decision across multiple agents by collecting and tallying their votes on candidate options, so the joint output reflects collective rather than single-agent judgement.
  • complementsDynamic Expert Recruitment·— Generate the agent team — role descriptions and instances — at run time based on the specific task, then adjust team composition between iterations based on evaluation feedback.
  • complementsSame-Model Self-Critique✕— Anti-pattern: have the same model both produce an answer and critique it, expecting independence.
  • complementsMulti-Agent on Sequential Workloads✕— Anti-pattern: split a fundamentally sequential workload across multiple agents, degrading accuracy by 39–70% with no parallelization benefit.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance