Consensus-Averaging Over Expertise

Problem

LLM teams tend toward integrative compromise: they average the expert's view with the non-experts' rather than deferring to the member most likely to be right, so the joint answer lands below what the expert would have produced alone. Telling the team who the expert is does not fix it — the failure is not identifying expertise but using it against the pull of consensus. And because every extra member adds more views to average in, the dilution grows with team size, so scaling the team makes the answer worse, not better.

Solution

Treat aggregation as the design problem, not team size. Weight each member's contribution by demonstrated competence on the specific task rather than averaging all views equally, and give the expert's judgement decisive weight on the parts of the task it covers, while still drawing on others where they are stronger. Verify that adding a member actually raises the team's expected answer before adding it, since more voices that get averaged in can lower it. Where a single member is clearly most reliable, let the team defer rather than negotiate the answer toward the mean. The goal is to draw on the best member, not to blend everyone.

When to use

• Recognising this failure when a multi-agent team's answer is worse than its single best member's.
• Reviewing a team that aggregates by undifferentiated consensus regardless of who is most competent.
• Diagnosing why adding agents to a team lowers accuracy instead of raising it.