Multi-Principal Welfare Aggregation
When an agent serves multiple humans with conflicting preferences, declare the aggregation rule explicitly rather than letting it be implicit in the prompt or fine-tune.
Problem
Without an explicit aggregation rule the agent silently picks one principal — usually the loudest, the most recently heard, or the one whose preferences were fine-tuned in earliest. Gibbard's theorem says any aggregation rule that aggregates more than two principals' preferences is manipulable: principals can strategically misreport. Pretending there is no aggregation rule does not avoid this; it picks the implicit rule and hides it from review.
Solution
When the agent's action space affects multiple principals, route the decision through an explicit aggregation function. Options: sum-of-utilities (utilitarian); weighted welfare (declared per-principal weights); collegial mechanism (each principal must be obtaining 'enough' reward through their own actions for their preferences to count); role-priority (some principals have veto). Surface the active rule in traces and documentation. Make it a configuration change, not a prompt change.
When to use
- Agent serves multiple principals whose preferences can conflict.
- Actions are zero-sum or rivalrous across principals.
- Operators or users need to understand and adjust how aggregation works.
Open the full interactive page →
Diagram, neighbourhood map, code examples, related patterns and full provenance.