Together Mixture-of-Agents (MoA)
Together Mixture-of-Agents sends a prompt to several open-source LLMs acting as proposers and has a final aggregator LLM synthesize their responses into one answer.
Description
Mixture-of-Agents is a method and reference implementation from Together AI that runs several LLMs in parallel on the same prompt, then passes their outputs to a final aggregator LLM that synthesizes a single response. The proposer models run independently in one layer; the aggregator combines their results in a second layer. The architecture can be stacked across multiple layers, where each layer comprises several LLM agents. The reference code is published under Apache 2.0.
Solution
A prompt is sent in parallel to several proposer LLMs, each producing an independent response. The collected responses are passed to a final aggregator LLM whose instruction is to synthesize them into a single high-quality answer. The layer can be repeated, feeding aggregated output back into another round of proposers and aggregation for further refinement.
Primary use cases
- combining multiple open-source LLMs on one prompt
- synthesizing several model responses into one answer
- quality improvement through proposer-aggregator layering
Open the full interactive page →
Diagram, neighbourhood map, code examples, related patterns and full provenance.