IX · Routing & CompositionMature★★

Parallelization

also known as Sectioning, Voting, Parallel Branches

Run independent LLM calls concurrently and combine results.

This pattern helps complete certain larger patterns —

  • used-byLLMCompiler·— Take ReWOO's plan-as-DAG and run independent steps in parallel through a task-fetching dispatcher.
  • used-byLead Researcher★★— A lead agent writes a research plan and dispatches parallel sub-agents that fan out for breadth-first information gathering, then merges results.

Context

A task either splits cleanly into independent subtasks that can run side by side — for example reviewing a pull request for security, style, and test coverage — or benefits from running the same prompt several times and combining the results, which is the basis of self-consistency style voting in mathematical reasoning. In both cases the agent is making more than one LLM call where none of the calls depend on each other's output. The provider's rate limits and the team's budget can absorb running these calls in parallel.

Problem

If independent subtasks run one after another, the user waits for the sum of every call even though nothing forces the order. If the model produces only one attempt at a hard reasoning problem, an unlucky sample can be wrong with no chance of catching it because there is nothing to compare against. Sequential single-attempt execution leaves both latency and quality on the table whenever the work is genuinely parallelisable.

Forces

  • Concurrency limits and rate limits.
  • Aggregation logic for voting (majority? best? union?).
  • Cost multiplies linearly with parallel branches.

Example

A code-review agent runs three independent checks on each PR — security scan, style review, and test-coverage analysis. Running them in series adds up to thirty seconds per PR. The team applies parallelization in its sectioning flavour: the three checks run as concurrent LLM calls and the results concatenate into one review. For high-stakes PRs they also use the voting flavour: the security check runs three times and an aggregator emits the majority verdict, catching the occasional outlier hit.

Diagram

Solution

Therefore:

Two flavours. Sectioning: split a task into independent subtasks, run them concurrently, concatenate results. Voting: run the same task multiple times, aggregate by majority or judge.

What this pattern forbids. Branches cannot share state during execution; aggregation is the only join point.

The smaller patterns that complete this one —

  • generalisesSelf-Consistency★★— Sample the same question multiple times at non-zero temperature and aggregate by majority or judge to mitigate hallucination.
  • generalisesMapReduce for Agents★— Split an oversize task into independent chunks, process each in parallel, then aggregate.
  • generalisesBest-of-N Sampling★— Sample N candidate outputs and select the highest-ranked by a reward model or scorer.
  • generalisesParallel Tool Calls★★— Allow the model to emit several independent tool calls in one assistant turn; the host executes them in parallel.
  • generalisesClone Fan-Out Research·— Spawn 100 or more identical, full-capability agent instances in parallel — each a complete general agent rather than a role-specialised worker — and aggregate their independent outputs into a single answer.
  • generalisesParallel Fan-Out / Gather★— Multiple independent agents execute in parallel on a partitioned task; a dedicated aggregator agent reconciles their results into a single output.
  • generalisesScatter-Gather Plus Saga★— Distribute tasks across worker agents and aggregate results while maintaining distributed-transaction semantics via compensating actions on partial failure.

And the patterns that stand alongside it, or against it —

  • alternative-toPrompt Chaining★★— Decompose a task into a fixed sequence of LLM calls where each step's output becomes the next step's input.
  • complementsIteration Node★★— Express map-over-collection inside a visual workflow as an explicit Iteration node that runs a subgraph once per element of an input array, with bounded, deterministic, observable execution.
  • alternative-toRace Conditions on Shared Tool Resources✕— Anti-pattern: let concurrent agents perform read-modify-write on shared external resources without locking, producing silent data corruption.
  • alternative-toMulti-Agent on Sequential Workloads✕— Anti-pattern: split a fundamentally sequential workload across multiple agents, degrading accuracy by 39–70% with no parallelization benefit.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

Used in frameworks

References

Provenance