I · ReasoningEmerging★

Least-to-Most Prompting

also known as L2M, Easy-First Decomposition

Decompose a hard problem into an ordered list of easier subproblems, then solve them sequentially with each answer feeding the next.

Context

A team is using a model on a task class where short, training-style examples work fine but longer or more complex instances fail. For example, the model can handle two-step word problems but starts losing pieces on five-step ones, or it follows two-clause instructions but drops information when there are seven. Plain chain-of-thought reasoning closes some of this gap but still breaks down at the hard end of the distribution.

Problem

Even with chain-of-thought, the model is still trying to span the whole problem in a single reasoning trace. As the problem grows, the trace gets long and the model loses track partway through, makes a wrong commitment early, and never recovers. Without an explicit way to break a hard instance into ordered, simpler subproblems and have the model see each one in turn with the prior answers in hand, accuracy collapses on exactly the cases where the technique was supposed to help.

Forces

  • Decomposition prompts are themselves a design problem.
  • Two stages double minimum cost.
  • Errors in the decomposition cascade.

Example

A maths-tutoring agent is asked a multi-step word problem that combines unit conversion, percentage, and ratio. Plain chain-of-thought gets the unit conversion right but loses the ratio. The team adds least-to-most: stage one prompts the model to list subproblems easiest-first ('1: convert km to m, 2: compute percentage, 3: apply ratio'); stage two solves each in order, feeding prior answers forward. Accuracy on the hard end of the eval set jumps because each step starts from a clean, simpler frame.

Diagram

Solution

Therefore:

Two-stage prompt. Stage 1 (decomposition): prompt the model to list subproblems from easiest to hardest. Stage 2 (sequential solve): for each subproblem in order, prompt the model with the original question, prior subproblem answers, and the current subproblem.

What this pattern forbids. Subproblems must be solved in the listed order; out-of-order solving is forbidden.

And the patterns that stand alongside it, or against it —

  • alternative-toChain of Thought★★— Elicit multi-step reasoning by prompting the model to produce intermediate steps before its final answer.
  • complementsSelf-Ask★★— Have the model emit explicit follow-up sub-questions, answer them (optionally via search), then compose the final answer.
  • complementsPlan-and-Execute★★— Plan all the steps once with a strong model, then execute each step with a cheaper model under the plan.
  • complementsGoal Decomposition★★— Decompose a goal into sub-goals recursively until each leaf is directly actionable.
  • alternative-toQuery-Decomposition Agent★★— An agent whose explicit job is to split an incoming user query into smaller independent sub-queries that can be answered sequentially or in parallel, then merge results.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance