Finetune-as-Last-Resort Escalation
Make teams use up prompt engineering, retrieval, and task splitting before they fine-tune, because fine-tuning is the most expensive and the hardest to undo.
Description
Treat fine-tuning as the last step on a ladder, not the first lever you grab. The rule of thumb: fine-tuning is for form, retrieval is for facts. Climb the ladder in order. First prompt engineering, then few-shot prompting, then retrieval-augmented generation (RAG), then advanced RAG, then splitting the task or using an agent to run the steps. Only reach for fine-tuning once those steps run out. Every step before it is cheap to undo. Fine-tuning is not.
When to apply
Use this whenever a team is thinking about fine-tuning a model for a specific behaviour or quality. Apply it as a gate: have we tried the cheaper steps first? Don't apply it when fine-tuning is clearly the right tool, such as teaching a fixed response format, adding a new input type, or shrinking a large model into a small one. Also skip the ladder when the data clearly shows prompt engineering has stalled and the gap that remains is about form.
What it involves
- Diagnose the gap before climbing
- Step 1: prompt engineering
- Step 2: few-shot prompting
- Step 3: retrieval-augmented generation
- Step 4: task decomposition or agent orchestration
- Step 5: fine-tune
Open the full interactive page →
Diagram, neighbourhood map, code examples, related patterns and full provenance.