IV · Retrieval & RAGEmerging★

RAFT

also known as Retrieval-Augmented Fine-Tuning, Distractor-Robust RAG

Train the model to ignore irrelevant retrieved documents (distractors) in a domain-specific RAG setting.

This pattern helps complete certain larger patterns —

  • specialisesNaive RAG★★— Condition the generator on top-k chunks retrieved from an external dense index so knowledge lives outside parameters.

Context

A team is using retrieval-augmented generation in a specific domain and has observed that retrieval almost always returns a mix of documents. Some of the retrieved chunks are genuinely relevant to the user's query; others are topically similar distractors that share keywords or themes but do not actually answer the question. An off-the-shelf retrieval-augmented model attends to all of these chunks and is over-confident on the distractors that look plausible at a glance.

Problem

Generic models trained on broadly relevant retrievals have not been taught to be sceptical of plausible-looking distractors in their context. When the retrieval mixes one relevant document with two or three convincing distractors, the model's answer drifts towards the loudest irrelevant source, often quoting it directly back at the user. The team needs the model to learn, during fine-tuning, how to ignore distractors in its context window and rely only on the truly relevant documents when those exist — and the team needs to do this with a training procedure that simulates the real retrieval mix rather than assuming clean inputs.

Forces

  • Training data construction (oracle docs + distractors) is its own pipeline.
  • Domain shift between training and serving distractors.
  • Trade-off between generalisation and domain specialisation.

Example

A clinical-coding RAG assistant keeps citing topically-similar but wrong ICD chapters when the retriever pulls in adjacent conditions. The team builds a RAFT-style training set where each prompt has the oracle code reference plus three convincing distractors, and the gold answer cites only the oracle. After fine-tuning, the model learns to ignore distractors even when they dominate the retrieved context. Production accuracy on the long-tail comorbidity codes climbs without changing the retriever.

Diagram

Solution

Therefore:

Construct training examples where some documents are oracle and others are distractors. Train the model to cite oracle documents and ignore distractors. Couples chain-of-thought with citation discipline.

What this pattern forbids. Cited claims must come from documents marked oracle in training; distractor citations are penalised.

And the patterns that stand alongside it, or against it —

  • alternative-toContextual Retrieval★— Prepend a short LLM-generated description to each chunk before embedding so the chunk carries its situating context.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

Used in recipes

References

Provenance