IV · Retrieval & RAGEmerging

Tacit-Knowledge Elicitation Agent

also known as Expert Knowledge Externalisation Agent, SECI Externalisation Agent

Run a front-loaded phase in which an agent interviews domain experts and converts their undocumented know-how into a structured, queryable knowledge base before downstream task agents read from it.

Context

A team wants to automate work that veterans perform from experience — judging which supplier substitution is safe, reading a machine's vibration, sensing when a deal is about to fall through. The decisive knowledge was never written down. It lives as habit and intuition in a few people's heads, and the manuals, tickets, and wikis the retrieval layer can index capture only the surface procedure, not the judgement criteria. When those people retire or move on, the criteria leave with them.

Problem

Tacit knowledge is, by definition, not in the corpus, so a retrieval layer pointed only at documents returns the documented procedure and silently omits the expert judgement that actually decides outcomes. The expert cannot simply be asked to write it down, because much of what they know is unconscious and surfaces only when a concrete situation is interrogated. Without a deliberate elicitation step, every downstream agent inherits a knowledge base that looks complete but is missing the load-bearing decision criteria, and its answers are confidently shallow.

Forces

  • The most valuable knowledge is the least documented, so the corpus the retrieval layer indexes is exactly where the gap is widest.
  • Experts cannot reliably introspect and write down criteria they apply unconsciously; the knowledge surfaces only when probed against concrete cases.
  • Interview output is unstructured, redundant, and contradictory, while downstream agents need clean, queryable, de-duplicated criteria.
  • Elicitation is expensive senior time spent up front, paying off only later when many task agents read from the resulting base.

Example

An accounting team wants an agent to flag invoices that need senior review. The documented rules catch the obvious cases, but the veteran reviewer catches subtle ones the manual never mentions. Before building the agent, an elicitation phase interviews the reviewer about specific past invoices, extracts the criteria she actually applies, and writes them into a structured knowledge base. The review agent then queries that base, so it reasons with her judgement rather than only the written policy.

Diagram

Solution

Therefore:

Treat externalisation as an explicit pipeline phase that runs before the task agents are switched on. An interviewing agent works through veteran staff with open-ended, case-anchored questions, capturing transcripts of how each concrete decision was actually made. A processing step transcribes the sessions, then automatically summarises and classifies the content, extracting the decision criteria and rules of thumb that distinguish an expert's call from a novice's. A consolidation step merges these with existing internal and external documents and writes them into a structured knowledge base — schema-tagged criteria, worked cases, and the conditions under which each applies — that downstream retrieval and task agents query. The phase maps onto the SECI externalisation move: turn what was tacit and individual into explicit, organisational knowledge that can be reused and reproduced.

What this pattern forbids. Downstream task agents may not invent decision criteria from parametric memory; they answer judgement questions only from criteria present in the elicited knowledge base, and surface a gap when the base lacks an applicable criterion.

The smaller patterns that complete this one —

  • usesSocratic Questioning AgentDrive the agent toward its goal by asking the user a sequence of strategic, open-ended questions that surface the user's own latent knowledge, goal, or context — rather than producing an answer directly.

And the patterns that stand alongside it, or against it —

  • complementsGraphRAGBuild an LLM-extracted entity-and-relation knowledge graph plus hierarchical community summaries, then answer global queries via map-reduce over those summaries.
  • complementsKnowledge Graph MemoryPersist agent memory as entities and relations in a structured graph so symbolic queries (path, neighbour, type) become possible.
  • alternative-toDynamic Expert Recruitment·Generate the agent team — role descriptions and instances — at run time based on the specific task, then adjust team composition between iterations based on evaluation feedback.
  • alternative-toUnstructured Human Capture LayerKeep a human-authored raw dump layer that the agent may read but never edit, and confine all structuring to a separate derived layer, so half-formed human thought survives as durable context.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.