IX · Routing & CompositionEmerging★

Agent-to-Rule Distillation

also known as Recurring-Solution Crystallization, Dynamic-to-Deterministic Promotion, Rule Harvesting from Agent Traces

Watch production traces for problems the agent solves the same way repeatedly, and promote each recurring solution into a deterministic rule outside the agent, so the stochastic surface shrinks as the system matures.

Context

An agent embedded in a business process handles the unstructured cases deterministic logic could not be written for upfront. Over months of operation its traces accumulate, and a share of the supposedly unstructured cases turns out to be structured after all: the same problem class arrives again and again, and the agent resolves it along the same path every time. Each of those resolutions pays full model cost and carries stochastic risk for an answer the organisation already knows.

Problem

Keeping a solved-often case inside the agent means re-deriving a known answer forever — at LLM latency and price, with a nonzero chance of fumbling it, and behind reasoning an auditor cannot replay. Writing the rule upfront was impossible because nobody knew the case existed until the agent surfaced it. The knowledge the agent earned in production stays locked in trace logs instead of hardening the process, and the system's cost and risk profile never improves with experience.

Forces

A rule evaluation costs a fraction of an LLM call, and a recurring case pays the model's price on every occurrence.
Auditors and regulators accept deterministic rules far more readily than stochastic reasoning, yet rules cannot be authored upfront for cases nobody has seen.
The agent's flexibility is what discovers the solution in the first place, so removing the agent entirely would also remove the discovery mechanism.

Example

An operations agent handles inbound IT tickets. After a few months, traces show it has resolved 'password reset for contractor account' identically dozens of times: verify identity, check the contract is active, reset, notify. The team promotes that path into a deterministic workflow branch validated against the historical tickets; the agent now only sees requests the branch's guard rejects. Cost per ticket drops sharply and the audit team signs off on the deterministic path without reviewing model reasoning.

Diagram

flowchart LR A[Agent resolves\nunstructured cases] --> T[Trace store] T --> M[Recurrence miner\nsame-solution clusters] M --> R[Rule authored +\nreplay-validated] R --> E[Rule engine] I[Incoming case] --> G{Guard\nmatches?} G -- yes --> E G -- no --> A E --> O[Resolved deterministically]

Solution

Therefore:

Instrument the agent so every case leaves a comparable trace: problem class, resolution path, outcome. Mine the traces for clusters where the same class was resolved along the same path repeatedly — on the order of ten identical resolutions is a workable trigger. For each stable cluster, author the deterministic equivalent (a DMN decision table, a BPMN branch, or plain code), validate it by replaying the historical traces it was mined from, and cut over so the rule short-circuits the agent for matching inputs. Keep the agent as fallback for anything the rule's guard rejects, and link each promoted rule to its source traces so its provenance stays reviewable. Repeated over time this inverts the system's economics: reported production programmes attribute cost reductions up to seventy percent to exactly this transition from dynamic reasoning to rule logic, and the agent's capacity concentrates on genuinely novel cases.

What this pattern forbids. A promoted case must no longer be answered by the model: the rule short-circuits the agent, the agent path survives only as fallback for inputs the rule's guard rejects, and no rule ships without replay validation against the traces it was mined from.

The smaller patterns that complete this one —

usesShadow Canary★★— Run a candidate agent version in shadow alongside the champion, comparing outputs without affecting users.

And the patterns that stand alongside it, or against it —

complementsBPMN/DMN Deterministic Shell Around Agent★— BPMN (Business Process Model and Notation) processes and DMN (Decision Model and Notation) decision tables form the deterministic spine; LLM-driven agents are invoked only at explicit 'unstructured problem' nodes inside the process.
alternative-toSkill Library★— Let the agent grow its own toolkit by writing reusable skills that subsequent runs can call.
alternative-toProcedural Memory★— Maintain a third agent memory type alongside episodic (past events) and semantic (facts): procedural memory captures *learned how-to* — reusable skills, workflows, and self-rewritten system instructions that map situations directly to actions.
complementsHybrid Symbolic-Neural Routing★— Per query, route between a symbolic path (rule engine, knowledge graph) and a neural path (LLM), using the LLM for interpretation and the symbolic layer for exact constraints.