IX · Routing & CompositionEmerging★

Semantic Decision Node

also known as Bounded Semantic Decision, Probabilistic Branch Condition, Smart If, System One Decision Call, Decision Model in the Harness

Place a small decision model at a branch point in harness-owned control flow, have it answer one declared question as a typed probability, and let code apply the threshold and take the branch.

Context

An agent harness is full of narrow judgements that code cannot write as rules: is this request risky, is this tool output an injection attempt, which of four tools fits this step, is the draft on topic. Each one decides which path the run takes next. The control flow around those judgements, meaning the tree, the graph or the state machine, is already explicit code that the team reviews and versions. Since 2026 there are models built only for this job; the San Francisco lab TypeSafe, which came out of stealth in September 2026, ships Jev as a 'System One' model that returns typed choices, scores and yes/no probabilities instead of text.

Problem

When a general-purpose LLM is asked to make these branch decisions, the decision and the branch fuse into one generated answer. The model may name an option outside the allowed set, phrase its answer so that parsing it is itself a judgement, or state high certainty on a coin-flip. The threshold that decides when to act, when to ask a human and when to fall back sits implicitly inside the prompt, where it cannot be reviewed, tuned per model version or audited afterwards. The call also costs frontier-model latency and price for a question whose answer is a single bit, and in a harness that asks dozens of such questions per turn that cost dominates. Writing the conditions as hard-coded rules instead fails on exactly the fuzzy inputs that made a model necessary.

Forces

The branch structure should be explicit, reviewable code, yet some branch conditions are semantic and cannot be written as rules.
A decision is only actionable if its answer is guaranteed to be one of the declared options, not a sentence that has to be interpreted.
A probability is only useful for thresholding if it is calibrated; a confident score that is wrong at the same rate as an unsure one is worse than no score.
Harnesses ask many narrow questions per turn, so each decision has to cost milliseconds and fractions of a cent rather than a frontier-model call.
One question that bundles several criteria is harder to calibrate and debug than several independent questions whose answers code combines.

When to use

The harness control flow is explicit code and some branch conditions need semantic judgement that rules cannot express.
The decision has a closed answer set: yes or no, one of a fixed list, or a position on a declared scale.
Many such decisions happen per turn or per item, so latency and cost per decision matter.
The team needs to review, audit and tune the thresholds that decide when the agent acts, asks or falls back.

When not to use

The step needs generated text, code, explanations or arithmetic rather than a choice among declared options.
The question requires reasoning across many steps or predicting another model's behaviour, where small decision models perform near chance.
No labelled traffic exists to tune thresholds and the decision drives an irreversible action.
A deterministic rule or a lookup answers the question exactly, which is cheaper and fully auditable.

Example

A support agent has to decide whether each incoming message is a billing question, a bug report or something else, and whether it contains a refund demand. Instead of asking the main model in a prompt, the harness asks a small decision model two separate questions and gets back, say, billing at 0.93 and refund demand at 0.41. The code sends the ticket to the billing lane because 0.93 clears the 0.8 threshold, and flags it for a person because 0.41 falls in the uncertain band between 0.3 and 0.7.

Diagram

flowchart TD S["Harness state"] --> Q1{{"Decision model: Is this risky? (yes-probability)"}} Q1 --> T1{"Code threshold"} T1 -- "p >= 0.8" --> H["Human gate"] T1 -- "0.3 < p < 0.8" --> AB["Abstain: fallback or review"] T1 -- "p <= 0.3" --> Q2{{"Decision model: Which tool? (choice + distribution)"}} Q2 --> T2{"Code: top choice confidence >= 0.6?"} T2 -- "DB" --> DB["Database tool"] T2 -- "Web" --> WEB["Web tool"] T2 -- "Code" --> CODE["Code tool"] T2 -- "low confidence" --> AB Q1 -.-> LOG["Decision log: question, answer, p, threshold, model version"] Q2 -.-> LOG

Solution

Therefore:

Declare each branch condition as a question with a fixed answer type: a yes/no probability, a choice from a closed set with a distribution over the options, or a score on declared levels. At the node, send the current state and the question to a decision model whose output is constrained to that type, so an answer outside the schema cannot occur. The model returns the answer, the probability or distribution, and a confidence; it never names the next step. Code holds the thresholds, kept in one reviewed file next to the questions and keyed by the model version that answered, and maps the result onto the branches: act above a high threshold, route to a human or a fallback in the uncertain band, take the other branch below a low one. A compound judgement is split into orthogonal questions answered independently over the same state, so one answer never becomes context for another, and code composes them. Every decision is logged with the question, the answer, the probability, the threshold and the model version, so a wrong branch can be traced to a wrong answer or a wrong threshold. Thresholds are tuned on labelled traffic as coverage-against-accuracy curves rather than set once by intuition.

What it gives you

The control flow stays readable code; only the branch conditions that need judgement are delegated, and each one is a named, testable question.
Typed, schema-constrained answers remove the parsing step and the out-of-set answer as failure classes.
Moving thresholds into code makes the act, ask and fall-back boundaries reviewable, tunable and auditable, and lets them change when the model version changes.
A small dedicated decision model answers in around a hundred milliseconds at a small fraction of frontier cost, so a harness can afford many decisions per turn.
Orthogonal questions composed in code are easier to calibrate and debug than one bundled prompt.

What it costs you

The pattern is only as good as the calibration: a probability that is not calibrated on this traffic turns every threshold into a guess.
A high confidence means the model applied the question as written, including a badly written question, so question design becomes a new source of defects.
Decision models built for local text judgements do poorly on questions that require predicting another model's behaviour or reasoning across many steps, and the harness has no signal that it asked such a question.
Each question and threshold pair is a new artefact to version, test and re-tune on every model upgrade.

What this pattern forbids. The decision model only answers a declared question with a typed value and probability: it must not choose or name the branch, call tools or produce free text, and the threshold that turns its answer into a branch lives only in versioned code.

The smaller patterns that complete this one —

generalisesRouting★★— Classify an incoming request and dispatch it to the specialist (lane / agent / model) best suited to handle it.
usesStructured Output★★— Constrain the model's output to conform to a JSON Schema (or similar typed shape).

And the patterns that stand alongside it, or against it —

complementsLLM as Periphery·— Invert the typical LLM-in-the-middle architecture: a deterministic state machine and event store form the core; the LLM is restricted to edge tasks — input interpretation and output synthesis only.
composes-withAgentic Behavior Tree·— Borrow the behavior-tree formalism: leaves are LLM calls or tools that return success/failure; a tree of selectors and sequences orchestrates control flow.
complementsStochastic-Deterministic Boundary (SDB)★— Formalize the seam between an LLM proposal and a system action as a four-part contract — proposer, verifier, commit step, reject signal — so the contract itself, not the agent's good intent, gates side-effects.
complementsCalibrated Help-Gate via Conformal Prediction·— Use conformal prediction to form a calibrated set of candidate actions and have the agent ask a human for help only when that set is not a singleton, giving a statistical task-completion guarantee.
complementsComplexity-Based Routing★— Estimate a request's difficulty up front and bind it to the cheapest model tier that can answer well, using an explicit complexity classifier as the routing key.
complementsHybrid Symbolic-Neural Routing★— Per query, route between a symbolic path (rule engine, knowledge graph) and a neural path (LLM), using the LLM for interpretation and the symbolic layer for exact constraints.
complementsCost-Aware Action Delegation★— Classify every agent action by risk/cost and route each tier to a different approval policy, bounding the autonomy surface per-action instead of by one global flag.
complementsConfidence Reporting★— Surface the agent's uncertainty about its answer alongside the answer itself.
complementsFalse Confidence Syndrome✕— Anti-pattern: the model produces incorrect answers with the same high confidence as correct ones, failing to vary its expressed certainty with its actual reliability — Oxford-documented for constraint-heavy prompts.
complementsHuman-in-the-Loop★★— Require explicit human approval at defined points before the agent performs an action.