XI · Structure & DataEmerging★

DSPy Signatures

also known as Prompt Programs, Compiled Prompts

Specify agent behaviour as declarative typed signatures and modules; compile prompts and few-shot examples automatically against a metric.

Context

A team is building an agent pipeline made of several language-model calls — retrieve a passage, summarise it, answer a question against it, check the answer — and wants the system to behave reliably across model upgrades without rewriting each prompt by hand every time. They are using DSPy, a framework from Stanford that lets the team describe each step as a typed input/output specification and then compiles the actual prompt strings and few-shot examples from those specifications. The compilation is driven by a metric the team cares about, the way an optimising compiler is driven by a benchmark.

Problem

When prompts are hand-written strings glued into application code, they drift over time and break in ways that are expensive to track down. A wording change that helps one model hurts another; small edits to phrasing change behaviour without anyone noticing; every pipeline reinvents the same prompt-engineering loop with no shared discipline. Without a way to express what each step expects and produces in a structured form, the team has no compiler to lean on and no metric-driven way to know whether a prompt change is an improvement or a regression.

Forces

  • Declarative coverage vs signature expressivity ceiling.
  • Compile-time optimization vs metric/data availability.
  • Portability vs per-model compilation gains.

Example

A team has six prompts across their pipeline and every model upgrade means rewriting all of them by hand against a vague vibes-test. They migrate to DSPy Signatures: each step is declared as a typed input/output module — for example summarise(article: str) -> Summary — and a compiler generates prompts and few-shot examples automatically against a metric they care about. When they swap models, the compiler re-optimises the prompts; the team stops hand-tuning strings.

Diagram

Solution

Therefore:

Define each step as a typed signature (input fields → output fields). Compose signatures into modules. Run a teleprompter (optimizer) that generates few-shot examples and refines instructions against a held-out metric. The compiled artefact replaces hand-tuned prompts.

What this pattern forbids. Module behaviour is constrained by its declared signature; ad-hoc string manipulation is replaced by typed input/output fields.

The smaller patterns that complete this one —

  • usesStructured Output★★— Constrain the model's output to conform to a JSON Schema (or similar typed shape).
  • usesEval Harness★★— Run a held-out dataset against agent versions to detect regressions and measure improvement.

And the patterns that stand alongside it, or against it —

  • complementsAgent Skills★— Package author-time procedures (markdown + optional resources) the agent loads on demand for specific task types.
  • alternative-toPrompt/Response Optimiser★★— At runtime, transform user inputs and model outputs into standardised, template-aligned prompts and responses against predefined constraints, so the agent and its downstream consumers see consistent shapes.
  • alternative-toAgentic Context Engineering Playbook·— Treat the agent's system prompt and long-lived memory as a structured, item-addressable playbook that evolves through small delta updates from a Generator/Reflector/Curator loop, so accumulated tactics resist the context collapse that monolithic rewrites cause.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

Used in recipes

Used in frameworks

References

Provenance