Eval & Observability
Type: recipe · Status in practice: mature
Description. How you keep an agent honest in production: harness, judge, decision log, provenance, shadow rollouts.
Patterns this recipe composes —
- ★★Eval Harness
- ★★Eval as Contract
- ★★Decision Log
- ★Agent-as-a-Judge
- ★★LLM-as-Judge
- ★Confidence Reporting
- ★★Provenance Ledger
- ★★Lineage Tracking
- ★Production Failure Triage Loop
Closes the observability loop: consumes the failures that eval and tracing surface and routes each class to its remediation path, making the feedback loop measurable rather than ad hoc.
- ★★Shadow Canary
Neighbourhood
Click any neighbour to follow the lineage. Scroll to zoom, drag to pan.
Instantiated by (2)
Provenance
- Last updated: