Cost Observability

Problem

Without per-feature, per-route, per-model attribution, an aggregate dashboard only shows that total tokens went up. A single bad routing decision, a chatty new prompt, or a runaway loop in one feature can multiply the bill for that feature ten times while the global average barely twitches. The team is forced to choose between learning about the problem from the monthly billing statement or building ad-hoc spreadsheets every time a number looks off.

Solution

Tag every model and tool call with feature, route, user (anonymised), and model id. Stream to a telemetry store. Build dashboards by feature, by model, by tier, by hour. Set alerts on anomalies. Pair with cost-gating for prevention.

When to use

• Per-feature cost visibility is needed before billing reveals a problem.
• Telemetry can be tagged with feature, route, model id, and anonymised user.
• Operators will actually act on dashboards and alerts that surface cost anomalies.