CMS: Production AI Observability
warningcms_production_observability
Production AI systems implement observability — traces, evaluations (EVALs), and key metrics tracked over time.
Formula
G((Deploy ∨ Output) → P(observability_enabled)) Why it matters
CMS Monitoring & Observability: track traces, EVALs, prompt management/versioning and key metrics for production AI systems, aligned with OMB M-25-21/M-25-22.
Examples
passes the CMS AI rule's evidence is present
fails the required control is absent
Use it
ponens policies add cms_production_observability --into ./trace.json
ponens trace check ./trace.json