AI Cost-per-Inference Governance

The Problem

The problem

The Detection

How you know it is happening

If the answer to the question, "what was the cost-per-inference of the customer-support agent yesterday, by model variant, at the customer cohort level," is anything other than a number rendered in seconds, the institution is below Best on this capability.

The Outcome

What success looks like

A per-call cost ledger that tags every inference with model, product, customer cohort, and agent identity. A live dashboard of cost-per-inference by domain. A budget guardrail policy that fires at the agent level, before runaway prompts compound into runaway invoices.

Ship It

The steps, in order.

1. I

   Step 01

   Inventory every inference surface

   List every product, agent, batch job, and ad-hoc surface that calls a hosted model. Capture the provider, the model identifier, the responsible team, and the current monthly invoice line. Treat any surface without a named owner as a control finding.

2. II

   Step 02

   Provision per-surface API credentials

   Issue a distinct API key, project, or service account per inference surface. Rotate any shared credentials. The aim is that every billable call can be traced to a single owner without log-side reconstruction.

   bashCopy

   gcloud ai-platform projects create inference-customer-support \
     --organization 0123456789 \
     --billing-account 01ABCD-234EFG-567HIJ

3. III

   Step 03

   Attach the UFMS inference tag schema

   Adopt the UFMS:003:2026 inference annex. Every inference call must carry: model_id, product_id, customer_cohort, agent_id, and intent. Bake the tag injection into the SDK wrapper used by every service. Reject calls that omit any required tag at the wrapper level.

4. IV

   Step 04

   Stream the per-call ledger to BigQuery

   Pipe every inference call (cost, tokens, latency, tags, model variant) to a BigQuery table partitioned by day. Avoid sampling. The ledger is the institutional source of truth and must be complete enough to defend against an auditor or a regulator.

5. V

   Step 05

   Stand up the cost-per-inference dashboard

   Render cost-per-inference by domain, product, cohort, and model variant. Layer in carbon attribution from the GreenOps annex. The dashboard is the single page the AI/ML lead opens every morning.

6. VI

   Step 06

   Wire up agent-level budget guardrails

   For every agent, declare a daily and monthly budget in code. The wrapper SDK halts further calls and pages the owner once the threshold is breached. Treat the guardrail itself as a controlled object with a documented exception path.

   jsonCopy

   {
     "agent_id": "support-l1",
     "daily_budget_usd": 250,
     "monthly_budget_usd": 6500,
     "on_breach": "halt-and-page",
     "owner": "ai-ml-lead@example.com"
   }

7. VII

   Step 07

   Review weekly with the AI/ML lead and FinOps lead

   Schedule a thirty-minute weekly review of the top-five most expensive agents, the top-five fastest-growing agents, and the top-five lowest-utilised agents. Each review produces a documented decision: optimise, cap, or sunset.

The Templates

Templates and starter code

The Evidence

Prove it.

• I

  Inventory of inference surfaces

  Markdown or CSV inventory listing every inference surface, owner, model, and current monthly invoice line.

  Sigstore
• II

  Inference ledger schema

  SQL DDL for the BigQuery ledger, plus a sample seven-day export to demonstrate completeness.

  Sigstore
• III

  Agent budget policy file

  YAML policy committed to a controlled repository with named approvers and a documented exception path.

  Sigstore
• IV

  Weekly review minutes

  Four consecutive weekly review notes documenting decisions taken on the top inference surfaces.

  Recorded

The Impact

What this earns you.

Pair this with

Related playbooks.