The sticker price of a token has never been lower, and it keeps falling. Stanford’s AI Index found that the inference cost to reach GPT-3.5-level performance dropped more than 280-fold between November 2022 and October 2024 . If unit economics were the whole story, every AI program in the enterprise would be printing margin by now. They are not. The gap between the falling price of a token and the rising cost of an AI program is where the real bill lives, and most organizations have no instrument pointed at it.

The bill is not the invoice

Ask a platform leader what their AI spend is, and you will get a provider invoice. That number is real, but it is the smallest part of the total. The full cost of a token includes the prompt scaffolding you send around it, the retries, the agent loops that fan out into dozens of calls per task, the context you re-send on every turn, and the human time spent reconciling all of it against a business outcome that may or may not exist.

Finance has noticed. In the FinOps Foundation’s latest survey, 98% of 1,192 respondents now manage AI spend, up from 31% just two years earlier, and “FinOps for AI” is the top forward-looking priority . In two years, AI went from a rounding error someone in engineering handled to the single line item finance most wants to control. The demand is there. The instrumentation, in most shops, is not.

Cheap tokens, expensive programs

The paradox is that adoption is nearly total while returns are not. McKinsey’s 2025 survey puts AI use at 88% of organizations, yet only about 39% report any enterprise-level EBIT impact, most of them below 5% . Near-universal usage, thin measurable value. That is what a cost problem masquerading as an adoption success looks like.

The consequences show up downstream as cancellations. Gartner expects at least 30% of generative AI projects to be abandoned after proof of concept by end of 2025 , citing escalating costs among the drivers, and over 40% of agentic AI projects to be canceled by end of 2027 for much the same reasons. Agents make this worse, not better: an agentic workflow multiplies token consumption per unit of work, and it does so autonomously, at a rate no quarterly budget review can catch in time.

What you can’t see, you can’t govern

Per-token pricing invites a specific mistake: treating cost as a procurement question. Negotiate the rate, pick the cheaper model, move on. But the token is not where the leverage is. The leverage is in the traffic: what gets sent, how often, how large, and to whom.

Most enterprises route AI traffic straight from application to provider with nothing in between that understands it. That means:

  • No per-team, per-application, or per-workflow attribution of spend to a business owner
  • No visibility into how much of each request is redundant scaffolding versus signal
  • No enforcement point to cap runaway agent loops before the invoice arrives
  • No record of what data left the building inside those prompts

The last point is the one that turns a cost problem into a risk problem. Every token you send is also data egress. Prompts carry source code, customer records, and internal documents to a third party by design. Without a measurement point at the boundary, sensitive-data exfiltration and cost overrun are the same blind spot viewed from two angles.

The waste is measurable, and removable

The encouraging part is that much of the spend is genuinely waste, not value. Peer-reviewed work on prompt compression shows redundancy on a scale that is hard to ignore: research has demonstrated up to 20x prompt compression with little performance loss , and long-context methods have reported around 4x fewer tokens and up to a 94.0% cost reduction on the LooGLE benchmark . You are, in many cases, paying to send the model context it does not need to produce the same answer.

Capturing that saving is not a model-selection decision or a per-team code change. It is a property of the layer your AI traffic passes through. The right question is not “which provider is cheapest per token” but “what is inspecting, attributing, and shaping the traffic before it becomes a bill.”

What good looks like

Treat the AI traffic boundary the way you already treat network and API boundaries: as a control plane, not a passthrough. A transparent firewall at that layer (sitting between your agents and applications and whatever model providers they call) gives you the one thing per-token pricing never will: an accounting of what you actually send and why.

Concretely, that means:

  • Attribution: spend tied to a team, app, and workflow, not a single blended invoice
  • Reduction: redundant context removed before it is billed, without touching application code
  • Containment: enforceable limits on runaway loops and oversized requests
  • Exposure: a measured, auditable view of what sensitive data leaves in your prompts

None of this requires switching providers or rewriting your stack. It requires a boundary that understands AI traffic and a policy you own, applied vendor-neutrally across every model you use.

The price per token will keep falling. That was never the number that decided whether your AI program pays for itself. The number that matters is the one you can only see if something is watching the traffic, which is exactly the layer Milgram is built to be.