Most FinOps practice was built for a world of VMs, storage, and network egress, resources that scale roughly in proportion to how many you provision. AI spend doesn't follow that pattern, and teams that try to manage it with the same playbook usually end up surprised by the bill.
The old FinOps playbook assumed infrastructure, not inference
Provision a VM, pay for the VM. Scale up, cost scales up proportionally, and roughly predictably. That relationship is what most cost dashboards, budgets, and forecasting models are built around.
AI inference breaks that assumption. Two teams can run the exact same infrastructure footprint and see wildly different bills, because the cost isn't driven by what's provisioned, it's driven by usage: how many tokens flow through the model, how long the output runs, which model got called. A single prompt change can move the bill more than a full infrastructure migration would.
Three billing models, three different mental models
The major AI platforms don't even bill the same way, which is part of why this is harder than ordinary cloud cost management:
- Azure OpenAI offers pay-as-you-go, per-token pricing with separate input and output rates, alongside Provisioned Throughput Units (PTUs), reserved capacity billed hourly regardless of utilization, similar in spirit to a reserved VM instance.
- AWS Bedrock defaults to serverless, on-demand per-token pricing across its multi-model marketplace (Anthropic, Meta, Mistral, Cohere, Amazon's own models), with optional provisioned throughput for predictable high-volume workloads at shorter commitment terms than Azure's PTUs.
- Google Vertex AI uses tiered token pricing with context-caching discounts, plus Generative AI Scale Units for provisioned capacity, its own distinct unit entirely.
Three platforms, three different names for provisioned capacity, three different discount structures. A team running all three, which is increasingly the norm rather than the exception, ends up needing three separate mental models just to read the bill.
Provisioned capacity works differently for AI than for compute
With a reserved VM instance, the sizing question is fairly mechanical: how many VMs, what size, for how long. With provisioned AI throughput, the sizing question is about sustained token volume and utilization rate, and getting it wrong is expensive in both directions. Under-provision and requests queue or fail. Over-provision and the reserved capacity bills hourly whether it's used or not, the same trap as an underused VM reservation, just with a steeper cost curve if it's misjudged.
The break-even point between pay-as-you-go and provisioned throughput depends heavily on sustained utilization, not just total volume. A team with high total token usage but spiky, uneven traffic can still lose money on a provisioned commitment sized for their average.
The metric that matters shifts from "cost per resource" to "cost per token"
Traditional infrastructure cost management centers on cost per resource: cost per VM, per GB stored, per instance-hour. None of that translates cleanly to AI spend. The equivalent unit is cost per token, but even that's not the end of the analysis, since input and output tokens are usually priced differently, and the real question is rarely "how many tokens" but "how much did it cost to produce the outcome those tokens were spent on."
That's the same unit economics idea covered in more depth in our piece on AI unit economics, tokens are the input, but the metric worth tracking is cost against a real business outcome.
Multi-provider AI spend is the norm, not the exception
Very few organizations end up on a single AI platform by design. More often, different teams pick different platforms for different workloads, one team standardizes on Azure OpenAI because the rest of the stack is Microsoft, another reaches for Bedrock because it needs a specific open-weight model, and finance ends up holding three incompatible bills with no easy way to compare them side by side. This mirrors the exact problem multi-cloud infrastructure spend has had for years, just newer and less mature.
What doesn't change
Not everything is different. The core FinOps discipline, visibility before optimization, ownership close to the spend, continuous iteration rather than a one-time cleanup, still applies exactly as it does to infrastructure. What changes is the shape of the data underneath it: usage-driven instead of resource-driven, multi-schema instead of one bill, and moving fast enough that a monthly review cycle is often too slow to catch a real problem before it compounds.
Where to start
- Get every AI platform's spend into one place before trying to optimize any of them individually. You can't compare what you can't see side by side.
- Identify which workloads are sustained versus bursty. That distinction drives the provisioned-versus-pay-as-you-go decision on every platform.
- Pick one real-world unit per AI feature and start tracking cost against it, not against raw token counts.
- Review weekly, not monthly. AI spend moves faster than infrastructure spend did.
This is the exact gap CloudNomix's AI cost intelligence view is built for: token volume, blended rate, and cost per outcome, tracked across providers in the same dashboard as the rest of your cloud spend, not a separate tool for every AI platform.