Why cheaper AI models can still produce bigger bills

When health AI company Heidi reconsidered its model stack, the numbers were large enough to make the choice consequential. Co-founder Yu Liu told Forbes that Heidi had been spending as much as US$3 million a month on models from OpenAI, Anthropic and Google. He said the company later moved 95 per cent of its workload to specialised open-weight models through Fireworks and cut its monthly AI bill by almost 90 per cent.

Those financial figures have not been independently audited. Fireworks, a commercial participant in the project, separately documented the partnership and said latency fell from 25 seconds to seven seconds. It also said the proof of concept reached production in four weeks, while noting that model quality was assessed through Heidi’s internal comparisons.

The case is not evidence of a collapse in corporate enthusiasm for AI. It illustrates a more prosaic and important transition: AI is becoming a variable operating expense that organisations must attribute, evaluate and govern. The central challenge is no longer simply gaining access to a capable model. It is deciding which model deserves each task, what the complete service costs and whether the resulting work is good enough to justify the bill.

Cheaper tokens, larger bills

At first glance, concern about AI costs appears to conflict with one of the industry’s clearest trends. Stanford’s 2025 AI Index found that the price of querying a model at roughly GPT-3.5 performance on the MMLU benchmark fell from US$20 per million tokens in November 2022 to US$0.07 by October 2024, a reduction of more than 280 times.

Yet a falling unit price does not determine total expenditure. Consumption can expand as more teams adopt AI and agentic systems generate additional model use. Costs also extend beyond the model call. FinOps Foundation guidance includes infrastructure, orchestration, observability, software, labour and process costs in the total AI bill.

Adoption provides the scale behind that distinction. Stanford’s 2026 AI Index reported that organisational AI adoption reached 88 per cent in 2025, while 70 per cent of surveyed organisations used generative AI in at least one business function. Agent deployment remained in single digits across nearly all functions, leaving mass adoption and extensive agent use at different stages.

Epoch AI offered a narrower illustration of expanding consumption in its October 2026 analysis of OpenAI coding-agent use. It found that usage valued at public API list prices rose rapidly through mid-August. The data covered researchers at one company, and Epoch explicitly cautioned that the figures did not represent OpenAI’s actual internal expenditure. The example therefore cannot establish a universal cost curve, but it shows how quickly usage can grow independently of the published price for each unit.

The same issue has entered the work of cloud cost specialists. The FinOps Foundation’s 2026 State of FinOps survey drew 1,192 respondents representing more than US$83 billion in annual cloud spending. It reported that 98 per cent managed AI expenditure and identified AI cost management as the leading skill set teams needed to develop. The respondents were a self-selected practitioner community, not a representative sample of every organisation, but their priorities show that AI accounting has become an operational discipline rather than a speculative concern.

The wrong number can look reassuring

A low token price is easy to compare, but it can conceal the factors that determine whether an application creates value. FinOps Foundation guidance instead emphasises use-case economics: the total cost of achieving a business outcome. That approach changes both the unit of measurement and the procurement question.

For a production workload, the useful comparison is not merely the price of sending the same volume to two models. It also includes whether each model completes the task to the required standard, how reliably it does so, how much human review remains necessary and what integration and operating work surrounds it. Latency matters where a slow response reduces the usefulness of a service. Migration work matters when an apparently inexpensive model requires extensive engineering or evaluation.

This helps explain why the most capable and expensive model can remain economical for difficult or valuable work. A premium price is not automatically wasteful when production evaluation shows a meaningful improvement in successful outcomes. Conversely, using a leading model for every routine task can impose a premium without producing a corresponding benefit.

The 2026 AI Index found that leading model performance had converged enough to move competition towards cost, reliability and domain-specific performance. As of March 2026, it reported a 3.3 per cent gap between the top closed and top open model. That result does not mean all models are interchangeable. Benchmark results are specific to their tests, while production quality depends on the task and the organisation’s evaluation method.

Routing does not remove the work

A mixed-model stack offers one response. Organisations can match workloads to different models, reserving a higher-cost option for tasks where its performance earns the premium and directing other work to cheaper or specialised systems. Heidi’s reported migration presents a prominent example, but its precise savings remain the company’s attributed account rather than an independently verified total-cost comparison.

The distinction between open-weight and open source also matters. Access to model weights does not by itself establish that a system meets a recognised open-source definition. Nor does changing the model eliminate expenditure. Open-weight and self-hosted deployments can move costs into infrastructure, engineering, evaluation and governance. The available research does not provide a like-for-like total-cost comparison covering all those categories against managed frontier model services.

Routing adds another layer that requires oversight. Quality must be tested as models and services change. Expenditure must be assigned to the team, application, user, service account or agent that generated it. Providers use inconsistent pricing structures and meters, while service variants change quickly, making a static annual estimate a weak substitute for continuing measurement.

FinOps Foundation guidance recommends regular cost and usage reviews, quotas, tagging and alignment with business outcomes. Together, those controls answer different questions. Tagging helps identify ownership. Quotas constrain consumption. Reviews test whether usage still matches the purpose approved for it. Outcome measures show whether a lower bill represents genuine efficiency or simply a decline in quality.

Cost visibility is also a governance question

Attribution has implications beyond finance. Knowing which identity, application or agent produced usage can support accountability and auditability, although cost telemetry does not automatically improve security. Monitoring systems can themselves collect sensitive operational information, so the telemetry requires appropriate control rather than indiscriminate retention.

Stanford’s 2026 AI Index found that information security was the business function most often given primary responsibility for AI governance, at 21 per cent. It also found that disclosure of responsible AI benchmarks remained less consistent than disclosure of capability benchmarks. That imbalance complicates procurement: buyers may have more information about what a model can do than about how responsibly its operation can be assessed.

Data handling therefore cannot be inferred from a model’s price, origin or label. Forbes reported claims from Fireworks president George Hu and OpenRouter co-founder Alex Atallah that workloads using Chinese-developed model weights could keep US customer data in non-Chinese or US infrastructure. Those arrangements were not independently established. Architecture, contracts, subprocessors and audit evidence would be needed to substantiate any particular data-residency or privacy claim.

The discipline emerging around AI expenditure is consequently broader than bargain hunting. Organisations need to know what they are paying for, who generated the cost, whether the output succeeded and what supporting systems were required. They also need to preserve reliability and governance when they change suppliers or models.

Rapid price declines may ease some pressure, but they do not make those questions temporary. As AI moves from experiments into recurring operations, the durable unit of accountability is not the cheapest token. It is the successful, governable outcome and the complete cost of producing it.

Cost telemetry sits at the junction of financial control and the careful handling of operational information. Let that connection guide a broader technical next step with The Hack Academy’s online cybersecurity training programme, where cybersecurity fundamentals and cloud security are listed subjects and self-paced learning combines theory with practical exercises.