APAC enterprises are burning through more AI tokens than ever, even as per-token prices have dropped sharply since 2023. The result is a rising cost curve that few organisations know how to flatten - and an uncomfortable question about what value they are getting for the money.
"If you look at everyone's token consumption, ours included and all of our clients included, it looks like a line that goes bottom left to top right," said Shayan Mohanty, chief data and AI officer at Thoughtworks. "Everyone's slope is different, but everyone is on the upswing, and it is unclear if anyone is ever going to achieve a steady state."
Input tokens are now 90% cheaper than in 2023 and output tokens 75% cheaper, according to Anshuman Sengar, digital and analytics practice lead for APAC at management consultancy Kearney. But the number of tokens used per request has jumped 184% over the same period, because newer models reason more and produce longer responses. The money is flowing somewhere: Anthropic's revenue run rate had grown more than sevenfold since the end of 2025, Bloomberg reported last month.
"If Anthropic is making so much money, someone is paying for it, so the question is who is paying for it and what value are they getting?" said Sengar.
Renting GPUs as a cost-control lever
Mohanty argues that pay-per-token pricing creates a structural problem - the bill rises in lockstep with usage, and he does not see usage levelling off. His advice is for organisations to rent their own accelerated compute and run open-weight models on it. This converts a variable cost into a fixed infrastructure expense.
"In a world where you rent your GPUs, you basically say, 'I'm going to allocate this much capital and I'm going to rent this infrastructure,'" he said. "Now it's predictable and it becomes an engineering problem around how you eke out as many tokens per second as you possibly can from the same infrastructure."
He also tells clients to rent as close to home as possible, for latency and sovereignty reasons. "If you're completely beholden to the West or to China for all of your tokens, inevitably you are allowing another entity to own possibly one of the most consequential decisions in your company's lifecycle, which is: where are you getting your intelligence from?"
Mohanty does not recommend buying hardware unless an organisation already runs datacentres with the capital expenditure processes and depreciation schedules that go with it. Everyone else should rent from the partner offering the best price. Neocloud providers - datacentres housing nothing but GPUs - can offer lower unit costs than hyperscalers whose facilities must accommodate every kind of hardware.
Sengar offered comparative figures: a million requests a month would cost $5,000 for a cloud-hosted model on AWS Bedrock, $10,400 for a self-hosted open-weight model, and $20,300 for API calls to a major provider's model. The more volume, the more viable self-hosting becomes.
Why user training won't fix the bill
Sengar recommends training users on which model suits which task and getting them to make more precise requests. Mohanty does not disagree but considers it a weak economic lever.
"It's just harder to get someone to change behaviours than it is to apply a new technical architecture," he said. "The trivially correct answer is yes, absolutely, people should be better trained and using the right tools at the right time with the right models. But that will never be the biggest economic lever for getting your arms wrapped around cost control."
There is a pricing reason for that. Input tokens cost a fraction of output tokens, so a tidier prompt trims the cheap end of the bill and does nothing to limit how much the model produces in reply. Some companies are setting token quotas, though Sengar noted that in India, compute shortages - and moves toward sovereign supply - are already pushing corporates toward rationing.
Thoughtworks takes a different approach: it gates access to models rather than rationing usage. "An uncalibrated user will always tend to use the most expensive model they have access to," Mohanty said. "Their mentality is, why would I ever use something that's worse than the best thing I can get?" Internally, the company runs an open-weight model on infrastructure with a hard hourly cost. Frontier models are available but placed in a slower lane - users can still get tokens, but they might have to wait.
Caching, routing and the stampeding herd
Sengar pointed to caching and local memory as a way to cut costs on repetitive tasks - an HR team checking CVs against the same five criteria, or a loan review running through a dozen standards each time. Neither requires a new call to a large language model for every case.
Mohanty described a problem he calls the stampeding herd. When multiple developers use agentic coding tools on the same repository without coordination, they spin up sub-agents simultaneously and send near-identical prompt prefixes to the provider. The provider's distributed caching layer turns many of those requests into cache writes, which cost more than ordinary input. Out of 10 requests, perhaps six become cache writes and four cache reads.
"If you're intelligent about it, before it ever leaves your infrastructure, you can see that I sent 10 of the same thing," he said. "I'm going to wait for a second, I'm going to buffer them, I'm going to elect one as the cache writer. That way I can guarantee the other nine are cache reads."
He was sceptical about model routing, noting that classifier-based routers need retraining whenever a model is added or removed. Newer approaches try to determine what capabilities a request needs and then find the cheapest model that has them, but that pushes the problem onto model benchmarks that are themselves contested. A routing decision never reveals the answer the other model would have given, so feedback loops are weak.
What nobody is measuring
Mohanty said he has seen only a handful of organisations get proper visibility of AI spending across multiple providers and clouds. "You want a single pane of glass, but it's near impossible to get it if you are across Google, AWS and whoever else you're using."
Sengar sees the same gap. In most organisations, nothing tells them which LLM requests are doing useful work and which are someone planning their holiday. His advice: determine who is using LLMs and for what, then incorporate cost figures into the dashboards and governance processes people already look at. He cited Canva as a cautionary example - the company built agentic AI into its products without realising how many tokens they would consume, and the surprise cost hit its gross margin.
Beneath all of this is a deeper question Mohanty thinks few organisations are equipped to answer: who defines a good answer? "Let's say I gave you two different outputs from two different models for the same input," he said. "You may have a preference of one versus the other, but is that truly arbitrating ground truth, or is that just your personal preference? Most organisations believe it's a technical problem, which to a certain degree it is, but actually it is deeply philosophical."
Thoughtworks does not benchmark client costs against peers because Mohanty believes cost is a poor comparative measure. "I can drive my costs up, no problem. That doesn't mean that I'm ahead or behind my peers." Tokens also are not interchangeable between models, so a company spending less may simply be running a smarter mix. "There are no tech moats any more - the only thing that matters is execution moats and adapting faster than everyone else."
Why this matters for management
The organisations that get AI cost management right are the ones that stop treating it as a cost problem. Mohanty described one company that worked out which KPIs mattered before consuming AI in volume, then set a rough equation for how much it would pay for a given improvement in those numbers. "Then at that point, we're no longer talking about costs. We're talking about profitability." Most companies, he said, are still measuring AI in terms of headcount and people's time - the least interesting approach. For AI for Management professionals, the practical takeaway is to shift the conversation from invoice size to unit economics: know what a good answer costs and what it is worth before scaling consumption. Without that equation, rising token bills will remain a source of surprise rather than a managed input to margin.
Your membership also unlocks:
