Per-token pricing has become the default commercial language of generative AI, and practically every budget conversation now happens in those terms (tokens consumed, tokens generated, cost per million tokens, etc., etc.).
Per-token pricing has become the default commercial language of generative AI, and practically every budget conversation now happens in those terms (tokens consumed, tokens generated, cost per million tokens, etc., etc.).
The problem is that this model treats every AI interaction as a discrete transaction, separate from the infrastructure required to process it. Yes, tokens are a billing unit, but the work they represent also has a physical footprint in memory and storage, and as agentic AI drives context windows into the hundreds of thousands of tokens, that footprint has outgrown the GPU.
So why does this matter? Well, according to recent analysis published in Harvard Business Review (HBR), corporate AI is about to experience a massive “cost shock”, as vendors move away from unmetered or bundled AI features and begin passing on the cost of the infrastructure required to deliver them.
As HBR points out, organizations are currently being paid, in effect, to adopt AI, with major software vendors quietly absorbing the steep costs of GPUs, inference and tokens on their customers’ behalf. That subsidy is not built to last.
Take memory and storage costs, for example. DRAM prices have risen sharply as manufacturers focus on High Bandwidth Memory (HBM) production for AI accelerators, tightening supply for conventional server memory. Meanwhile, demand for AI is exerting sustained pressure on the availability and pricing of high-capacity SSDs.
This is totally unsustainable, and there was always going to come a point at which vendors could no longer continue covering these costs themselves and would need to pass more of them on to customers. That point is now rapidly approaching.
In context
- Topic: Inteligencia Artificial — IA aplicada al negocio: casos, límites, costos y gobierno.
- Source: CIO
- Published: 12/10/2026
Continue reading at the original source →
Excerpt published automatically by the site radar. The full text belongs to its publisher and is linked above.
Why it matters
The conversation about artificial intelligence has become noisy, and that makes it harder to see what matters. What matters is rarely the model. It is which concrete process ends up cheaper, faster or more reliable, and who is accountable when the answer comes out wrong.
I measure AI maturity with an uncomfortable question: can anyone explain, in one line, what decision the model makes and who answers if it gets it wrong? Where that is clear, the project moves. Where it is not, it stays in permanent pilot and gets cancelled with the excuse that the technology was not ready.
What usually goes wrong
The point where it usually breaks is the handover to operations. The team that built the pilot knows how to read the results; the team receiving it does not. Without training and written criteria, people start accepting everything the model says or ignoring it entirely, and both are equally expensive.
What to watch
- Who reviews the model outputs and how often — the point at which AI becomes auditable.
- Cost per query at real volume, not pilot volume: the variable that produces the most surprises.
- Where the data ends up and under what contract, especially when customer information is involved.
How I read this entry
My practical advice is to fix on day one what happens when the model is wrong: who reviews, how often, and at what point it gets switched off. It sounds defensive, but it is exactly what lets you be aggressive later, because the risk stopped being an unknown.
This entry is an excerpt from the original source, selected by the site radar. The commentary above is the site's own and does not belong to the cited publisher.
Comments
No comments yet. Yours can be the first.
Living through this in your own team?
Open the chat and tell me how you're handling it. I'm interested in comparing notes.
