By Zhang Jinyi
For decades, the fundamental units of the global economy were barrels of oil, kilowatt-hours of electricity and man-hours of labour. But in the corridors of Silicon Valley and the high-tech hubs of China, a new metric is beginning to rival them: the token.
Once an obscure technical term for how large language models process text, the token is undergoing a metamorphosis. To delve deeper into the significance of tokens in the AI era, we invited Dai Guohao, associate professor at Shanghai Jiao Tong University and co-founder of Infinigence-AI, a startup that aims to cut the cost of deploying AI models, for an in-depth conversation. Dai was selected as one of MIT Technology Review’s “35 Innovators Under 35 of China” in 2024.
He argues that tokens are evolving from a measure of computing cost into a proxy for productivity, value and even labour. Recent examples illustrate the shift. A lawyer in Manhattan reportedly used an AI model to complete an M&A proposal in two hours, a task that would typically require a legal team working through the night. A pipeline engineer with no coding experience used AI to diagnose complex network failures in minutes, replacing days of manual inspection.
Such cases highlight a narrowing productivity gap between large teams and individuals. The “one-person company” is no longer a theoretical ambition but a functional reality, Dai argues. Yet this efficiency comes at a cost. For heavy users of autonomous agents, monthly bills can run from thousands to tens of thousands of dollars.
At the center of the value chain
Tokens, once a purely technical unit used to price model usage or measure inference cost, are becoming an economic variable. The reason, Dai argues, is that AI models have crossed a threshold into practical usefulness. Earlier systems were largely conversational and difficult to integrate into production workflows. Today’s models can complete real tasks, meaning that token consumption increasingly maps directly onto output.
Tokens now sit at the centre of a value chain: electricity powers hardware; hardware generates compute; compute produces tokens; and tokens, through models, generate productivity that can be translated into economic value. As the efficiency of converting tokens into useful output rises, so too does the economic significance of each token.
This helps explain an apparent paradox. Although improvements in hardware and system design are reducing the cost of generating tokens, some providers are raising prices, as advances in model capability increase the value generated per token. As AI moves from a subsidised, growth-oriented phase to a market-driven one, tokens are increasingly priced according to the value they create.
The implications extend beyond pricing. Jensen Huang, Nvidia’s chief executive, has described data centres as “AI factories” that produce tokens, and has suggested that token consumption could become a measure of employee productivity, akin to wages or bonuses. But Dai cautions that this risks misunderstanding the metric. What matters is not how many tokens are consumed, but how effectively they are used.
Higher quality, higher prices
That, in turn, shifts the focus of skills. Effective use of AI depends on structuring tasks, designing prompts and selecting appropriate models. At the same time, engineers are finding ways to make AI systems more efficient. Instead of treating every token the same, newer approaches allow models to spend more effort on complex parts of a task while simplifying or skipping easier ones. The aim is to reduce costs without sacrificing performance.
As tokens become more central, pricing structures are also evolving. One emerging approach is tiered pricing based on model capability, generation speed and context length. Higher-quality tokens — those produced by more advanced models — command higher prices, reflecting their greater value in certain applications.
Looking ahead, entirely new computing paradigms may emerge, such as continuous or even quantum computing, Dai says. If successful, they could eventually replace today’s token-based systems. But such a shift would require a dramatic leap in efficiency — large enough to justify rebuilding the entire technological ecosystem.
For now, however, such a transformation remains distant. The AI era is still centred on tokens. The challenge is to improve how efficiently they are converted into value — and to build the infrastructure and applications needed to support that process.
Source:
DeepTech
