What happens when token costs decline to zero?

We're rapidly heading to a world where using LLMs is almost free

My mental model for the ecosystem of large language models (LLMs) is that there are three distinct groups of participants:

• LLM builders: Google, OpenAI, Anthropic, etc.

• Startups sitting on top of LLM substrates: Jasper, Harvey, etc.

• End users: These people may access the LLMs directly via their API, or via ChatGPT or Claude or use one of the startups building on top of LLMs.

Token costs are trending towards zero, and this creates some interesting dynamics for these groups of users.

The trend is your friend. Chart sourced from here.

What follows is a breakdown of the likely impact of zero cost tokens on these different groups.

1. LLM Builders

   1. Intense Cost Competition and Consolidation: Foundational model builders (OpenAI, Google DeepMind, Anthropic, etc.) will face intense pressure to lower their own operational costs, especially compute expenses. The competitive landscape will consolidate around a few players who can sustain the massive capex investments needed to offer zero-cost tokens while still innovating on model quality.

   2. Shift in Monetization Models: These companies will explore alternatives to per-token pricing. For example, they could implement tiered access pricing, in which prices are based on capabilities. Other possibilities include custom fine-tuning services, or partnerships with industries that need high levels of specialization. Additionally, they could monetize through vertically integrated solutions, or by licensing specific model capabilities rather than tokens.

   3. Greater R&D Investment: R&D investments may pivot towards maximizing the utility and efficiency of the models themselves. This would accelerate advancements in parameter efficiency, model distillation, and the development of self-optimizing training algorithms. All of this could enhance foundational models’ performance without increasing infrastructure needs.

2. Startups Building on LLM Substrates

   1. Focus on Differentiation: Startups building products on top of LLMs will need to differentiate on features, UI/UX, and integration with workflows, rather than the cost of LLM usage itself. As token costs are no longer a barrier to entry, competition will intensify, and successful products will likely need to offer unique functionality, reliability, and seamless integration rather than simply riding on token-based cost advantages.

   2. Pressure on Business Models: Traditional SaaS business models that incorporate usage-based pricing on top of LLMs will face new challenges. Startups will need to pivot towards value-based pricing, or offer premium layers such as proprietary fine-tuning, enterprise-grade security, or workflow customization as a basis for charging. Products that combine LLM capabilities with domain-specific data and workflow insights could command premium prices despite low token costs.

   3. Higher Standards for Privacy and Compliance: As token costs trend to zero, more industries will start to look at AI-powered solutions to optimize their operations. Many industries, such as healthcare and finance, are highly regulated, which will require startups building products for these industries to prioritize features like privacy, regulatory compliance, and on-premises solutions.

3. End Users

   1. Broad Accessibility: A zero token cost world will make high-quality language models broadly accessible. Small businesses, individuals, and hobbyists will integrate advanced LLMs into their workflows. This will expand AI’s presence across sectors, from education to local government, to virtually any other sector you can conceive.

   2. Increased Personalization and Customization: As token costs become negligible, more users will use fine-tuning to allow for greater customization. Users could develop bespoke AI assistants tuned to their workflows, preferences and domains. The rise of AI agents will only accelerate this trend.

   3. Lower Barriers for Innovation: Zero cost tokens will spur grassroots innovation. Users will experiment with AI capabilities without financial constraint. This might lead to unforeseen applications in niche areas—creative arts, non-profit sectors, or scientific research—where token costs were previously a barrier.