Where is the moat in AI?
All the major models compete on price but don't offer much differentiation. That's great for consumers, but bad for the companies that run these models
The weird thing about large language models (LLMs) is that, as amazing as the technology is, they seem to be pretty crappy businesses. The various LLM providers all compete on price, with little to differentiate one model from another. Sure, OpenAI’s GPT4 is more powerful than any of the others, but Google claims that its as yet unreleased1 Gemini Ultra is as powerful. And there is speculation that OpenAI will soon release GPT5. But once you have enough cash, you can buy (or rent) the infrastructure required to train large language models. There is no moat in LLMs. Further, with the rise of smaller, more efficient LLMs optimized for specific purposes, it’s not clear that the larger LLMs are universally relevant tools.
Investors frequently talk about companies’ moats. Warren Bufett famously used this metaphor at the 1995 Berkshire Hathaway annual meeting2:
What we’re trying to do, is we’re trying to find a business with a wide and long-lasting moat around it,…protecting a terrific economic castle with an honest lord in charge of the castle.
The metaphor is an apt one. Consider a medieval castle. Its principal defense mechanism is its moat—the wide, water-filled ditch that surrounds the edifice, which keeps aspiring conquerors at bay. Indeed, much strategic thought went into figuring out how to defeat moats. So a company which has a metaphorical moat around its business is one which ought to be an attractive investment.
Much of technology investing, especially early stage technology investing, doesn’t consider moats for the simple reason that they don’t really exist in early stage companies. When Peter Thiel invested in Facebook, for example, he didn’t provide the cash because he thought that Mark Zuckerberg had built a defensible moat around his social networking idea. Rather, Thiel invested the money in the hopes that somehow, eventually, Zuckerberg and his team would build that moat. So, saying that LLMs have no moat isn’t exactly perceptive or original: it’s obvious. And yet, I think, it’s frequently ignored.
Consider this table3:
We have six different LLM builders, all offering essentially the same token cost, whether we’re talking about input, output, or API calls, as the others. Sure, some of these companies’ tokens cost a fraction of a penny less than the others, but for all intents and purposes, these LLMs’ costs are all at parity. This suggests that these companies have to compete on something other than price to acquire customers.
Further, when we consider that open source LLMs are rapidly improving their performance, the lack of moats that these LLMs have is all the more apparent. Here’s a lively discussion on StackExchange about open-source vs closed-source LLMs.
If you’re a developer or a venture capitalist, you may be reading this post with some skepticisim. But consider this: for the average user, one LLM’s capabilities are indistinguishable from another’s. Consider the tyranny of the marginal user.
The conclusion that I am left with is this: AI is like aioli. That is to say, you can buy aioli in every sandwich shop in America, but there are no stores which only sell aioli. I think that something similar will happen with LLMs: you can buy an LLM from any technology company in America, but there will be no pure-play LLM providers. It is best to think of LLMs as the substrate on which other products and services are built. LLMs themselves are not great investments. The intelligence which LLMs give rise to is an ambient subtrate which allows other companies to build interesting new products and services.
The release date for Gemini Ultra is at present uncertain; you can see a prediction market about its release date here.
This table is adapted from here. I don’t know how frequently the table from which I adapted the table in this post is updated, but given that all of these model providers are competing vigorously against the others, I would expect pricing data for AI models to change fairly frequently. Thus what the numbers that you see at the link provided in this footnote may differ, significantly, from what is shown in this post.