What if we're building the wrong kind of AI?

Ilya Sutskever's case that AGI won't come from a bigger LLM, and why that possibility destabilizes every GPU forecast

TL;DR

• Ilya isn’t warning about alignment. He’s arguing that trillions in GPU capex may be aimed at the wrong paradigm.

• His premise: transformers interpolate; humans generalize. The gap is sample efficiency, robustness, and continual learning.

• Bigger models now show diminishing returns: scaling improves benchmarks but not true generalization; failures get stranger, not rarer.

• If transformers can’t reach human-level learning, the compute regime flips: less episodic mega-training, more 24/7 inference-as-learning spread across the real economy.

• Capex → opex; elastic training clusters → persistent agent habitats; compute demand becomes distributed, domain-tied, and less fungible.

• Three scenarios:
  • Benign: GPUs remain the workhorses; mix shifts but demand persists.
  • Bear: Algorithmic breakthroughs make intelligence cheaper; training clusters look overbuilt.
  • Radical: GPUs are the wrong substrate; new architectures dominate.

• SSI doesn’t need the biggest cluster, only a qualitatively different learner. That shifts competitive advantage from scale to research insight.

• The GPU supercycle assumes scaling continues; continual learning would invert its economics.

• Even a modest probability that transformers aren’t the road to AGI should meaningfully alter how multi-billion-dollar clusters are valued.

---

When Ilya Sutskever told Dwarkesh Patel in a recent interview that today’s frontier models “generalize dramatically worse than people” and that current approaches “will go some distance and then peter out,” he wasn’t making an alignment claim. He was making an economic one, and more specifically, he was making a claim about whether the trillions going into GPU farms are financing the right problem. Take his worldview as a temporary axiom—call this Ilya World—and a very different picture of AI compute emerges.

Ilya’s core thesis is simple: transformers interpolate; humans generalize. The difference isn’t philosophical. It’s about sample efficiency, robustness under distribution shift, and the ability to learn new tasks without a bespoke, fragile training curriculum. Humans can learn to drive after ten hours. Models need billions of tokens to learn far simpler skills. They ace coding competitions while whiffing on reasoning that any competent engineer can do after a week on the job. They have no internal value function resembling a teenager’s real-time sense of this is promising versus this is a dead end. You can pour compute into reinforcement learning to paper over this deficit, but you are still scaffolding a brittle learner with vast amounts of curated experience.

This is why Ilya repeatedly points to continual learning, value functions, and deployment-side adaptation as the real frontier. An actual generalizer wouldn’t need to have all of its knowledge dumped into it all at once. It would learn the way humans do: incrementally, contextually, and with a self-generated sense of progress. The AGI as a giant pre-trained slab paradigm is, in this view, a historical accident. It’s a conceptual overshoot caused by the dominance of pre-training, the fetishization of scaling laws, and an industry optimized around the cheapest-to-explain recipe for turning money into benchmarks.

The economic implication is direct: the last five years were the Age of Scaling. The next five are the return of the Age of Research, but now with planetary-scale computers. Frontier training was the scaling era’s single growth vector. You buy more H100s, you get a bigger model, the loss curve improves, the benchmark PR writes itself. This virtuous cycle made capex predictable. The returns to adding compute were linear enough that CFOs could model them. Today, something more ambiguous has arrived. The models get bigger, but their generalization doesn’t deepen proportionally. Their test time behavior improves in narrow ways while regressing in global ways. Their failures are weirder, not rarer.

If the transformer paradigm hits diminishing returns before it yields a human-level learner, then compute demand shifts. Demand does not decline to zero, but its shape changes. A continual learning system doesn’t rely on three gigantic training runs per year. It relies on 24/7 deployment across millions of tasks, each feeding back into a global model. Instead of spikes of demand for massive clusters, you get a persistent, constant, geographically distributed appetite for compute embedded across the real economy. Training compute becomes less important than inference-as-learning. Capex becomes opex. GPU clearinghouses designed for short-lived training reservations suddenly face a world in which the most valuable compute is not elastic, but resident.

From here, three futures branch. None are predictions; they are simply the consequence of taking Ilya World seriously.

In the benign future, GPUs remain the workhorses of the agentic economy. Even if transformers stall, the new paradigms could still be implemented on GPU-like dense linear algebra hardware. The mix shifts: less frontier training, more inference and continual learning. But demand doesn’t collapse. Instead, it spreads. Clusters become a patchwork of persistent agent habitats rather than cathedral-like training centers. Nvidia’s growth curve bends but doesn’t break.

In the bear case for GPUs, the bottleneck turns out to be algorithmic, not architectural. Suppose the new learning algorithm is radically more sample efficient. Suppose it can form durable abstractions with one or two orders of magnitude less compute. Then you can get to superhuman at task X with smaller models and much cheaper hardware. The next intelligence leap comes from software, not silicon. The entire training cluster economy suddenly looks overbuilt. The market is currently pricing this scenario at essentially zero; that’s rarely a reliable guide to reality.

And in the radical case, GPUs are not just insufficient. They’re the wrong type of machine. If continual learning systems require completely different primitives—spiking architectures, local connectivity fabrics, or hardware that can natively represent value functions and internal credit assignment—then today’s GPU-led stack is the wrong substrate. This would be like trying to build the internet with vacuum tubes. It might work for a while, but the curve points elsewhere. What looks like incremental capex today could be, in retrospect, the peak of the GPU epoch.

It’s here that Ilya’s view becomes economically explosive. If AGI is not a bigger transformer but a qualitatively different learner, then the decisive variable is not the size of your training cluster but the cleverness of your training recipe. Ilya’s startup SSI does not need the largest farm on Earth to prove their concept; they only need enough compute to demonstrate a qualitatively different generalizer. And once such a model exists, it won’t immediately be obvious how to copy it. But it will be obvious that something new is possible. The competitive landscape would shift from who can scale training the fastest to who can understand and reproduce this strange, new learning principle. Capex dominance becomes less decisive; research taste and architectural insight matter more.

This leads to a broader point: the GPU supercycle is implicitly priced as a continuation of the scaling era. Every marginal dollar assumes that bigger general purpose clusters will remain the right bet for at least the next decade. But if continual learning displaces pre-training as the central act, the economics invert. Compute becomes a substrate for distributed cognitive apprenticeship, not a furnace for trillion-token batches. Clusters differentiate; niches form; specialization increases. High value compute is no longer fungible. It’s tied to the domain in which an agent has lived, learned, and accumulated its unique experience.

You don’t have to believe SSI will succeed. You don’t even need to believe Ilya’s model is the future. You only need to accept the conditional: if the transformer paradigm isn’t the final step before AGI, then the GPU capex wave is building out infrastructure for the tail end of a regime rather than the beginning of one. In capital markets, that distinction matters enormously. Even a 10% probability on the algorithmic bottleneck scenario radically changes the expected value of multi-billion-dollar training clusters. Allocators who treat this as a convexity problem, not a trend extrapolation problem, are thinking correctly.

If Ilya World is even partially true, then we are not in the late stages of a GPU boom. We are in the early stages of a much stranger transition: from compute as training fuel to compute as the substrate of a continually adapting, agentic civilization. The question is not whether GPUs will be used. It’s whether they’re the right substrate for the mind we’re about to build. At minimum, this should move some probability mass in your decision tree from more of the same, but bigger to different game entirely.

---

If you enjoy this newsletter, consider sharing it with a colleague.

I’m always happy to receive comments, questions, and pushback. If you want to connect with me directly, you can:

• follow me on Twitter,

• connect with me on LinkedIn, or

• send an email to dave [at] davefriedman dot co. (Not .com!)