Does Ilya Sutskever’s Safe Superintelligence company make any sense?
Billions in funding, no product, no revenue, claims about the fragility of LLMs…
This post serves as a complement to my recent post about Ilya Sutskever’s interview with Dwarkesh Patel, in which Sutskever outlines his vision for a post-LLM road to AGI.
I received a DM from a reader who wishes to remain anonymous: “Does Ilya Sutskever’s Safe Superintelligence company make any sense? Billions in funding, no product, no revenue, claims about the fragility of LLMs…”
This post attempts to provide an answer.
In short, SSI (aka, Safe Superintelligence) makes sense as a very particular kind of bet (on Ilya + short-ish AGI timelines + LLM limits). It makes almost no sense if you judge it as a normal startup with a product roadmap.
Let’s unpack it.
What SSI is, structurally
Basic facts:
Founded June 2024 by Ilya Sutskever, Daniel Gross, and Daniel Levy.
Public positioning: one goal, one product. Safe superintelligence. No side products, no SaaS, no consulting.
Funding: on the order of a few billion raised, valuation reported in ~$30 billion+ range, despite zero revenue.
As of mid-2025, headcount seems to be around tens of people.
Key design choices:
Straight-shot SSI. No intermediate commercial product; the first and only product is the thing itself, i.e., safe superintelligence.
Safety baked into the branding. The company name is the safety story.
VC- and cloud-aligned. Big name investors + a cloud deal (e.g., TPUs from Google Clou) to secure compute.
So this is not a normal seed stage startup. It’s closer to a private, fully-levered research institute with a giant call option on owning the AGI stack.
Ilya’s technical thesis
In the Dwarkesh interview, Ilya is now saying explicitly:
Current frontier models “generalize dramatically worse than people” and this is a fundamental limitation, not just “needs 10x more GPUs.” Scale, in other words, is not all you need.
The “age of scaling” (make transformers bigger and train on more text) is hitting diminishing returns; further progress is research-constrained, not GPU-constrained.
Pretraining on the internet is hitting limits (finite data, weird long-tail behavior, brittle out-of-distribution performance), so you need qualitatively different approaches. Think: better generalization, value functions, richer environments, etc.
Call this Ilya World: AGI soon, but not via bigger LLMs; instead via some new architecture/training regime optimized for human-like generalization and safety.
SSI is basically Ilya World in corporate form.
So: does that thesis itself make sense?
It is not crazy to say that LLMs are inefficient and brittle. That’s just obviously true.
It is not crazy to say you eventually need something that can learn fast online, with some internal value structure. That’s basically the agents/continual learning agenda everyone is groping toward.
Where it becomes speculative is the jump from: LLMs are limited; therefore we need a new, possibly radically different, system; therefore we should build a Manhattan Project around that, without shipping anything in the meantime.
That’s a stack of conjectures. They’re plausible, but not empirically grounded, yet.
Does the organizational design make sense?
The binding constraint here is “no product until SSI.”
Upsides of this constraint:
Focus and internal coherence. All talent, infra and governance is aligned on one hard problem. There are no product managers nagging for Q3 feature delivery to hit an enterprise ARR target, for example.
Reduced short-term race pressure. They’re trying to avoid the OpenAI trap: ship a ChatGPT-like thing, get addicted to revenue, then optimize for user growth rather than alignment.
Brand for regulators and talent. “We exist only to build safe superintelligence” is a hell of a recruiting and lobbying line, for those predisposed to value those things, regardless of how pure it stays.
Downsides of the constraint:
No real-world feedback loop. OpenAI, Anthropic, and others learn a lot about their models from deployment: red-team data, weird jailbreaks, emergent misuse patterns. If SSI stays purely focused on its internal lab, they sacrifice that empirical safety signal.
No revenue = pure belief financing. Investors and employees are paid on the belief that “someday this will be the superintelligence.” If progress is slower than expected, that belief dissipates. If they’re not able to create a superintelligence, they will either:
pivot to a product, or
become quasi-academic and accept massive down rounds, or
soft land via acquisition
Safety vs secrecy tension. If they really get close to SSI, their incentive is to hold their cards close to their chest, but if they don’t open the work to scrutiny, their safety claims become un-auditable vibes. That tension is structural.
So as an org design, it’s coherent given a short timeline + enormous confidence in Ilya’s technical hunch. It’s not robust if timelines stretch or if their approach underperforms relative to the status quo LLM + agent stacks.
The capital allocation question: $30 billion valuation, zero revenue
From a normal tech equity lens, this is insane: a ~$30 billion valuation for a few dozen people, no product, no revenue, and a mandate to not monetize until they solve what might be the most important problem in the history of technology.
From the perspective of AGI is worth a large percentage of global GDP, it’s less insane. Suppose the eventual winner of the AGI race captures even 1-5% of global surplus over the next few decades. That present value is very large; on any plausible distribution of winners, one or two labs get absolutely absurd payouts. Then a $30 billion valuation on a plausible contender is just a way to own a stake in that possibility. Investors not doing a discounted cashflow of API revenue. Investors are buying a lottery ticket on the opportunity to own the mind that runs civilization.
So:
For investors: SSI is a long-dated call option on (a) Ilya’s inside knowledge of what’s wrong with current LLMs; (b) his ability to recruit top talent; and (c) the chance that OpenAI and/or Anthropic get politically, ethically, or structurally stuck. In a power law world, overpaying for optionality can be rational.
For the broader economy: this is a symptom of capital desperately trying to find exposure to AGI in a world with very few pure plays. A lot of that capital will be misallocated; that doesn’t make each individual bet obviously irrational ex ante.
How robust is his claim that LLMs are fragile?
A lot of the public messaging around LLMs’ weaknesses is inchoate, but there is some structure to the claim. Roughly, the argument that Ilya is making is:
Transformers interpolate; humans generalize. We can get LLMs to very high benchmark scores but they lack the fast, robust adaptation humans show when conditions change.
Pretraining on static corpora hits asymptotes.
Internet data is finite.
The signal gets lossy.
You don’t get proper exploration or structured tasks.
You need a system that learns like a scientist, not an autocomplete. That means some notion of internal curiosity/value function, active data gathering, long-horizon planning, etc.
You can quibble with all of this (maybe model-based RL + tool use + LLMs get you scientist-like behavior), but it’s not empty hand-waving. It’s a working theory of where current deep learning is bending.
The real open questions:
Does this require a qualitatively new architecture, or just more scaffolding and clever training around LLM cores?
Are his claims about the “age of scaling” being over actually binding on a 5-10 year horizon, or do we have another 1-2 orders of magnitude of useful scale left?
SSI is essentially a bet on “we’ll win by going past the transformer-as-core paradigm while everyone else is still extracting the last juice from LLMs.”
Is SSI actually a safety company?
This is where the story is most questionable.
On the plus side:
Name, org design, and Ilya’s personal history all push toward: “we care about alignment and are not optimizing for a quick SaaS exit.”
The lack of short-term product pressure does make it easier to walk away from obviously dangerous deployments.
On the minus side:
Taking billions from top-tier VCs with a gargantuan valuation is not a safety-first move; it’s a “we’re here to win the AGI race” move. Investors will smile about safety in public, but their payoff is overwhelmingly dominated by “build the thing first.”
Their comparative advantage is capabilities. If your main moat is “we have Ilya and his weird ideas plus a ton of compute,” you’re not going to sit back and be the moral conscience of the field while others ship AGI-ish systems.
So I’d say: SSI is a capabilities lab with a safety aesthetic, not a safety lab that happens to touch capabilities.
Whether that’s bad depends on how you already feel about OpenAI, DeepMind, Anthropic, xAI, etc. It’s not qualitatively different; it’s just more naked about the “straight shot to SSI” part.
Relative to the rest of the field, does SSI’s strategy dominate anything?
If you line up the big labs:
OpenAI/Anthropic: productized APIs + assistants; massive infra; direct real-world feedback; strong, but messy incentives.
DeepMind: deep research bench, giant infra, multiple product surfaces (Search, YouTube, etc.).
Meta/xAI: various mixes of open-source, research, and product.
SSI: tiny headcount, big capital, no products, single ultra-ambitious research target.
Advantages SSI might actually have:
Can run extremely weird, compute-intensive experiments without asking “how does this ship into the app?”
Less internal politicking around product roadmaps.
Maybe easier to build a very tight, fanatically aligned culture around a single vision.
Disadvantages:
They are behind on infra, tooling, and deployment experience. You don’t get the industrial-grade MLOps, eval pipelines, abuse feedback, etc. that OpenAI and Anthropic earn the hard way.
If LLM scaling and agent scaffolding turn out to be “good enough” to get to usable AGI, SSI may end up beautifully solving the wrong problem while the world standardizes on “crude but works” systems.
So strategically, SSI is a high-variance move:
If Ilya’s model of the world is approximately right, they could leapfrog.
If he’s wrong or just early, they become a very expensive footnote in “we underestimated how far LLMs could go.”
Bottom line: does it “make sense”?
You can slice the answer:
As a rational play by Ilya: absolutely. If he honestly believes:
AGI is near-ish,
transformers-as-we-know-them will stall, and
he has specific ideas for “what’s next,” then locking in huge capital + freedom from short-term product pressure is exactly what he should do.
As a safety institution: mixed at best. They are another front in the superintelligence race with a “safety first” marketing layer. That’s still possibly better than a lab that doesn’t care at all, but it’s not clean.
As a VC bet: it’s a power-law portfolio move. In a world where there might only be 3–5 true AGI contenders, and one winner captures insane surplus, overpaying on valuation today is almost irrelevant if you think SSI has, say, a 5–10% chance of being that winner.
As a “normal startup with no product, no revenue”: no, it doesn’t make sense, but that’s the wrong reference class. Comparing SSI to a B2B SaaS startup is like comparing Manhattan Project budgets to a seed-stage materials startup. The reference class for SSI is “private AGI skunkworks,” not “company that should get to $10M ARR in 3 years.”
The more interesting meta-point is: the mere existence of SSI at these valuations is strong evidence about capital’s beliefs: Enough serious people believe AGI/SSI is near enough, tractable enough, and monetizable enough that they’re willing to pour billions into a lab that refuses to even pretend to have a normal business model.
In that sense, the sanity question is almost inverted. SSI is weirdly honest about what almost every frontier lab implicitly wants to do: build the thing that ends the game. The rest of the ecosystem is still play-acting as if this is about enterprise productivity tools rather than restructuring civilization.
If you enjoy this newsletter, consider sharing it with a colleague.
I’m always happy to receive comments, questions, and pushback. If you want to connect with me directly, you can: