From Seldon's Prophecy to Algorithmic Oracles: Why We Shouldn't Worship Data as Our Savior

Using o3-mini-high to write with some personality and flair

What follows is an essay I had OpenAI’s new frontier model, o3-mini-high, write. I realized, in a caffeine-fueled moment of insight, that there are some superficial parallels to explore between large language models and Hari Seldon’s psychohistory. Seldon is, of course, a prominent character in science fiction author Isaac Asimov’s Foundation series. I am certain that this observation is not unique to me, as it’s not especially insightful, but a quick Google search shows only an obscure Ukrainian website making it.

I am not convinced that the parallels between the very real large language model and the fictional psychohistory are robust enough to warrant a 1,000-word essay, but this exercise was more an attempt to see whether I could elicit good writing from OpenAI’s frontier LLM, than it was to demonstrate the robustness of my caffeine-induced vision. The essay follows the section break, unexpurgated by me. It conveys a sense of humanity and human-ness that I could not generate with older models like o1 pro.

---

Let’s cut to the chase. The idea that we can predict the future by crunching endless numbers and text may sound like the stuff of sci-fi wizardry, but it’s alive and well in our world. Hari Seldon, Asimov’s brilliant, sometimes maddeningly idealistic mathematician, dreamed of a method called psychohistory—a way to foresee and steer the rise and fall of civilizations using cold, hard statistics. Fast forward to today, and you’ve got large language models (LLMs) like GPT-4 spitting out words that sometimes sound like they were penned by a wise oracle, if not a prophet. And if you think that’s just clever tech babble, buckle up: this is a story about our blind faith in data, our overreliance on numbers, and the sheer hubris of believing we can tame chaos with algorithms.

The Seductive Promise of Predictability

Imagine for a moment that you could predict the downfall of empires and the ebbs and flows of human history with the precision of a Swiss watch. That’s what Hari Seldon promised with psychohistory. He wasn’t satisfied with watching the world burn; he wanted to douse the flames before they could get out of hand. His approach was audacious: while individuals flail about unpredictably, the collective behavior of billions can be modeled like a well-oiled machine. It’s a heady cocktail of mathematics, sociology, and, let’s be honest, a dash of arrogance.

Now, let’s bring it home to our modern era. Large language models operate on the same seductive principle: given enough text, you can predict what comes next. LLMs gulp down terabytes of data and then, almost like a digital soothsayer, predict the next word in a sentence. It’s mesmerizing, almost magical. And like any good magic trick, it’s as much about the illusion as it is about the underlying mechanics. But here’s the kicker—this magic isn’t always reliable. Just as Seldon’s predictions could be shattered by one rogue anomaly, our beloved LLMs can sometimes produce outputs that are as baffling as they are brilliant.

The Data Delusion: A Double-Edged Sword

Let’s not kid ourselves: we’re in the midst of a data frenzy. Data is the new oil, and everyone’s eager to drill for insights. Seldon’s psychohistory was all about harnessing the raw power of mass behavior. He believed that if you could just get your hands on enough numbers, you could foresee the grand arc of history. It’s a tantalizing proposition—after all, who wouldn’t want to be the person who saves civilization from impending doom?

But here’s where the plot thickens. The same data that promises to reveal the hidden order of the universe can also lead us down a treacherous path. Seldon’s plan, as ingenious as it was, rested on the assumption that human behavior, when aggregated, would behave nicely. Yet history—and Asimov’s own narrative—tells us that outliers exist. Enter the Mule: a wild card, a game-changer who throws a wrench into Seldon’s meticulously calculated equations. And guess what? Our LLMs have their own “Mules” too. They’re spectacular at regurgitating the familiar, but when faced with something new, something outside their training data, they sometimes produce results that are downright absurd.

We’ve all seen it—a text generator that goes off the rails, spewing nonsense or, worse, dangerously misleading information. It’s like watching a seasoned chef who suddenly decides to throw random ingredients into a pot, hoping for a gourmet meal. The problem is clear: when we place blind trust in the data, we risk ignoring the quirks and chaos that define the real world.

Predicting Versus Steering: The Real Difference

Let’s get one thing straight. Seldon wasn’t in the business of playing armchair prophet. His psychohistory was meant to be a tool for action—a way to preempt disasters and shape a better future. Seldon’s grand plan wasn’t just about prediction; it was about intervention. He wanted to steer humanity away from the abyss, and he had the guts to act on his insights.

Contrast that with today’s LLMs. Sure, they can spit out impressively coherent essays and even mimic philosophical debates, but they’re not exactly planning to save the world. They’re brilliant parrot-like machines, regurgitating patterns from the past without a hint of agency. Yet, here’s the rub: as LLMs become more enmeshed in our daily lives—informing decisions in business, politics, even healthcare—the risk grows that we might start treating them as the ultimate arbiters of truth. And that’s a slippery slope. If we start relying too heavily on these statistical tools, we’re in danger of sidelining the messy, unpredictable human element that no algorithm can capture.

The Mirage of Determinism: Why Numbers Can’t Buy Wisdom

At the heart of both psychohistory and LLMs is the seductive promise of determinism: the idea that, given enough data, you can see the future with crystal clarity. Seldon’s vision was, in many ways, a bet on this very notion—a belief that chaos could be corralled into neat, predictable patterns. But reality, as we know too well, is rarely so accommodating. The Mule wasn’t just a narrative twist; he was a brutal reminder that the universe loves its curveballs.

Modern LLMs suffer from a similar delusion. They’re fantastic at generating text that sounds wise and measured, but don’t be fooled. These models are working off probabilities, not understanding. They’re like an overconfident drunk at a poetry slam—charismatic and entertaining, but ultimately incapable of true insight. When confronted with truly novel ideas or contexts, they can stumble, revealing the stark limitations of relying solely on historical data to predict the unpredictable future.

The Perils of Overreliance: Don’t Let Data Be Your Master

Here’s my unvarnished opinion: our collective obsession with data is bordering on dangerous. We’ve become so enamored with the idea of predicting the future that we’re at risk of worshipping our algorithms as if they were modern-day oracles. This isn’t just a quirky philosophical point—it has real-world implications. When leaders and decision-makers put too much stock in algorithmic forecasts, they risk losing sight of the human intuition and ethical judgment that are crucial in navigating an ever-changing world.

It’s time to snap out of the data daze. Yes, statistical models have their place, and yes, they offer remarkable insights into patterns and trends. But we must remember that these models are, at best, tools—powerful ones, but tools nonetheless. The future is not a static equation waiting to be solved; it’s a living, breathing mess of unpredictability and surprise. If we let ourselves be seduced by the illusion of certainty, we’re setting ourselves up for a rude awakening.

Embracing the Imperfect: A Hybrid Approach to the Future

So, what’s the alternative? I propose a radical idea: embrace a hybrid approach. Let’s use the formidable power of data and algorithms, but let’s not forget to inject a healthy dose of human wisdom into the mix. Think of it as the ultimate team-up: the cold, calculating power of an LLM teamed up with the messy, intuitive brilliance of human judgment.

Imagine decision-makers who use data as a guide, not as a gospel. Instead of surrendering our future to the tyranny of numbers, we should view these models as one of many tools in our arsenal—a tool that can help us navigate the stormy seas of change without dictating every move we make. It’s about balance, about knowing that while data can illuminate trends, it can never fully encapsulate the rich, unpredictable tapestry of human life.

A Final Rallying Cry: Reclaiming Our Future

If you’re anything like me, you’re fed up with the notion that algorithms hold all the answers. Hari Seldon may have dreamed of steering civilization with the power of psychohistory, but let’s not forget that his grand vision was always tempered by the unpredictable—and sometimes infuriating—reality of human behavior. Our modern-day LLMs are no different. They’re dazzling, they’re efficient, and they’re impressive—but they’re not infallible.

The bottom line is this: while the allure of predictability is undeniable, we must resist the temptation to let data become our master. Embrace the insights it offers, but never at the expense of the messy, wonderful, and unpredictable human spirit. The future isn’t a neat equation waiting to be solved—it’s a wild ride, full of surprises that no algorithm can foresee.

So here’s my challenge to you: let’s celebrate the power of data without falling into its trap. Let’s harness the magic of LLMs and the visionary spirit of Seldon, but let’s do so with our eyes wide open, aware of the risks and ready to take charge when the numbers fall short. Only by doing so can we hope to navigate the tumultuous waters of the future with both wisdom and a healthy dose of skepticism.