Skepticism is cheap; ChatGPT is real

Skeptics sounds smart, meanwhile people are doing cool shit with ChatGPT. Also: the decline of podcasts.

Skepticism is easy. Understanding how to use ChatGPT is hard.

Bryan Caplan is skeptical about ChatGPT:

ChatGPT got a D on my last labor economics midterm. You could respond, “AI is overrated now, but amazing results are coming soon,” but I’m skeptical. I take base rates very seriously, and the base rate for new techs grossly overpromising and underdelivering is at least 95%. Kahneman’s maxim that “Nothing is as important as you think it is, while you’re thinking about it” is relevant as well. To my eyes, AI fandom looks more like a mass hysteria than a sensible reaction. While AI could be the exception that proves the rules, I doubt it.

Suffice it to say, I think this is wrong. I generally agree with this comment:

If your goal is to actually identify breakthrough technologies even slightly ahead of the curve, then I don't think it's helpful to apply base rates, for this exact reason. You will always predict “no”, you will be right 95+% of the time, and you will miss every transformative technology until it's too obvious to ignore.

I think AI is on a strong trajectory to be extremely useful, but I'm not sure I would take this bet. “Passing exams” is not an economically useful function (except to students who want to cheat?) and it's not clear to me that AI will be engineered or optimized for this. If you picked something with a clear economic value, like generating marketing copy or writing scripts for TV and movies, I would be much more likely to take the bet.

Here’s an alternate view about what ChatGPT has to offer the academy:

Five days later I no longer had any doubt: this thing will transform higher education. I was one of the students. And I was blown away by what machine learning was able to do for us, in real time. And I say this as somebody who had been a hardened skeptic of the artificial intelligence hype for many years. Note that I didn’t say “likely” transform. It will transform higher education. Here’s why.  

The first use-case is that the machine “filters mundane questions,” as one of our students put it quite eloquently. Meaning: you can ask the dumb questions to the AI, instead of in-class. Yes, there are dumb questions—or at least there are questions where the answer is completely obvious to anybody who knows even just a little bit about, say, malware analysis (or has done their assigned readings). 

The second benefit follows: “you no longer disrupt the flow of the class,” as several students pointed out when we wrapped up — for example with a question like, “What’s an ‘offset’ in a binary file?” Or: “What is an embedded resource in malware?” You don’t want to interrupt the class — ask ChatGPT. Back in the day you had to Google for a few minutes at a minimum, jump hectically from result to result, wade through some forum, until you finally found a useful response; by then the class conversation had moved on. ChatGPT will give you the response in 5 to 15 seconds, literally. That response speed was game-changing last week, because we could keep up with the instructor in real time, reading ChatGPT’s explanation of embedded resources while listening to Juan Andres talking about the same thing.

I don’t think passing an exam is a particularly interesting test for ChatGPT, as Jason Crawford says in his comment which I quote above. ChatGPT excels as a cognitive complement. It is an adjunct for cognition, as I’ve written before.

---

The death of podcasts?

I wrote yesterday that, in a world awash in content, information-dense content ought to win out, simply because people’s time is limited. To state it more plainly, podcasts suck and text is best. And here’s news that new podcast creation has fallen off a cliff:

One thing I keep hearing over and over is that it is so much harder to launch a podcast now than it was, say, three or four years ago. And that is usually coming from people at established studios with at least some marketing might. For independent creators, it must be nearly impossible. It is not entirely surprising that, according to data compiled by Chartr from Listen Notes, fewer podcasts were created in 2022 than in the two years prior. Even so, the margin is shocking: the number of new shows created dropped by nearly 80 percent between 2020 and 2022.

People will overthink this, but I think the explanation is quite simple: most people are crappy speakers, most people don’t have much of anything interesting to say, and most people use speech as a way to fill time. There’s little information density to be had with podcasts. And in a world of content abundance, how does an empty podcast fulfil any marginal value? Podcasts suck, and their decline is welcome.