Going viral on Twitter amidst AI angst
A lot of people don't like the idea of an AI autonomously performing math
I went viral on Twitter the other day, and I have some thoughts. I posted about my amazement at getting ChatGPT, using GPT4o1, OpenAI’s newest, and most curiously named, model, to build a discounted cash flow analysis for Apple.2 I was interested in putting this LLM through its paces, and was casting about for some analyses that I could ask it to run. I settled on a discounted cash flow analysis for no real reason in particular; I haven’t worked in finance since before the Global Financial Crisis, and I don’t think I have thought about DCFs since then.
I first provided ChatGPT with Apple’s annual report, and instructed it to extract the relevant information from the financial statements included with the report. This did not work. ChatGPT does not seem to parse PDFs well, in contrast to Anthropic’s Claude 3 Opus model. I then realized that I would have to find the financial statements, copy them into a spreadsheet, uploade that spreadsheet, and then ask ChatGPT to run the analysis. This worked, and I shared the chat session to Twitter. That tweet is here, but the link is borked—I think that the traffic from Twitter triggered something on OpenAI’s end which basicallly killed access to the link.
What interests me about this is mainly the reactions I received on Twitter. They ranged from positive to very negative, and the very negative reactions are, I think, worth thinking a bit about. Negative reactions ranged from calling me clueless to calling it ‘performative financial analysis’, among other criticisms. One of the more substantive criticisms is here.
This substantive criticism is worth parsing a bit, because it gets to the heart of both ChatGPT’s strengths and its very real weaknesses. My operating thesis is that generative AI is going to take over much analytical work in the near future. Therefore, it is worth understanding how well it functions today, what its weaknesses are, how it might improve, etc. As the saying goes, today’s AI is the worst you’ll ever use.
This person’s main criticism is “ChatGPT clearly does not actually have a thesis or have a scenario modeled out. It’s just throwing numbers at the wall….this is pretty illustrative of the problem with ChatGPT. It can sort of use terms in a way that makes sense to people who don’t know tha tmuch about it, but to anybody actually in the industry, it’s making unclear assumptions and delivering inaccurate results.”
Overall, a set of reasonable, and accurate, criticisms. This is a real problem with ChatGPT: because it is so capable, it’s seductive. You want to believe its output. And if you’re not sufficiently knowledgeable about the subject at hand, you can be fooled into thinking that its analysis is correct. This is dangerous.
On the other hand, we might say the same thing about other technology that has made analytical projects easier: calculators, and Excel, too, were, at times, blamed for people blindly accepting erroneous results. As ever, the problem isn’t really with the technology. The problem is with the people. And if a person uses ChatGPT for work, but fails to check that the work is correct and accurate, well, that’s on the person, not the technology.
Blaming the technology is barking up the wrong tree here.
Ultimately, I think much of the negative reaction to stuff like this is people expressing their anxieties at the prospect of having been obviated by technology. I mean, they’re not yet obviated, but at some future point they may well be.
GPT4o, where o stands for omni, not the number you get when you subtract 5 from 5. Thank the Silicon Valley brain trust at OpenAI for this (in)elegant naming convention.
This isn’t a post about finance, and this Substack newsletter isn’t a newsletter about finance, so I’m not going to explain what a discounted cash flow analysis is. If you want to know what it is, or you want to refresh your memory from a freshman finance class, here is one link which explains what this is.