Understanding Inference and the "Stochastic Parrot" in Large Language Models

Everyone uses LLMs but few people understand how they work or what their limitations are. Let's fix that

This is a high level overview of inference in large language models (LLMs), and the associated gibe that LLMs are merely stochastic parrots. It is aimed at non-technical people. As such, it necessarily elides certain technical details. I’ve attempted to provide some additional, more technical, information and nuance in the form of footnotes. ChatGPT ass…