A thought-provoking essay about how when today’s LLMs run out of real world models, they don’t hallucinate gibberish; they perform Beckett, elegant, confident, and utterly stranded in a reality that never arrives. Wonderfully written! 👏
2. How to model "gut feelings" combined with the data that the literal gut - stomach, intestines - are filled with neurons that generate and respond to the same neurotransmitters as the brain.
3. What if neurons cannot be modeled as simple connections to other neurons defined by mathematical functions?
If you've known anyone very intuitive, it is clear there is some other system at work that does not easily lend itself to narration, which makes it illegible, but no less powerful. An example: Years ago I was at a parade in Chicago's Lakeview, on Clark St. A small, kind of awkward man was walking in the parade saying hi to everyone and shaking hands. My wife saw him and immediately and visibly recoiled. Who is that?? He is so slimy! It was future disgraced Illinois governor Rod Blagovitch, campaigning.
[I don't think she's clarvoyant; I think she has a system that used tools other than language to notice his gait, eye movement, tone of voice, etc etc and calculate he was maximally untrustworthy.]
There was also a line that "neurons may be much more complicated than we think." I am reminded of Roger Penrose's conjecture that human cognition is defined by quantum effects, combined with Hameroff's assertion that nanotubes in neurons are capable of hosting coherent quantum states. So each neuron contains a quantum computer, effectively.
I have no idea if these speculations are true, but it does hint that there is room for a lot more complexity than can be modeled by a digital computer of any magnitude.
LLMs predict the next word. When that word is grammatically correct, semantically sensible, yet factually false, that is in fact a failure of prediction.
A thought-provoking essay about how when today’s LLMs run out of real world models, they don’t hallucinate gibberish; they perform Beckett, elegant, confident, and utterly stranded in a reality that never arrives. Wonderfully written! 👏
👍👍👍
I listened to Sutskever’s interview with Dwarkesh -- the most striking thing was what wasn't discussed much at all: Emotions. Even the section titled "Emotions and Value Functions" was 10% about emotions. https://www.dwarkesh.com/p/ilya-sutskever-2?open=false#%C2%A7emotions-and-value-functions
1. How to model intuition?
2. How to model "gut feelings" combined with the data that the literal gut - stomach, intestines - are filled with neurons that generate and respond to the same neurotransmitters as the brain.
3. What if neurons cannot be modeled as simple connections to other neurons defined by mathematical functions?
If you've known anyone very intuitive, it is clear there is some other system at work that does not easily lend itself to narration, which makes it illegible, but no less powerful. An example: Years ago I was at a parade in Chicago's Lakeview, on Clark St. A small, kind of awkward man was walking in the parade saying hi to everyone and shaking hands. My wife saw him and immediately and visibly recoiled. Who is that?? He is so slimy! It was future disgraced Illinois governor Rod Blagovitch, campaigning.
[I don't think she's clarvoyant; I think she has a system that used tools other than language to notice his gait, eye movement, tone of voice, etc etc and calculate he was maximally untrustworthy.]
There was also a line that "neurons may be much more complicated than we think." I am reminded of Roger Penrose's conjecture that human cognition is defined by quantum effects, combined with Hameroff's assertion that nanotubes in neurons are capable of hosting coherent quantum states. So each neuron contains a quantum computer, effectively.
I have no idea if these speculations are true, but it does hint that there is room for a lot more complexity than can be modeled by a digital computer of any magnitude.
LLMs predict the next word. When that word is grammatically correct, semantically sensible, yet factually false, that is in fact a failure of prediction.
"On the internet no one knows you're a dog."