I think electricity is the correct model for foundational models. In the open source community the tooling is being built to make it “pluggable” into any foundational model. Some tools will be power hungry and be more like industrial machinery that needs a specialized connection to the grid. Certain consumer grade products will need just regular “household” electricity. And small single-purpose tools will need foundational models that are more like batteries (I.e. highly optimized for local on-device usage).
Yup, that makes sense. The reason why I also compare these models to the printing press is that, as with the printing press, they distribute knowledge and intelligence. That's powerful, and different from most other software.
Computing itself is a general purpose technology that hasn't reached high maturation, because another general purpose technology, electricity, is the barrier to entry. The factors limiting or slowing increased access are cultural, political, and economical. Global/Network States may be interested in connecting more individuals, whereas exceptionalists/ isolationists do not see this as a priority. I have written on this topic, although I do not want to share a link unless it is requested :)
Yes, but technological convergence gives priority to memory-intensive smartphones, which have absorbed computers, and created unnecessarily high complexity for some markets.
Smartphones today have integrated satellite receivers, accelerometers, cameras, and countless other hardware, except for a solar panel. (I am not paid by a solar lobby; it is just an observation). The solar panel, is actually a super effective general purpose technology- because it provides electricity.
When Bill Gates and Steve Jobs were interviewed in 2007 (https://gizmodo.com/bill-gates-told-steve-jobs-about-the-ipad-in-2007-5541969), Jobs mentioned the iPhone (new at the time) smartphone having a computer, which was a novelty at the time as prior cellular ones used microcontrollers. Technically there were some smartphones prior to that with Symbian and memory management units (MMUs), but most were microcontrollers before the Nokia 6110.
Gates describes the smartphone/iPad like Windows [circa[ 1992] (which was when the higher bit-depth GUI arrived, compared to monochrome flip phones). Gates also described there being multiple devices- a tablet, a pocketphone, rather than a single device.
If having multiple devices is no longer cost-prohibitive, then having a technologically de-converged flip phone with a with a solar panel that just makes calls and sends texts, can be manufactured today due to Koomey's Law and the power consumption of smaller transistor nodes being that of what a credit card solar panel can produce nearly anywhere (including by a lightbulb). As Gates describes, there is no need to use a single device for all applications. Yet, the smartphone market designs phones that way.
So as AI is being touted as a foundational, general technology, there are many other users who could get access to the internet like it's 1995, if smartphone makers didn't try to make the power consumption at least ~1 watt (e.g. "You must be this tall to ride".
Jobs also mentioned in the interview, "so what?" :
"Only a few months later, the iPhone changed that vision. And the iPhone or the iPad are anything but specialized devices around the PC. They have a life on their own. They are general purpose computing devices in a phone and tablet format. Jobs later pointed this out:
We're getting to the point where everything's a computer in a different form factor. So what, right? So what if it's built with a computer inside it? It doesn't matter. It's, what is it? How do you use it? You know, how does the consumer approach it? And so who cares what's inside it anymore"
He recognized the portable form factor eclipsing the CPU as the "central" most advanced processing unit by the more lucrative app store above it in software-scanning QR codes, mobile payments, etc.
Computers have become so inexpensive and mass produced that they can offer more competition in smartphone features, rather than pure clock speed.
What has fallen behind though, is an emphasis on power consumption, and a percentage of users worldwide are perpetually out of battery, or are in a region with no electricity. 16 years after the iPhone was a novelty, I am asking smartphone makers the same "so what?" about their leading edge features. Having multiple hardware devices or different power design limits/settings is just as much of a feature/value than software variety, including AI.
Jobs describes at 3:10 the computer as the prominent "digital hub", but those technologies going mobile - "post-PC devices"- the host at 5:00 even mentions to Steve that the term garnered letters to the editor.
"Nowadays, Steve Jobs agrees with this vision. He thinks that the iPad is the future and the traditional PC is dying. Like he told Ryan Tate: "The times are a changin', and some traditional PC folks feel like their world is slipping away."
I think electricity is the correct model for foundational models. In the open source community the tooling is being built to make it “pluggable” into any foundational model. Some tools will be power hungry and be more like industrial machinery that needs a specialized connection to the grid. Certain consumer grade products will need just regular “household” electricity. And small single-purpose tools will need foundational models that are more like batteries (I.e. highly optimized for local on-device usage).
Yup, that makes sense. The reason why I also compare these models to the printing press is that, as with the printing press, they distribute knowledge and intelligence. That's powerful, and different from most other software.
Computing itself is a general purpose technology that hasn't reached high maturation, because another general purpose technology, electricity, is the barrier to entry. The factors limiting or slowing increased access are cultural, political, and economical. Global/Network States may be interested in connecting more individuals, whereas exceptionalists/ isolationists do not see this as a priority. I have written on this topic, although I do not want to share a link unless it is requested :)
Hm I think of computing as an application of technology, rather than a technology unto itself.
Yes, but technological convergence gives priority to memory-intensive smartphones, which have absorbed computers, and created unnecessarily high complexity for some markets.
Smartphones today have integrated satellite receivers, accelerometers, cameras, and countless other hardware, except for a solar panel. (I am not paid by a solar lobby; it is just an observation). The solar panel, is actually a super effective general purpose technology- because it provides electricity.
When Bill Gates and Steve Jobs were interviewed in 2007 (https://gizmodo.com/bill-gates-told-steve-jobs-about-the-ipad-in-2007-5541969), Jobs mentioned the iPhone (new at the time) smartphone having a computer, which was a novelty at the time as prior cellular ones used microcontrollers. Technically there were some smartphones prior to that with Symbian and memory management units (MMUs), but most were microcontrollers before the Nokia 6110.
Gates describes the smartphone/iPad like Windows [circa[ 1992] (which was when the higher bit-depth GUI arrived, compared to monochrome flip phones). Gates also described there being multiple devices- a tablet, a pocketphone, rather than a single device.
If having multiple devices is no longer cost-prohibitive, then having a technologically de-converged flip phone with a with a solar panel that just makes calls and sends texts, can be manufactured today due to Koomey's Law and the power consumption of smaller transistor nodes being that of what a credit card solar panel can produce nearly anywhere (including by a lightbulb). As Gates describes, there is no need to use a single device for all applications. Yet, the smartphone market designs phones that way.
https://hackaday.io/project/177716-the-open-source-autarkic-motherboard
https://github.com/EI2030/Low-power-E-Paper-OS/blob/master/Remaking%20the%20Nokia%206110%20and%20Psion%20Series%203%20on%2022nm%20PDF%20pages%2020-56.pdf
So as AI is being touted as a foundational, general technology, there are many other users who could get access to the internet like it's 1995, if smartphone makers didn't try to make the power consumption at least ~1 watt (e.g. "You must be this tall to ride".
Jobs also mentioned in the interview, "so what?" :
"Only a few months later, the iPhone changed that vision. And the iPhone or the iPad are anything but specialized devices around the PC. They have a life on their own. They are general purpose computing devices in a phone and tablet format. Jobs later pointed this out:
We're getting to the point where everything's a computer in a different form factor. So what, right? So what if it's built with a computer inside it? It doesn't matter. It's, what is it? How do you use it? You know, how does the consumer approach it? And so who cares what's inside it anymore"
He recognized the portable form factor eclipsing the CPU as the "central" most advanced processing unit by the more lucrative app store above it in software-scanning QR codes, mobile payments, etc.
Computers have become so inexpensive and mass produced that they can offer more competition in smartphone features, rather than pure clock speed.
What has fallen behind though, is an emphasis on power consumption, and a percentage of users worldwide are perpetually out of battery, or are in a region with no electricity. 16 years after the iPhone was a novelty, I am asking smartphone makers the same "so what?" about their leading edge features. Having multiple hardware devices or different power design limits/settings is just as much of a feature/value than software variety, including AI.
Jobs describes at 3:10 the computer as the prominent "digital hub", but those technologies going mobile - "post-PC devices"- the host at 5:00 even mentions to Steve that the term garnered letters to the editor.
"Nowadays, Steve Jobs agrees with this vision. He thinks that the iPad is the future and the traditional PC is dying. Like he told Ryan Tate: "The times are a changin', and some traditional PC folks feel like their world is slipping away."