Dave - in a world where a meaningful % of inference moves to the edge, that would obviously be bad for Nvidia's GPU business, but what do you think that would mean for TSMC? Does this net out neutral for them in terms of revenue, just with a different mix of semis (and probably a reorganization of their fabs)? Or does this really hurt them?
I'm not sure that it would be bad for either Nvidia or TSMC, actually, if inference moved to the edge. First, not all inference would move to the edge. Large scale inference would still be done in large data centers, as would training.
And, to the extent that inference *does* move to the edge, Apple's silicon (and others') is made by TSMC.
Already see this in my enterprise customers. All want to use gen ai, coding assistants and similar, but none want to put their proprietary code and designs into someone else’s model.
Really excellent . I believe you are one of the better strategic thinkers on substack .
Read my take on nvidia and demand . It’s all about the volume (demand) of tokens and a function of how to process these .
You are correct inference token requirements are continuous and ever increasing as we move to video , 3D and immersive worlds ( at the funfair or the manufacturing floor.
Throw in simulation for robotics , Robotics, autonomous etc , all relying on token generation and my view the demand is insatiable
Dave - in a world where a meaningful % of inference moves to the edge, that would obviously be bad for Nvidia's GPU business, but what do you think that would mean for TSMC? Does this net out neutral for them in terms of revenue, just with a different mix of semis (and probably a reorganization of their fabs)? Or does this really hurt them?
I'm not sure that it would be bad for either Nvidia or TSMC, actually, if inference moved to the edge. First, not all inference would move to the edge. Large scale inference would still be done in large data centers, as would training.
And, to the extent that inference *does* move to the edge, Apple's silicon (and others') is made by TSMC.
Already see this in my enterprise customers. All want to use gen ai, coding assistants and similar, but none want to put their proprietary code and designs into someone else’s model.
To your list of bottlenecks, you could add reliability and autonomy. Edge inference keeps working in a disaster and is not subject to silent revision.
Really excellent . I believe you are one of the better strategic thinkers on substack .
Read my take on nvidia and demand . It’s all about the volume (demand) of tokens and a function of how to process these .
You are correct inference token requirements are continuous and ever increasing as we move to video , 3D and immersive worlds ( at the funfair or the manufacturing floor.
Throw in simulation for robotics , Robotics, autonomous etc , all relying on token generation and my view the demand is insatiable
Please read my post . Feisal Nanji
Thanks.
I will take a look at your post soon.