/biz/ - Business & Finance

>>57736404
Where will new senior devs come from after the current crop dies?

>>57722226
Memory is the major bottleneck with LLM training. NVIDIA is now producing the H100 chips which are not strictly faster but have much more memory, and an even larger price tag to go with it.
On the other hand, the unified memory model overcomes this limitation much more elegantly where you can get an M-chip macbook with 97GB RAM shared between CPU and GPU. AMD also touts this unified memory model which allows much more efficient use of memory (was pushed into mainstream with PS4).

FWIW here is an LLM running on 128-cpu machine.
https://youtu.be/argfZlPZKdY?t=527