I find it wild companies were so ready to ship intellectual property to another company. I sort of thought they would experiment then use all the massive RAM on our Mac books to implement something more local we can use and keeps our stuff out of another companies hands but it was just Claude Claude Claude tokenmax tokenmax tokenmax.
FWIW I still think that future is likely, but at least three things hold this back (and I say as someone doing low-budget local LLM experiments)
1) the typical (non-Mac) PC architecture beige box isn't really ready for this; it's starting to change a bit, the new Nvidia laptop RTX chips might change it a little, and maybe the Ryzen AI/Strix Halo stuff will catch up, but I suspect the lack of unified VRAM in boring corporate machines is holding back routine corporate experimentation (since most corporate machines also have bare minimum GPUs)
2) frankly, the Apple Silicon architecture isn't truly ready for it either! Even with unified VRAM, memory bandwidth is limiting, and the processing power of the GPU holds back prompt processing for things like document analysis. So far the MoE models and the really small footprint dense models are lacking somewhat, though you can use them for some things (and the Mac is a much better pre-existing-machine playground for early experiments).
3) perhaps more importantly, the apparent progress of the cloud LLMs has persuaded corporate customers that what they are aiming for is increasing delegation rather than increasing developer capability (the former does not automatically lead to the latter)
yeah the big tech companies approached our company to buy our private codebases and data for training purposes. I advised we do not sell anything to them because their goal isn't just to try to replace our workers with bots, the real goal is to replace our entire business. There will be one mega corp to rule them all.
Just think, if they can sell your a virtual worker that can compete with human employees in your industry/niche, they would be a few steps away from being your DIRECT competitor or at the very least selling those same workers to someone else.
wookmaster · · focus · HN ↗
dofm · · focus · HN ↗
1) the typical (non-Mac) PC architecture beige box isn't really ready for this; it's starting to change a bit, the new Nvidia laptop RTX chips might change it a little, and maybe the Ryzen AI/Strix Halo stuff will catch up, but I suspect the lack of unified VRAM in boring corporate machines is holding back routine corporate experimentation (since most corporate machines also have bare minimum GPUs)
2) frankly, the Apple Silicon architecture isn't truly ready for it either! Even with unified VRAM, memory bandwidth is limiting, and the processing power of the GPU holds back prompt processing for things like document analysis. So far the MoE models and the really small footprint dense models are lacking somewhat, though you can use them for some things (and the Mac is a much better pre-existing-machine playground for early experiments).
3) perhaps more importantly, the apparent progress of the cloud LLMs has persuaded corporate customers that what they are aiming for is increasing delegation rather than increasing developer capability (the former does not automatically lead to the latter)
The first two will be fixed.
reworkal · · focus · HN ↗
Just think, if they can sell your a virtual worker that can compete with human employees in your industry/niche, they would be a few steps away from being your DIRECT competitor or at the very least selling those same workers to someone else.