Thread

1. dghlsakjg · 1785816328 · focus · HN ↗

   I know everyone wants to crap all over these setups that are impractical, but this is how progress happens.

   People will keep plugging away at this and figure out how to avoid wearing the hard drive, how to make it run faster, custom hardware buses etc.

   Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips.

   1. pizza234 · 1785836695 · focus · HN ↗

      > but this is how progress happens.

      This is progress in the same way that a man climbing a tree is making progress toward reaching the moon.

      This project is essentially the MoE-of-the day, with some platform-related optimizations.

      > Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips.

      That won't happen. Projects like this just give the illusion that that will be possible.

      1. sipjca · 1785840520 · focus · HN ↗

         Unfortunately you are wrong. Time marches on and progress in silicon does too. You may not like the timeline, but it will happen.

         While not exactly an “SSD”, High Bandwidth Flash will make it to market and make big sparse MOE models even more accessible. Everyone knows bandwidth is the game right now and you think hardware vendors are going to sit on their heels and not take part of the market? Absolutely silly to discount progress both on the software and hardware sides.

Unofficial Hacker News client; not affiliated with Y Combinator.