They should just give up at this point, it's just embarrassing to watch.
As PrimeTime said; these are the guys that invented the 'T' in 'GPT', that deployed their first TPU in 2015, that is using billions on AI - and they are beaten by 300 people startup named Moonshot AI even. People are going to write books about this complete fumble.
Strongly disagree with that take. Kimi K3 is a distilled model. I'm not saying that as a moral judgement, or to disparage the team behind it, but distilling and building on that is significantly easier and cheaper than building from the ground up.
And Gemini is kinda good enough at everything. Never the top, but it is decent at every task, and it is much faster than Kimi K3 and significantly cheaper. Kimi is very focussed on coding, Gemini isn't.
More importantly, it natively understands text, audio and video. If/when we are able to make the jump to robotics, this becomes essential. As you say, Google has a lot of deep background and deep pockets, they are able to make more of a long play. No idea if it will pay off, but it is way to early in the game to count them out.
Distillation does not make a model. There is so much more that goes into creating LLM Ai, traces from better models alone is insufficient. You need a good pre training run as foundation, a great RL reward function to make the other model's traces useful, and serious engineering chops to manage training runs at that scale. (non-exhaustive list)
bronlund · · focus · HN ↗
As PrimeTime said; these are the guys that invented the 'T' in 'GPT', that deployed their first TPU in 2015, that is using billions on AI - and they are beaten by 300 people startup named Moonshot AI even. People are going to write books about this complete fumble.
arw0n · · focus · HN ↗
And Gemini is kinda good enough at everything. Never the top, but it is decent at every task, and it is much faster than Kimi K3 and significantly cheaper. Kimi is very focussed on coding, Gemini isn't.
More importantly, it natively understands text, audio and video. If/when we are able to make the jump to robotics, this becomes essential. As you say, Google has a lot of deep background and deep pockets, they are able to make more of a long play. No idea if it will pay off, but it is way to early in the game to count them out.
verdverm · · focus · HN ↗