I honestly think the diffusion LLM approach is a dead end
It's telling that frontier labs like Google toyed around with it but didn't invest further even for their most speed and cost sensitive small models
Still unclear for what, if any use cases this is pareto frontier
nylonstrung · · focus · HN ↗
It's telling that frontier labs like Google toyed around with it but didn't invest further even for their most speed and cost sensitive small models
Still unclear for what, if any use cases this is pareto frontier
kennywinker · · focus · HN ↗
K2-Horizon-7B has a diffusion and non-diffusion variant, and they claim the same level of intelligence from both models.