I honestly think the diffusion LLM approach is a dead end
It's telling that frontier labs like Google toyed around with it but didn't invest further even for their most speed and cost sensitive small models
Still unclear for what, if any use cases this is pareto frontier
There were too many unsolved problems with diffusion to commercialize it, where autoregressive loops had a more straightforward roadmap. Doesn't mean the diffusion problems are insurmountable though.
nylonstrung · · focus · HN ↗
It's telling that frontier labs like Google toyed around with it but didn't invest further even for their most speed and cost sensitive small models
Still unclear for what, if any use cases this is pareto frontier
naasking · · focus · HN ↗