We’ve always had output schemas for LLMs, and we’ve had small language classifiers for decades, so what’s new? Is it just some sweet spot in between in terms of quality vs speed?
1) it's very fast (they claim 40-200x faster than frontier models [1], would roughly line up with it doing diffusion)
2) each answer carries a calibrated probability (ie. frequency of outcome is close to predicted)
Another point being that it doesn't reason, hence designed for "System One" tasks.
I wonder if in continuous control with discrete actions (eg. their DOOM demo) it can make sense to blend answer by confidence instead of taking the argmax.
brap · · focus · HN ↗
We’ve always had output schemas for LLMs, and we’ve had small language classifiers for decades, so what’s new? Is it just some sweet spot in between in terms of quality vs speed?
brausepulver · · focus · HN ↗
1) it's very fast (they claim 40-200x faster than frontier models [1], would roughly line up with it doing diffusion)
2) each answer carries a calibrated probability (ie. frequency of outcome is close to predicted)
Another point being that it doesn't reason, hence designed for "System One" tasks.
I wonder if in continuous control with discrete actions (eg. their DOOM demo) it can make sense to blend answer by confidence instead of taking the argmax.
[1] <a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev" rel="nofollow">https://typesafe.ai/blog/introducing-system-one-models-and-j...