Thread

Jev in 25 Lines of Python

691 points · 212 comments · bashbjorn

chpatrick · 1790182010 · focus · HN ↗
Could someone explain how Jev is different from using any old model and constraining the output to "My choice is a/b/c..."?
1. jimmyl02 · 1790182633 · focus · HN ↗
  The argument the article makes is it's not that different. Jev's argument is that they have trained the model to better output probabilities (which is not necessarily a training object of LLMs but we don't actually know that)
  Ultimately Jev claims to have a data advantage which is likely where the future lies. They'll have a unique edge in improving general purpose classification / decisioning.
2. kccqzy · 1790192257 · focus · HN ↗
  If you believe the marketing, constraining the output this way can make the model much faster and much more type-safe (the model didn’t give you a fifth choice not present in the choices).
  1. chpatrick · 1790222914 · focus · HN ↗
    How is that different from just not letting it output option 5 when you sample from the probabilities?
3. xigoi · 1790197021 · focus · HN ↗
  The model can spend more “mental energy” on the decision because it doesn’t have to spend any on phrasing the output.
  1. chpatrick · 1790222858 · focus · HN ↗
    LLMs are constant time per token, if you only let it choose a/b/c/d then there's nothing else it can "spend".

Unofficial Hacker News client; not affiliated with Y Combinator.