‹ BackHN Continuity

Thread

A single function Jev-like wrapper for LLMs, including vision models

157 points · 45 comments · allanrbo

  1. prathje · · focus · HN ↗
    Nice! I would love to use it for images as well. Then again is using Grammar-Based Decoding with a json response not the same? Is Jev just that with nice caching? Because then I have been using that already…
    1. incompressible · · focus · HN ↗
      A functional difference is the probabilities associated with each option. True, you could use the constrained next-token distribution at the appropriate generation step, but those would not be calibrated by any means, which Jev's claim to be. An empirical analysis would be interesting.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.