Thread

A single function Jev-like wrapper for LLMs, including vision models

157 points · 45 comments · allanrbo

prathje · 1790408875 · focus · HN ↗
Nice! I would love to use it for images as well. Then again is using Grammar-Based Decoding with a json response not the same? Is Jev just that with nice caching? Because then I have been using that already…
1. allanrbo · 1790443112 · focus · HN ↗
  Almost the same. The neat trick here is to ask the model to reply with just 1 letter, which is one token, rather than a long string of json. Its quicker. And just the fact that Jev made a pretty decent API for structuring your questions. And they do the massive parallelization.

Unofficial Hacker News client; not affiliated with Y Combinator.