Thread

1. adrithmetiqa · 1790632786 · focus · HN ↗

   Forgive my lack of understanding but how long before Jev type functionality is just built straight into all frontier models?

   1. wgd · 1790641609 · focus · HN ↗

      Negative three years, give or take. Although recent Anthropic and OpenAI models no longer expose the capability. But for any open model you just tell it to respond with a single token "Y/N" and take the logit difference. If you want multiple distinct questions answered you just ask them independently and put the shared context first so it gets cached.

      OpenAI and Anthropic don't want to give out logprobs these days but could trivially add a dedicated classification API to their existing models if there was enough demand.

      1. Renaud · 1790641964 · focus · HN ↗

         I think the main differentiator offered by Jev is not the ability to answer questions, most models can be coerced into that function if they don’t already have a dedicated pipeline for it, rather it’s the extreme speed of the evaluation, and very low cost that opens new possibilities.

         1. selcuka · 1790643344 · focus · HN ↗

            > it’s the extreme speed of the evaluation, and very low cost that opens new possibilities.

            It also returns confidence scores for all choices.

            Granted, they are not stable. They fluctuate even when you reorder choices, but it still counts as an additional feature.

            1. wonnage · 1790645099 · focus · HN ↗

               They’re useable if you’re trying to rank autosuggestions or something, not great at implying actual understanding

Unofficial Hacker News client; not affiliated with Y Combinator.