Thread

1. adrithmetiqa · 1790632786 · focus · HN ↗

   Forgive my lack of understanding but how long before Jev type functionality is just built straight into all frontier models?

   1. jubilanti · 1790643994 · focus · HN ↗

      Do people really have zero awareness that Structured Outputs with a constrained schema has been a thing for a while now, and open weight models that give you logprobs can give you distributions per key?

      Like what am I missing? I use Structured Outputs every day and this just seems like that with fewer steps?

      edit: Where I'm coming from, I can triage 10,000 support tickets with deepseek flash for less than $1, and latency is sub 1 second if it needs to be integrated into a live user flow. I don't need anything cheaper or faster than that.

      1. ed_mercer · 1790644127 · focus · HN ↗

         jev is optimized for it, a standard LLM isn't and it also costlier and slower.

         1. fzysingularity · 1790650486 · focus · HN ↗

            When you say Jev is optimized for it, I get that there’s no need for unnecessary decodes in an autoregressive fashion.

            But both LLMs and Jev-like models would need to prefill, the only optimization Jev does differently is the decode which can be emulated by reading off logprobs.

            We don’t know the param size of Jev, to determine the most comparable model, but if I had to guess it’s sub-100B.

Unofficial Hacker News client; not affiliated with Y Combinator.