Thread

1. adrithmetiqa · 1790632786 · focus · HN ↗

   Forgive my lack of understanding but how long before Jev type functionality is just built straight into all frontier models?

   1. jubilanti · 1790643994 · focus · HN ↗

      Do people really have zero awareness that Structured Outputs with a constrained schema has been a thing for a while now, and open weight models that give you logprobs can give you distributions per key?

      Like what am I missing? I use Structured Outputs every day and this just seems like that with fewer steps?

      edit: Where I'm coming from, I can triage 10,000 support tickets with deepseek flash for less than $1, and latency is sub 1 second if it needs to be integrated into a live user flow. I don't need anything cheaper or faster than that.

      1. Lerc · 1790647904 · focus · HN ↗

         I remember reading a paper about a model that used a parser bound to the end of a LLM where it squashed all logprobs for outputs that were parsing errors. I have not seen this used in the manner I thought I would. I thought it would have been entirely possible for a model to construct it's own grammar for how it would prefer to respond and then opt to generate tokens that matched (with a metacode to turn it off obviously).

         That said, I think the advantage of Jev style approaches is not their capabilities, but rather the capabilities that they have for a much lower resource requirement.

Unofficial Hacker News client; not affiliated with Y Combinator.