I don't understand how this is different from oai "structured output" (and whatever the similar paradigm was on Sonnet ~3.7 back then) which everyone moved on from. On their gh they say:
"Jev is TypeSafe's closed service for runtime-defined semantic decisions. This project reproduces that interface pattern with open models; it does not reproduce Jev's undisclosed model or training"
As someone else pointed out it isn't actually Jev... can someone enlighten me
Structured decoding limits next-token probabilities to ensure valid JSON. The main issue is that if the model puts a substantial probability on an invalid token, then it was already confused, and in that case, you don't actually want whatever the next-most-likely valid token is: even if it's syntactically valid, it's likely semantically erroneous.
Not sure why you think everyone has moved on from this. Valid JSON is only one aspect. It almost certainly has far more uptake today than ca. Sonnet 3.7. Grammar engines can enforce a lot of other things too. Models are actually really good at emitting valid JSON, but complex schemas effectively require it.
wuhhh · · focus · HN ↗
"Jev is TypeSafe's closed service for runtime-defined semantic decisions. This project reproduces that interface pattern with open models; it does not reproduce Jev's undisclosed model or training"
As someone else pointed out it isn't actually Jev... can someone enlighten me
jLaForest · · focus · HN ↗
Could you please explain what you mean by "which everyone moved on from"?
slickytail · · focus · HN ↗
qeternity · · focus · HN ↗