I don't understand how this is different from oai "structured output" (and whatever the similar paradigm was on Sonnet ~3.7 back then) which everyone moved on from. On their gh they say:
"Jev is TypeSafe's closed service for runtime-defined semantic decisions. This project reproduces that interface pattern with open models; it does not reproduce Jev's undisclosed model or training"
As someone else pointed out it isn't actually Jev... can someone enlighten me
Structured decoding limits next-token probabilities to ensure valid JSON. The main issue is that if the model puts a substantial probability on an invalid token, then it was already confused, and in that case, you don't actually want whatever the next-most-likely valid token is: even if it's syntactically valid, it's likely semantically erroneous.
Alternative is to fail the validation and regenerate the request, but I have no idea what this person is talking about. It is absolutely not the case that everyone has moved on from structured generation. Absurd claim.
wuhhh · · focus · HN ↗
"Jev is TypeSafe's closed service for runtime-defined semantic decisions. This project reproduces that interface pattern with open models; it does not reproduce Jev's undisclosed model or training"
As someone else pointed out it isn't actually Jev... can someone enlighten me
jLaForest · · focus · HN ↗
Could you please explain what you mean by "which everyone moved on from"?
slickytail · · focus · HN ↗
rhodysurf · · focus · HN ↗
qeternity · · focus · HN ↗