I am hearing about Jev for the first time here so no idea about the hype.
So their(Jev) is that the thing is faster at classification than a frontier model? Because the whole type safe aspect is already fully solvable with structured output.
But their example is classification but that would also be possible and faster with a classic BERT model.
So their pitch is a task specific smaller model or am I completely misunderstanding the whole thing?
I think this discourse is still in the "figuring it out" phase. But here's where my thoughts are currently:
If you accept the premise that there are use cases where you might ask a frontier model a classification-shaped question and expect an ok enough answer, rather than creating a purpose specific classifier on some dataset that you have, then it follows that this is quite an inefficient thing to do, because you're doing extra work to turn the output tokens into a structured output and mostly throwing them away. So then if you could instead train a frontier level model that skips the output tokens and directly returns the structured classification information, that would be more efficient, and that's what jev seems to be.
But a lot rides on that initial premise of whether this is a use case that makes sense. But if you find yourself asking a model like Opus arbitrary yes/no questions and then maybe you switch to a faster and cheaper model because it's too slow and expensive, it seems like jev might be a great replacement for that.
bruhhhhhh · · focus · HN ↗
sanderjd · · focus · HN ↗
If you accept the premise that there are use cases where you might ask a frontier model a classification-shaped question and expect an ok enough answer, rather than creating a purpose specific classifier on some dataset that you have, then it follows that this is quite an inefficient thing to do, because you're doing extra work to turn the output tokens into a structured output and mostly throwing them away. So then if you could instead train a frontier level model that skips the output tokens and directly returns the structured classification information, that would be more efficient, and that's what jev seems to be.
But a lot rides on that initial premise of whether this is a use case that makes sense. But if you find yourself asking a model like Opus arbitrary yes/no questions and then maybe you switch to a faster and cheaper model because it's too slow and expensive, it seems like jev might be a great replacement for that.