I'm not exactly following through with the claim, can someone explain how the built-in classification would not necessitate more tokens used, or be much different from turning on reasoning? Not that I don't see the difference, I just doing see how OpenAI would do it well.
AFAIK Jev is nothing special technically so it's easy to embed it as an another tool for the LLM? For many batch tasks it can still be quite a token saver I think.
Or they can even offer it as a standalone API if deemed worth it.
1) It's very cheap and fast - you provide one input and many potential classifications, and the compute to ingest the input is shared.
2) It generates structured output natively - guaranteed to be correct
3) It's output probabilities are calibrated to actually mean something
OpenAI, or anyone else, could certainly replicate it - there are already articles guessing how Jev achieves its "parallel" classifications, but it seems the AI companies need to decide are they in the business of providing intelligence/tokens, or are they in the application business trying to compete with all their customers (not that Jev uses OpenAI).
> It's output probabilities are calibrated to actually mean something
Don't fall for marketing BS so easily.
Jev can output drastically different probabilities if you simply reorder the list of choices. And Jev's "confidence" output is fake/redundant - it's just a formula applied to probabilities, it conveys no additional information.
I bet they will eventually "fix" (read hide under the rug) the ordering problem by ordering the list on the backend before feeding to the model.
It seems that anyway most of the value is in the speed and cost.
If it really matters to you whether whether some business-specific classification confidence is above/below some specific threshold (vs just relative order), then you'd be better off training or fine tuning a custom model for that. Maybe that is something that TypeSafe are planning to also provide?
tolugenius · · focus · HN ↗
mnicky · · focus · HN ↗
Or they can even offer it as a standalone API if deemed worth it.
HarHarVeryFunny · · focus · HN ↗
1) It's very cheap and fast - you provide one input and many potential classifications, and the compute to ingest the input is shared.
2) It generates structured output natively - guaranteed to be correct
3) It's output probabilities are calibrated to actually mean something
OpenAI, or anyone else, could certainly replicate it - there are already articles guessing how Jev achieves its "parallel" classifications, but it seems the AI companies need to decide are they in the business of providing intelligence/tokens, or are they in the application business trying to compete with all their customers (not that Jev uses OpenAI).
hbrn · · focus · HN ↗
Don't fall for marketing BS so easily.
Jev can output drastically different probabilities if you simply reorder the list of choices. And Jev's "confidence" output is fake/redundant - it's just a formula applied to probabilities, it conveys no additional information.
I bet they will eventually "fix" (read hide under the rug) the ordering problem by ordering the list on the backend before feeding to the model.
HarHarVeryFunny · · focus · HN ↗
If it really matters to you whether whether some business-specific classification confidence is above/below some specific threshold (vs just relative order), then you'd be better off training or fine tuning a custom model for that. Maybe that is something that TypeSafe are planning to also provide?