I don't understand Jev. Its a generic classifier right? Like the classifiers we were building 15 years ago with random forrests and logistic regressions, but just generic. What's so revolutionary? And how can the accuracy be any better than a custom trained classifier that can be built in a day (an hour using Claude).
Have you tried Jev and compared it against your alternative classifiers? Should take half an hour to do that, then you’ll have your answer (or someone who already did it can tell you here).
You need to do it to know the results for your own use case. The barrier for trying things out is extremely low now, just relying on others to evaluate tools for you is not very effective as their use case is unlikely to be the same as yours and these tools vary a lot from case to case.
I did this yesterday with user data to find duplicates and determine which conflicting fields should win in a merge. Took me about an hour-ish to run the experiment. I provided a good amount of context for each pair. It was underwhelming / OK at best and didn't add any value to my existing merge workflow, so I threw it out after the experiment (I kept the findings though). It was quite easy to setup and use via openrouter and extremely cheap and fast.
For fun I also tried using it at my local LLM router (all of my prompts and responses go through it for personal analytics) to decide which model to route tasks to based complexity etc. Again, underwhelming for my purposes, and so I'm not using it.
Not sure what real world use case it's best at, but I agree that it's simple enough to implement it yourself and see if it fits the type of work you are doing.
I don’t think its much cheaper like luna is double the cost. and you can easily use luna to generate a training set and train your own model. I think the main things its faster so a lot of people go the Wow moment I can classify my whole db at once. Or this model decides so quickly. Speed is the key differentiator, it’s a strong one, always.
I was watching some youtube hype demo of it to pick items (each an emoji) from a pile according to questions. One of the questions was: "what can a magnet attract?" and it picked like 8 metal items, while a literal magnet was left unpicked. Ofc the video was too busy praising it to notice. Was all I needed to see
That it can make mistakes? That’s expected, it deal with plausibility like other models no? You would need to look at the actual response and the assigned probabilities to evaluate. Generally demos are not a great way to evaluate a technology, it’s a way to get hype but the next step is to actually look at the details
Yes, I don’t think hallucinations will ever go away. But because it doesn’t work with language the type of hallucinations aren’t too comparable to LLMs. But it is still a risk of course, whatever system you design should take that in consideration
The "moat" is data. And compute. But mostly data.
It is a classifier, that you don't have to train, that seems to work well, and they have made available by API.
One of the reasons for LLM success has not just been that they are "smart" but that they are smart while not requiring you to collect your own data and perform your own training.
There is also the marginal advantage of it not requiring a PhD to know how to use. But most companies can get a competent data scientist on board, so that is not a major problem, they could find someone to train their classifiers. But even a good data scientist can't make data appear out of nowhere. If you can get a pretty good result without needing data and training, for cheap, you're going to take it.
(Let's put aside the fact that almost every company using LLMs these days is irresponsibly NOT comparing the results with their own internal gold standard datasets for validation and calibration. They are just winging it. And Jev allows them to continue to do that.)
The core feature is good quality out-of-the-box domain and cross-domain calibrated probability you can use as a threshold.
SO you do not need to have YOUR data to train classifier. All you need is to benchmark it on your process/data to see that it really is good for your domain (as authors promise) and choose a threshold for acting.
Ahh, and the most important thing you need to do - design well the input context (state), questions with their context (instructions) and output type (because the same question asked as Noul (yes probability) or Choice with yes/no variants give different probability of yes.
> Like the classifiers we were building 15 years ago with random forrests and logistic regressions
I think no one mentioned here, but the obvious difference is that Jev can spit out decisions directly from natural language input. None of these ML models could do that, and other than a full-fledged LLM (which is optimized for conversation and agentic tasks) or some classical NLP models (which underperform compared to LLMs, AFAIK), there is nothing right now that rivals Jev-like models.
Of course it's not perfect, but seems like the right step forward for quick classification/decision tasks based on natural language.
As far as I can tell from just looking at the documentation, yeah, it looks like a "better than nothing" and "definitely better than LLMs for a few tasks" tool, and quite possibly something that I'd build into a demo, but I'd be really careful deploying it in any kind of product where exactness matters.
gatapia2 · · focus · HN ↗
I don't understand the hype.
nnevatie · · focus · HN ↗
Because it's all "AI" now <insert Ancient Aliens meme>
brabel · · focus · HN ↗
dgellow · · focus · HN ↗
brabel · · focus · HN ↗
rapind · · focus · HN ↗
For fun I also tried using it at my local LLM router (all of my prompts and responses go through it for personal analytics) to decide which model to route tasks to based complexity etc. Again, underwhelming for my purposes, and so I'm not using it.
Not sure what real world use case it's best at, but I agree that it's simple enough to implement it yourself and see if it fits the type of work you are doing.
asutekku · · focus · HN ↗
jwpapi · · focus · HN ↗
asutekku · · focus · HN ↗
vrighter · · focus · HN ↗
MaxikCZ · · focus · HN ↗
dgellow · · focus · HN ↗
MaxikCZ · · focus · HN ↗
dgellow · · focus · HN ↗
radarsat1 · · focus · HN ↗
The "moat" is data. And compute. But mostly data.
It is a classifier, that you don't have to train, that seems to work well, and they have made available by API.
One of the reasons for LLM success has not just been that they are "smart" but that they are smart while not requiring you to collect your own data and perform your own training.
There is also the marginal advantage of it not requiring a PhD to know how to use. But most companies can get a competent data scientist on board, so that is not a major problem, they could find someone to train their classifiers. But even a good data scientist can't make data appear out of nowhere. If you can get a pretty good result without needing data and training, for cheap, you're going to take it.
(Let's put aside the fact that almost every company using LLMs these days is irresponsibly NOT comparing the results with their own internal gold standard datasets for validation and calibration. They are just winging it. And Jev allows them to continue to do that.)
flashnik · · focus · HN ↗
SO you do not need to have YOUR data to train classifier. All you need is to benchmark it on your process/data to see that it really is good for your domain (as authors promise) and choose a threshold for acting.
Ahh, and the most important thing you need to do - design well the input context (state), questions with their context (instructions) and output type (because the same question asked as Noul (yes probability) or Choice with yes/no variants give different probability of yes.
pedrosbmartins · · focus · HN ↗
I think no one mentioned here, but the obvious difference is that Jev can spit out decisions directly from natural language input. None of these ML models could do that, and other than a full-fledged LLM (which is optimized for conversation and agentic tasks) or some classical NLP models (which underperform compared to LLMs, AFAIK), there is nothing right now that rivals Jev-like models.
Of course it's not perfect, but seems like the right step forward for quick classification/decision tasks based on natural language.
Yoric · · focus · HN ↗