I'm not exactly following through with the claim, can someone explain how the built-in classification would not necessitate more tokens used, or be much different from turning on reasoning? Not that I don't see the difference, I just doing see how OpenAI would do it well.
AFAIK Jev is nothing special technically so it's easy to embed it as an another tool for the LLM? For many batch tasks it can still be quite a token saver I think.
Or they can even offer it as a standalone API if deemed worth it.
1) It's very cheap and fast - you provide one input and many potential classifications, and the compute to ingest the input is shared.
2) It generates structured output natively - guaranteed to be correct
3) It's output probabilities are calibrated to actually mean something
OpenAI, or anyone else, could certainly replicate it - there are already articles guessing how Jev achieves its "parallel" classifications, but it seems the AI companies need to decide are they in the business of providing intelligence/tokens, or are they in the application business trying to compete with all their customers (not that Jev uses OpenAI).
the algorithms technically, sure, however the outcomes definitely depend on data quality and coverage like any other training method, this is well known
Read the paper. They train RLCR on existing big math problems. They subtract a brier score penalty from the correctness reward. No new confidence labels are needed.
I did, in the first days Jev came out, when people were bringing it up. Another assumption. Please review the HN commenting guidelines, the one which starts with "Please don't comment on whether someone read an article." is relevant here.
Nothing in that paper changes that ML algorithms are dependent on the training data. We can step back from Jev and algos to consider Bayes Theorem. If your sample is not representative of the population, your resulting statistics will be off. The same is true here. If the data you train a model like Jev with is not representative, the probabilities and confidences it outputs will not be representative.
What makes Jev interesting is that it works well out of the box across domains. What people who are well known in the field believe is that this is the result of Typesafe having a really good training data set. People are saying similar of MiMo-2.6 today.
"Did you read the article" doesn't apply to a link someone put in a comment. If you are going to be a hall monitor, at least do it properly. You are just acting in bad faith at this point.
The relevant data is the reasoning trace. Doesn't need user data. You can learn from people's detailed reasoning steps how confident they are, even outside your domain.
We were talking about Jev and probability, now you're changing the problem, a rhetorical trick some people try to employ.
Another that uses dice rolling, coin flips, and an inventory level example to drive home the point that Jev's output are not real probabilities for outcomes.
I taught it (ML course; a day on RL, at a university), you should really stop making assumptions friend. Data quality and coverage matters in learning algorithms.
Here's one of the books used in that course <a href="https://amlbook.com/" rel="nofollow">https://amlbook.com/
Thinking blocks are not a place you can derive real confidence scores in LLMs
My initial comment and every one following is about RLCR and that paper. You don't appear to grasp the basics of that paper, it's reward function or how the optimizer is updating weights.
> You are out of your depth and grasping at straws.
Do you have any credentials or evidence that others can use to determine if this statement is not more accurately describing the author who wrote it?
Perhaps a PhD in ML, research output like published papers, or teaching/professional experience - all things I have
We could debate the merits of the paper contents, but I suspect you have intentionally moved on to personal attacks. Regardless, nothing you have said (nor can be found in this paper) has been a counter argument that learning algorithms are sensitive to training data, where the measured output difference is used by the optimization algorithm when updating the parameters. Garbage in, garbage out is a saying for a reason. No algorithm fixes non-representative data.
>you have to have training data with accurate probabilities
This was your claim. If you can't read and understand that paper in relation to your claim, you are out of your depth. You haven't made a single claim relevant to that paper - just hand wavy comments about data.
I clarified multiple times that I meant "accurate, representative data" to "derive accurate probabilities"
you are still employing underhanded techniques in an attempt "win an internet debate" (my impression)
try being more accommodating and flexible over repeating the same lame things
it's not hard to say, "ah I see what you were trying to say..." and move towards a more constructive conversation
RLCR / Jev et al. can only give as accurate predictions and probabilities as the underlying data they are trained on represents. Biased data results in biased probabilities, no algorithm fixes this. Can we agree on this point?
They post train on big math and improve calibration across five different non math benchmarks so the new claim "can only give as accurate predictions and probabilities as the underlying data they are trained on represents" is also off base. RLCR generates its own calibration examples from ordinary questions and answer keys. RL usually isn't trying to represent a data set, it's closer to search.
I'm not expecting you to human RL here, but I do hope next time you remember there is another imperfect human behind the screen and are less too online in future interactions
tolugenius · · focus · HN ↗
mnicky · · focus · HN ↗
Or they can even offer it as a standalone API if deemed worth it.
HarHarVeryFunny · · focus · HN ↗
1) It's very cheap and fast - you provide one input and many potential classifications, and the compute to ingest the input is shared.
2) It generates structured output natively - guaranteed to be correct
3) It's output probabilities are calibrated to actually mean something
OpenAI, or anyone else, could certainly replicate it - there are already articles guessing how Jev achieves its "parallel" classifications, but it seems the AI companies need to decide are they in the business of providing intelligence/tokens, or are they in the application business trying to compete with all their customers (not that Jev uses OpenAI).
verdverm · · focus · HN ↗
danielmarkbruce · · focus · HN ↗
<a href="https://arxiv.org/pdf/2507.16806" rel="nofollow">https://arxiv.org/pdf/2507.16806
verdverm · · focus · HN ↗
danielmarkbruce · · focus · HN ↗
verdverm · · focus · HN ↗
the underlying data set needs to be representative
danielmarkbruce · · focus · HN ↗
verdverm · · focus · HN ↗
danielmarkbruce · · focus · HN ↗
verdverm · · focus · HN ↗
and then you are going to ignore all the research and results that clearly show otherwise? why?
what might we infer about the importance of data from a learning algorithm like decision trees?
danielmarkbruce · · focus · HN ↗
Existing datasets, different reward function.
verdverm · · focus · HN ↗
I did, in the first days Jev came out, when people were bringing it up. Another assumption. Please review the HN commenting guidelines, the one which starts with "Please don't comment on whether someone read an article." is relevant here.
Nothing in that paper changes that ML algorithms are dependent on the training data. We can step back from Jev and algos to consider Bayes Theorem. If your sample is not representative of the population, your resulting statistics will be off. The same is true here. If the data you train a model like Jev with is not representative, the probabilities and confidences it outputs will not be representative.
What makes Jev interesting is that it works well out of the box across domains. What people who are well known in the field believe is that this is the result of Typesafe having a really good training data set. People are saying similar of MiMo-2.6 today.
danielmarkbruce · · focus · HN ↗
verdverm · · focus · HN ↗
We both know who is
> just acting in bad faith at this point.
danielmarkbruce · · focus · HN ↗
Take RL 101. This is a common pattern.
verdverm · · focus · HN ↗
Another that uses dice rolling, coin flips, and an inventory level example to drive home the point that Jev's output are not real probabilities for outcomes.
<a href="https://news.ycombinator.com/item?id=49830385">https://news.ycombinator.com/item?id=49830385
> Take RL 101
I taught it (ML course; a day on RL, at a university), you should really stop making assumptions friend. Data quality and coverage matters in learning algorithms.
Here's one of the books used in that course <a href="https://amlbook.com/" rel="nofollow">https://amlbook.com/
Thinking blocks are not a place you can derive real confidence scores in LLMs
danielmarkbruce · · focus · HN ↗
You are out of your depth and grasping at straws.
verdverm · · focus · HN ↗
Do you have any credentials or evidence that others can use to determine if this statement is not more accurately describing the author who wrote it?
Perhaps a PhD in ML, research output like published papers, or teaching/professional experience - all things I have
We could debate the merits of the paper contents, but I suspect you have intentionally moved on to personal attacks. Regardless, nothing you have said (nor can be found in this paper) has been a counter argument that learning algorithms are sensitive to training data, where the measured output difference is used by the optimization algorithm when updating the parameters. Garbage in, garbage out is a saying for a reason. No algorithm fixes non-representative data.
danielmarkbruce · · focus · HN ↗
This was your claim. If you can't read and understand that paper in relation to your claim, you are out of your depth. You haven't made a single claim relevant to that paper - just hand wavy comments about data.
verdverm · · focus · HN ↗
you are still employing underhanded techniques in an attempt "win an internet debate" (my impression)
try being more accommodating and flexible over repeating the same lame things
it's not hard to say, "ah I see what you were trying to say..." and move towards a more constructive conversation
RLCR / Jev et al. can only give as accurate predictions and probabilities as the underlying data they are trained on represents. Biased data results in biased probabilities, no algorithm fixes this. Can we agree on this point?
danielmarkbruce · · focus · HN ↗
verdverm · · focus · HN ↗
<a href="https://www.youtube.com/watch?v=c1Fv1uKTd-w" rel="nofollow">https://www.youtube.com/watch?v=c1Fv1uKTd-w
oh-seven
danielmarkbruce · · focus · HN ↗