First, congrats to the team on launching something genuinely interesting and new.
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
I'm biased but I wouldn't call it misleading - generating text is super awesome and flexible, (we describe that in the blog post - and I personally use string models all the time) but it's true you pay a high tax for autoregressive generation
> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do
His claim was that the title is misleading, not sure how it's relevant to that claim that you use "string models" (full LLMs).
The original title before it changed less than an hour ago was:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm going to agree that was misleading.
And on the second point:
>>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
>that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do"
Also going to disagree here, and I don't think it's semantics.
User input: "Hey, have your human support agent call me, tomorrow at 5pm."
Model input: "Does the user want to speak to a human support agent?"
Output: Yes.
I imagine that your model would produce this, and I think it's fair to say this is a hallucination. A human would caveat it with: "Yes, but not right now.", your model is incapable of that. Yes is technically correct, but within the context of being in a live chat, a human would understand that the caveat is required.
No, the model has answered correctly. Your question is poorly phrased (possibly deliberately).
Your question would correctly classify the user's input as requesting a human support agent, but at an indeterminate time.
If you wanted to determine whether the user wants to speak to a human support agent immediately, you would have to correctly qualify your question, e.g. "Does the user want to speak to a human support agent now?". You could have another question which is "Is the user requesting a call-back from a human support agent?". Or you could have a multiple choice query which would filter the conversation into one of a number of pre-written possibilities.
This is nothing to do with accuracy or hallucination. It's a different method of interacting with the model where you are relied upon to be precise.
"call_back_day": {
"criteria": {
"none": "The user does not want a call back.",
"today": "The user wants a call back today.",
"tomorrow": "The user wants a call back tomorrow."
},
"instructions": "When does the user want a call back?",
"type": "choice"
}
jacobgold · · focus · HN ↗
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
CompleteSkeptic · · focus · HN ↗
> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do
WhitneyLand · · focus · HN ↗
The original title before it changed less than an hour ago was:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm going to agree that was misleading.
And on the second point:
>>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
>that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do"
Also going to disagree here, and I don't think it's semantics.
Type safety is not factual correctness.
CompleteSkeptic · · focus · HN ↗
I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?
bigglebear · · focus · HN ↗
Model input: "Does the user want to speak to a human support agent?"
Output: Yes.
I imagine that your model would produce this, and I think it's fair to say this is a hallucination. A human would caveat it with: "Yes, but not right now.", your model is incapable of that. Yes is technically correct, but within the context of being in a live chat, a human would understand that the caveat is required.
monkpit · · focus · HN ↗
djray · · focus · HN ↗
Your question would correctly classify the user's input as requesting a human support agent, but at an indeterminate time.
If you wanted to determine whether the user wants to speak to a human support agent immediately, you would have to correctly qualify your question, e.g. "Does the user want to speak to a human support agent now?". You could have another question which is "Is the user requesting a call-back from a human support agent?". Or you could have a multiple choice query which would filter the conversation into one of a number of pre-written possibilities.
This is nothing to do with accuracy or hallucination. It's a different method of interacting with the model where you are relied upon to be precise.
matja · · focus · HN ↗