First, congrats to the team on launching something genuinely interesting and new.
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
I think the meaning of can't hallucinate in this model is that the type won't be hallucinated.
So if the generated schema is for a tool call for calculator, then the numbers will be valid numbers for sure (and not random words).
To me, it looks similar to BNF schema already introduced and implemented few years ago: generally speaking - it limits the next token that is allowed to be generated, probs are drawn from a subset tokens.
(tbh, I'm not sure why it didn't pick up as a more standard interface to LLMs, as it made a lot of sense back then, and now.)
AFAICT it is the same interface as you describe, but the underlying inference algorithm is fundamentally different, hence the speed gains. There is an application I am currently working on right now where this typed output predictor is the performance bottleneck. I'd be very interested to see how this performs.
jacobgold · · focus · HN ↗
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.
soleveloper · · focus · HN ↗
So if the generated schema is for a tool call for calculator, then the numbers will be valid numbers for sure (and not random words).
To me, it looks similar to BNF schema already introduced and implemented few years ago: generally speaking - it limits the next token that is allowed to be generated, probs are drawn from a subset tokens.
(tbh, I'm not sure why it didn't pick up as a more standard interface to LLMs, as it made a lot of sense back then, and now.)
adastra22 · · focus · HN ↗