Thread

1. prometheus1992 · 1789830629 · focus · HN ↗

   I think the main gripe that people had with Jev and Typesafe was the language used when they launched. To me personally it seemed like a parody/con/shady at first.

   "Breakthrough", "our research went in another direction" , "Two years in stealth", "System One thinking model", "Jev can't hallucinate", "RLCD","We are doing very cool stuff, but we will have to hire you to tell you", - these are some of the things that they said on their website on the launch blog.

   I had used versions of bert to achieve the same functionality years ago. But to me it seems like they were able to trick the VCs with "can't hallucinate" etc.

   To the above author, kudos for sharing your work and making it open. Something like this shouldn't be closed in the first place when it has been available for so many years

   1. seizethecheese · 1789836567 · focus · HN ↗

      I was confused by the “can’t hallucinate” thing, because it sounded like BS but people were taking it seriously. I purposefully asked a stupid question sort of like “this can’t hallucinate because it only has one output and there’s a schema?”. Was disappointed to learn the answer was yes.

      1. MisterMunchkin · 1789836965 · focus · HN ↗

         Yeah it’s hilarious, it definitely can hallucinate. Just because it can only hallucinate “A” or “B” rather than a whole paragraph, doesn’t mean it is suddenly more accurate.

         And they’re acting like their probability isn’t as hallucinated as any other LLM guess.

         1. seizethecheese · 1789837716 · focus · HN ↗

            They’re definining hallucination as a property of iterative generation, which is fair enough, but then it’s sort of like selling a boat and saying it doesn’t need tire changes.

            1. TeMPOraL · 1789839753 · focus · HN ↗

               It does make some sense given they're positioning it as alternative to the normal way you'd implement such output shape, which is to slap a prompt on a frontier LLM and maybe run it in "constrained output" mode if you like things fancy. Against that use case, the "no hallucinations" and parallelism and cost claims all sound legitimate and useful -- and similarly, "but we could do that with BERT two years ago" does not.

               1. seizethecheese · 1789848392 · focus · HN ↗

                  I mean, the constrained output mode also doesn’t hallucinate in this sense.

                  1. dropofwill · 1789917509 · focus · HN ↗

                     They do actually admit that about constrained decoding somewhere in the docs. They argue it’s useless in practice because when the constraints actually kick in it harms the output too much and that it’s better to just error and retry in those cases.

                     That does align with my experience, though we’re not using anything close to frontier for these sort of tasks.

                     I am interested if it can actually improve on that. As an engineer i like the elegance of guaranteed output, but the retry works pretty well in practice.

            2. fastball · 1789840668 · focus · HN ↗

               I don't think that is an entirely fair comparison. They are comparing Jev to the way people are currently using generative LLMs for things like classifying/tool calling/any kind of structured output.

               For example, if you feed in some context to Jev and Claude Haiku and say "make the appropriate tool call based on this context", Claude (or any other frontier LLM) will hallucinate tool calls some percentage of the time. Jev will not. While yes, the "will not" is constrained by Jev's (lack of) capabilities in some sense, this is actually a very real need for a wide variety of use-cases people are currently using off-the-shelf LLMs for at the moment.

               Probably the better example is the whole probability thing, where even if you use something like constrained decoding to ensure an LLM only outputs a certain schema, and therefore can't hallucinate a class, if you ask for probabilities, the probabilities output by the model are just hallucinations. Jev meanwhile is outputting calibrated probabilities for different choices based on the actual landscape.

               1. refulgentis · 1789842255 · focus · HN ↗

                  "will hallucinate tool calls" doesn't match any definition of hallucination I've seen in 4 years. Tool calls are output of the model, it can't "hallucinate" they happened. Maybe you're describing output we'd disagree with?

                  1. fastball · 1789843682 · focus · HN ↗

                     Hallucinate tools that don't exist.
                  2. adastra22 · 1789843904 · focus · HN ↗

                     Not the person you’re replying to, but I think that was a bad example. Because an LLM‘s output is iterative, the output ends up being influenced by various attractors. That doesn’t happen when you one-shot a single prediction (or multiple parallel predictions). That is a whole category of things, that people traditionally call hallucinations, that are structurally cut off by Jev’s architecture.

                     That doesn’t mean the models outputs are correct, nor is TypeSafe claiming that afaict.

Unofficial Hacker News client; not affiliated with Y Combinator.