Thread

1. hbrn · 1790115782 · focus · HN ↗

   $40m in funding, 2 years in stealth.

   Performs on-par with SemIf which was built in a couple days and apparently uses raw Qwen, with no fine-tuning. SemIf runs in your freaking browser. Oh and Jev is twice as expensive?

   Is it surprising that Jev consistently thinks it's Qwen?

   I'm almost convinced that Jev is a scam. Take Qwen, fine tune it a little, tell investors it cost $10m, spend $1m on advertising, profit.

   1. vessenes · 1790161758 · focus · HN ↗

      I think it’s much more likely that these benchmarks are not good, in that they do not explore much of the space that jev was (likely) trained to cover. The doom demo is a good example — I’d like to see a wide variety of things like that included in any benchmark, not just ‘email classification’ or what have you.

      Think of it this way: there’s some time needed to optimize / design an architecture, and the world gets that for free when it’s described. As to the rest of the last two years spent, is it more likely a former oAI lead spent them fucking around, or adding as many RL environments as possible to its model that is supposed to be a generalized classifier?

      Right now my prior is that jev is probably better than these rando weekend models, whether or not we know how to test and demonstrate that in a benchmark. It’s also super cheap, so I don’t think there’s a strong reason not to try out building with it first, then walk down the ladder to an open model if you need to for some reason.

      1. florianstandhar · 1790162838 · focus · HN ↗

         Agree the coverage question is real. Today it's text decisions (intent, routing, extraction, scoring, harder multi-constraint cases). We're working on screen/computer-use and image decisions (a noindex preview exists) because that's closer to where Jev-class models will actually be used. Suggestions for task families are very welcome as GitHub issues.

Unofficial Hacker News client; not affiliated with Y Combinator.