Thread

1. wuhhh · 1789728098 · focus · HN ↗

   I don't understand how this is different from oai "structured output" (and whatever the similar paradigm was on Sonnet ~3.7 back then) which everyone moved on from. On their gh they say:

   "Jev is TypeSafe's closed service for runtime-defined semantic decisions. This project reproduces that interface pattern with open models; it does not reproduce Jev's undisclosed model or training"

   As someone else pointed out it isn't actually Jev... can someone enlighten me

   1. mritchie712 · 1789728642 · focus · HN ↗

      in short: it's faster, cheaper, smart structured output.

      each "question" is answered in parallel instead of a sequential (like an LLM). so if you have an input like:

          {"is_it_hotdog": noul, "is_it_apple", noul}
      
      

      it answers is_it_hotdog and is_it_apple in parallel and gives a probability.

      1. satvikpendem · 1789730206 · focus · HN ↗

         Can't I just parallelize my LLM calls myself for each question?

         1. orbital-decay · 1789730453 · focus · HN ↗

            You can. It will be expensive, slow, and less reliable than a specialized model.
         2. zwily · 1789734092 · focus · HN ↗

            Anything you can do in Jev can be done with an LLM at much greater cost and latency.

            1. mmnfrdmcx · 1789735747 · focus · HN ↗

               Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs.

               1. esafak · 1789738725 · focus · HN ↗

                  That number is a big deal, assuming it is well calibrated. Did they talk about calibration?

                  1. Matticus_Rex · 1789741054 · focus · HN ↗

                     I've seen them talk about it a bit on Twitter -- it seems to be fairly well-calibrated in general, but obviously you need to test it on your use case and dial it in comparison with known data for best results.
         3. shados · 1789860132 · focus · HN ↗

            Yup. LLMs can do almost anything. Can they do it at the speed, cost and confidence of a model like Jev is a different story. In Jev output tokens are straight up free because it's not doing text token generations.

Unofficial Hacker News client; not affiliated with Y Combinator.