Thread

1. adrithmetiqa · 1790632786 · focus · HN ↗

   Forgive my lack of understanding but how long before Jev type functionality is just built straight into all frontier models?

   1. bigyabai · 1790633193 · focus · HN ↗

      BeRT and FLAN-T5 were used as classifiers 5-7 years ago, they were technically "frontier" for their time.

      1. alanwreath · 1790633851 · focus · HN ↗

         This is the exact comment I’ve been waiting for, what is the difference between classifiers and jev?

         1. prometheus1992 · 1790634043 · focus · HN ↗

            nothing in utility. we used various bert variants to satisfy our usecases and still in uses. free and they run locally.
         2. yfontana · 1790634751 · focus · HN ↗

            Jev is a classifier. The big thing about it is that it has high accuracy on domains it wasn't fine-tuned for, like an LLM, but with speed and cost comparable to traditional classifiers.
         3. fra · 1790635060 · focus · HN ↗

            BERT need to be fine tuned for your use case, Jev generalizes. It’s a pretty big difference!

            1. latentsea · 1790650820 · focus · HN ↗

               So what you're saying is it's artificial... general... intelligence? /s
         4. make3 · 1790636268 · focus · HN ↗

            FLAN-T5 generated text (Jev does not generate text), and BERT wasn't able to do tasks without fine-tuning.

            Jev is basically a kind of FLAN-BERT, if you want, where it has built-in multi-task ability, but doesn't generate text. It only generates 255 floats all at once, making it much faster, and what those floats mean (if anything) depends on the prompt.

            Eg, the following query is put in the encoder model:

            {"question": "Rank these 5 things by increasing order of how big they are", "choices": ["truck", "cow", "mouse", "ant", "building"] }

            The model returns [3., 2., 1., 0., 4.], and 249 other meaningless floats that are hidden from you by the UI.

            The UI stitches the first 5 floats with the choices and returns something like:

            {"rank": ["ant", "mouse", "cow", "truck", "tower"]}

            1. dannyw · 1790638660 · focus · HN ↗

               So it generates logits in a 255 token output space? ;)

               1. make3 · 1790642685 · focus · HN ↗

                  logits assumed some form of softmax or logistic, which may not be the case
            2. vlovich123 · 1790641854 · focus · HN ↗

               How does it know that you’re asking for “rank” instead of something else if it’s not generating text?

               1. jmalicki · 1790644592 · focus · HN ↗

                  By reading, not generating, text.

                  1. vlovich123 · 1790651728 · focus · HN ↗

                     It has to output “rank” - that requires generation unless I’m mistaken

                     1. make3 · 1790653312 · focus · HN ↗

                        no, 255 numbers come out at once all the time, the order is determined by the inputs

                        1. vlovich123 · 1790653792 · focus · HN ↗

                           That’s not what I’m asking about - the text prompted for rank and it output “rank” in the response object. How did it do that? Like if I’d asked it to group similar items, how would it know how to structure that output and know that the key should be “groupings”

                           1. pests · 1790665974 · focus · HN ↗

                              [dead]

                           2. make3 · 1790699084 · focus · HN ↗

                              The output key is determined through code by the harness from the inputs, and the model generates 255 floats in order that follow that schema by reading the expected return type "rank" in the input. The harness then programmatically uses the floats of the 255 floats that are useful. The harness can assume that the correct float will be in the correct position as the model is trained to follow the schemas
            3. aftbit · 1790649425 · focus · HN ↗

               Is Jev's architecture public somewhere? I'd love to read more about it.

               1. Rzor · 1790650132 · focus · HN ↗

                  <a href="https:&#x2F;&#x2F;laya.convaiinnovations.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;laya.convaiinnovations.com&#x2F;

                  1. aftbit · 1790693728 · focus · HN ↗

                     That&#x27;s one of the copycats. They reproduced the shape of System One decisions, but who knows if they did it in the same way.
         5. _menelaus · 1790646466 · focus · HN ↗

            The fact that its not narrow and stupid is what&#x27;s different

Unofficial Hacker News client; not affiliated with Y Combinator.