‹ BackHN Continuity

Thread

Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms

575 points · 225 comments · firelex

  1. adrithmetiqa · · focus · HN ↗
    Forgive my lack of understanding but how long before Jev type functionality is just built straight into all frontier models?
    1. jubilanti · · focus · HN ↗
      Do people really have zero awareness that Structured Outputs with a constrained schema has been a thing for a while now, and open weight models that give you logprobs can give you distributions per key?

      Like what am I missing? I use Structured Outputs every day and this just seems like that with fewer steps?

      edit: Where I'm coming from, I can triage 10,000 support tickets with deepseek flash for less than $1, and latency is sub 1 second if it needs to be integrated into a live user flow. I don't need anything cheaper or faster than that.

      1. fooker · · focus · HN ↗
        > Like what am I missing?

        Orders of magnitude faster and cheaper answers.

        Sure a MacBook Pro can control a servo motor but maybe an arduino or a cheaper microcontroller for deployment?

        1. vengadanathan · · focus · HN ↗
          No necessarily, we are getting similar result gemma 4 12b, predicting one output token for label. it is just prefill and with nvfp4 caliberated , we are getting 70-80 ms p90 for the classification workload we have.

          not sure what made you think you cant go cheaper than jev. it is being done for a long time.

          1. fooker · · focus · HN ↗
            Can you host files cheaper than Dropbox? Of course you can. :)
            1. vengadanathan · · focus · HN ↗
              Well if there is compliance / data privacy need to it, it makes absolute sense. We operate in vocie ai industry and run in regulated environment. Jev etc doesnt support languages we want (indic languages) and for our usecase and it is not self hosted either, so running fine tuned LLM based classifier is more accurate and low latent (no external network hop as it runs in VPC). Idea is to re-use as much as possible but if compliance, accuracy becomes bottle neck it makes more sense to do it yourself.
              1. fooker · · focus · HN ↗
                Right.

                Once you discover what the useful problem to solve is and how to utilize it in production, you can of course replicate it.

                This is not alchemy taught by some wizard in secret.

                Very often, the innovation is in identifying what to build, what has interesting use cases, what people will pay for.

                Once this is established, there is further research optimizing it further, replicating it locally, etc.

                The fact that anyone could have come up with it is irrelevant.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.