Thread

Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms

575 points · 225 comments · firelex

adrithmetiqa · 1790632786 · focus · HN ↗
Forgive my lack of understanding but how long before Jev type functionality is just built straight into all frontier models?
1. jubilanti · 1790643994 · focus · HN ↗
  Do people really have zero awareness that Structured Outputs with a constrained schema has been a thing for a while now, and open weight models that give you logprobs can give you distributions per key?
  Like what am I missing? I use Structured Outputs every day and this just seems like that with fewer steps?
  edit: Where I'm coming from, I can triage 10,000 support tickets with deepseek flash for less than $1, and latency is sub 1 second if it needs to be integrated into a live user flow. I don't need anything cheaper or faster than that.
  1. fzysingularity · 1790645513 · focus · HN ↗
    FWIW you’re absolutely correct on how most developers are unaware of this as they’re mostly operating on the API level and not aware of server side (vLLM/SGLang) capabilities.
    The aspect that I like the most is the typesafe API that introduces new probabilistic concepts that are more sound than json schema and constrained decoding with quasi-confidence scores. Developers were asking LLMs to also emit confidences which made absolutely no sense whatsoever.
    tclancy · 1790645743 · focus · HN ↗
    Can you link to some info on this? I am just starting to poke around in the space and would love a leg up.
    phoghed · 1790646859 · focus · HN ↗
    This is an example implementation that was making the rounds recently. If you point whatever model you prefer at it, it’ll do a good job of explaining it.
    No comment on this model itself, might be over fitted to the Jev benchmarks just to beat it.
    <a href="https://github.com/Mushroom-Systems/lichen" rel="nofollow">https://github.com/Mushroom-Systems/lichen

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.