‹ BackHN Continuity

Thread

Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms

575 points · 225 comments · firelex

  1. velominati · · focus · HN ↗
    Typesafe has been quite about the underlying technology behind Jev. Given the speed and cost my hypothesis is that it doesn’t input tokens the way that LLMs do, ie iterating over every word and drawing the connections between each. That is an o(n^2) problem which is why LLMs are so expensive as they scale.
    1. mrbonner · · focus · HN ↗
      My guess is that they use an encoder-only model as the foundation and then do RLDC. Why? 1) it doesn’t need text generation 2) limited context window (40k last time I check)

      Those are telltales of a Bert model.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.