Thread

1. druskacik · 1789730315 · focus · HN ↗

   I'm really interested in technical details behind Jev (not this), how it can work so fast and so cheap. It's probably large (must be since the performance is so good) but somehow still fast, so it must include some really non-trivial stuff. The price suggests it may be runnable locally, but who knows.

   If it was possible to re-create it as an open-weight, it would be exciting!

   1. cmrdporcupine · 1789736585 · focus · HN ↗

      Basically it's: skip decode, just do prefill then do some measurements. That's the crude description anyways.

      And prefill is way faster on GPU type hardware.

Unofficial Hacker News client; not affiliated with Y Combinator.