Yeah, real Jev got really weird, no benchmarking clause. Their Terms of Use (1(v)) and MCA (2.3(f)) both prohibit users from publishing "benchmarks or performance information about the Services". No major AI has it; we are back to Oracle-style legal.
Though Jev is original, it looks highly replicable.
and frankly for many of the kind of thing people probably want to use this for... you would want to run locally anyways.
why even bother with a network hop? build a specialized engine which does the prefill->measure cycle on local GPU/TPU/NPU with a model fine tuned for your application (e.g. gaming NPCs, autonomous driving, agricultural intelligence, drone.. target... selection, whatever)
the nice thing is that if you're skipping decode you're not as memory bandwidth bound.
jakozaur · · focus · HN ↗
Though Jev is original, it looks highly replicable.
sodimel · · focus · HN ↗
cmrdporcupine · · focus · HN ↗
why even bother with a network hop? build a specialized engine which does the prefill->measure cycle on local GPU/TPU/NPU with a model fine tuned for your application (e.g. gaming NPCs, autonomous driving, agricultural intelligence, drone.. target... selection, whatever)
the nice thing is that if you're skipping decode you're not as memory bandwidth bound.
toasty228 · · focus · HN ↗