If you want to try a _legit_ Jev implementation that matches (at least in my evals), the vLLM patch to turn DiffusionGemma into Jev is available.
On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).
I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.
This PR is interesting but it's making the assumption that what Jev has done is based on a diffusion model or that a diffusion model is superior for this work. Which may or may not be the case.
If I understand it though it does mean you can evaluate a bunch of questions simultaneously, which is an advantage.
Also: While I think it's expected/normal to see LLM-generated programs... there's a lot of LLM written comments in that PR, which is sad to see. Auto-human.
mmastrac · · focus · HN ↗
On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).
I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.
<a href="https://github.com/vllm-project/vllm/pull/57250" rel="nofollow">https://github.com/vllm-project/vllm/pull/57250
cmrdporcupine · · focus · HN ↗
If I understand it though it does mean you can evaluate a bunch of questions simultaneously, which is an advantage.
Also: While I think it's expected/normal to see LLM-generated programs... there's a lot of LLM written comments in that PR, which is sad to see. Auto-human.