If you want to try a _legit_ Jev implementation that matches (at least in my evals), the vLLM patch to turn DiffusionGemma into Jev is available.
On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).
I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.
This is very interesting! Seems like a promising direction.
I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?
Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?
Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?
> It seems by sharing KV cache all questions will be visible
Yeah, this is partially why in my approach I've done this instead, and not used diffusion model:
1. Convert the state into one shared prompt.
2. Run that shared prompt through the model once.
3. Fork the model’s internal state once per question.
4. Add a different question to each fork.
5. Ask each fork for its next-token scores.
6. Calculate only 64 possible label scores—not the whole vocabulary.
Basically ... skip decode.
Won't be as fast as doing diffusion model parallel across a pile of questions at once, but:
a) let's you use pretty much any existing text model (with some modifications). I've got qwen3.6 moe and qwen3.8 flash next running, am getting gemma4 working now
b) the problem you identified
It's possible I'm getting high on my own supply and misunderstand entirely the whole thing, but it seems to work?
mmastrac · · focus · HN ↗
On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).
I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.
<a href="https://github.com/vllm-project/vllm/pull/57250" rel="nofollow">https://github.com/vllm-project/vllm/pull/57250
mungoman2 · · focus · HN ↗
I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?
Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?
Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?
cmrdporcupine · · focus · HN ↗
Yeah, this is partially why in my approach I've done this instead, and not used diffusion model:
1. Convert the state into one shared prompt.
2. Run that shared prompt through the model once.
3. Fork the model’s internal state once per question.
4. Add a different question to each fork.
5. Ask each fork for its next-token scores.
6. Calculate only 64 possible label scores—not the whole vocabulary.
Basically ... skip decode.
Won't be as fast as doing diffusion model parallel across a pile of questions at once, but:
a) let's you use pretty much any existing text model (with some modifications). I've got qwen3.6 moe and qwen3.8 flash next running, am getting gemma4 working now
b) the problem you identified
It's possible I'm getting high on my own supply and misunderstand entirely the whole thing, but it seems to work?
<a href="https://github.com/rdaum/eider/commit/9c2d5c049068c33da2a48bc9f131f4b4dd868b92" rel="nofollow">https://github.com/rdaum/eider/commit/9c2d5c049068c33da2a48b...
I don't have the chutzpah to go creating PRs for vLLM to do the same.
aaquibahm · · focus · HN ↗
cmrdporcupine · · focus · HN ↗
But only works for Gemma4, or other attention-only models. (i.e. not the Qwen3.8 models I had working with the other way)
I'm getting 63ms per answer for 1 question, 79 for 3, 93.7ms for 5. So scales nicely, too. ~55ms fixed cost + ~7-8ms per additional question.
Much better. Thanks. I'll commit and share work after testing it some more.
EDIT: how can I credit you in the commit body?
aaquibahm · · focus · HN ↗