If you want to try a _legit_ Jev implementation that matches (at least in my evals), the vLLM patch to turn DiffusionGemma into Jev is available.
On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).
I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.
This is very interesting! Seems like a promising direction.
I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?
Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?
Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?
> It seems by sharing KV cache all questions will be visible
Yeah, this is partially why in my approach I've done this instead, and not used diffusion model:
1. Convert the state into one shared prompt.
2. Run that shared prompt through the model once.
3. Fork the model’s internal state once per question.
4. Add a different question to each fork.
5. Ask each fork for its next-token scores.
6. Calculate only 64 possible label scores—not the whole vocabulary.
Basically ... skip decode.
Won't be as fast as doing diffusion model parallel across a pile of questions at once, but:
a) let's you use pretty much any existing text model (with some modifications). I've got qwen3.6 moe and qwen3.8 flash next running, am getting gemma4 working now
b) the problem you identified
It's possible I'm getting high on my own supply and misunderstand entirely the whole thing, but it seems to work?
I think you're right. Better. I will have to think through if it would be faster or slower. If understand what you're getting at with this.. broken analogy...
What I described was -- we have a bunch of orders to the kitchen, all of which have the same "base" meal but different topics.
> Cook the "base" meal once, divide servings onto multiple plates, then add different toppings to each plate.
vs what you suggest:
> Put the base meal and every topping through the kitchen together, but use some kind of dividers so the toppings never mix up together.
mmastrac · · focus · HN ↗
On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).
I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.
<a href="https://github.com/vllm-project/vllm/pull/57250" rel="nofollow">https://github.com/vllm-project/vllm/pull/57250
mungoman2 · · focus · HN ↗
I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?
Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?
Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?
cmrdporcupine · · focus · HN ↗
Yeah, this is partially why in my approach I've done this instead, and not used diffusion model:
1. Convert the state into one shared prompt.
2. Run that shared prompt through the model once.
3. Fork the model’s internal state once per question.
4. Add a different question to each fork.
5. Ask each fork for its next-token scores.
6. Calculate only 64 possible label scores—not the whole vocabulary.
Basically ... skip decode.
Won't be as fast as doing diffusion model parallel across a pile of questions at once, but:
a) let's you use pretty much any existing text model (with some modifications). I've got qwen3.6 moe and qwen3.8 flash next running, am getting gemma4 working now
b) the problem you identified
It's possible I'm getting high on my own supply and misunderstand entirely the whole thing, but it seems to work?
<a href="https://github.com/rdaum/eider/commit/9c2d5c049068c33da2a48bc9f131f4b4dd868b92" rel="nofollow">https://github.com/rdaum/eider/commit/9c2d5c049068c33da2a48b...
I don't have the chutzpah to go creating PRs for vLLM to do the same.
aaquibahm · · focus · HN ↗
cmrdporcupine · · focus · HN ↗
What I described was -- we have a bunch of orders to the kitchen, all of which have the same "base" meal but different topics.
> Cook the "base" meal once, divide servings onto multiple plates, then add different toppings to each plate.
vs what you suggest:
> Put the base meal and every topping through the kitchen together, but use some kind of dividers so the toppings never mix up together.
Except.. ok, that analogy is confusing lol.