If you want to try a _legit_ Jev implementation that matches (at least in my evals), the vLLM patch to turn DiffusionGemma into Jev is available.
On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).
I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.
This is very interesting! Seems like a promising direction.
I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?
Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?
Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?
> It seems by sharing KV cache all questions will be visible
Yeah, this is partially why in my approach I've done this instead, and not used diffusion model:
1. Convert the state into one shared prompt.
2. Run that shared prompt through the model once.
3. Fork the model’s internal state once per question.
4. Add a different question to each fork.
5. Ask each fork for its next-token scores.
6. Calculate only 64 possible label scores—not the whole vocabulary.
Basically ... skip decode.
Won't be as fast as doing diffusion model parallel across a pile of questions at once, but:
a) let's you use pretty much any existing text model (with some modifications). I've got qwen3.6 moe and qwen3.8 flash next running, am getting gemma4 working now
b) the problem you identified
It's possible I'm getting high on my own supply and misunderstand entirely the whole thing, but it seems to work?
So the model normally (like during a normal decode) takes its final hidden vector and multiplies it by the entire vocabulary head -- so like about 250K rows -- to produce one logit per possible next token (and then so on and so on...)
Instead I use a fixed set of up to only 64 single-token labels. At model load, I gather only those 64 rows from the vocabulary head into a small matrix. Each question maps its permitted answers onto some of those labels.
It is a probability distribution conditional on the allowed labels. Calibration is a separate problem that I have to solve still and will be model specific :-) But I do seem to get reasonable answers right now.
So "64" is just in the end the endpoint’s maximum answer-label set. Most questions use only two or three of those rows. And, yeah, some calibration required. WIP on that
mmastrac · · focus · HN ↗
On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).
I ran the same evals against a Qwen36 and it clearly lost to both of them, so you are leaving both knowledge and instinctual reasoning on the table with any smaller models, FWIW.
<a href="https://github.com/vllm-project/vllm/pull/57250" rel="nofollow">https://github.com/vllm-project/vllm/pull/57250
mungoman2 · · focus · HN ↗
I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?
Also I think without fine-tuning we can not say that the probabilities it output are actually probabilities. Maybe fine tuning using Brier scoring would do the trick?
Maybe some kind of hierarchical structure of the KV cache can make questions independent, and with smaller diffusion canvas’ generated in batch can be a way to make answers generate independently?
cmrdporcupine · · focus · HN ↗
Yeah, this is partially why in my approach I've done this instead, and not used diffusion model:
1. Convert the state into one shared prompt.
2. Run that shared prompt through the model once.
3. Fork the model’s internal state once per question.
4. Add a different question to each fork.
5. Ask each fork for its next-token scores.
6. Calculate only 64 possible label scores—not the whole vocabulary.
Basically ... skip decode.
Won't be as fast as doing diffusion model parallel across a pile of questions at once, but:
a) let's you use pretty much any existing text model (with some modifications). I've got qwen3.6 moe and qwen3.8 flash next running, am getting gemma4 working now
b) the problem you identified
It's possible I'm getting high on my own supply and misunderstand entirely the whole thing, but it seems to work?
<a href="https://github.com/rdaum/eider/commit/9c2d5c049068c33da2a48bc9f131f4b4dd868b92" rel="nofollow">https://github.com/rdaum/eider/commit/9c2d5c049068c33da2a48b...
I don't have the chutzpah to go creating PRs for vLLM to do the same.
mungoman2 · · focus · HN ↗
> 6. Calculate only 64 possible label scores—not the whole vocabulary.
This I don’t understand though, could you expand this please?
cmrdporcupine · · focus · HN ↗
Instead I use a fixed set of up to only 64 single-token labels. At model load, I gather only those 64 rows from the vocabulary head into a small matrix. Each question maps its permitted answers onto some of those labels.
It is a probability distribution conditional on the allowed labels. Calibration is a separate problem that I have to solve still and will be model specific :-) But I do seem to get reasonable answers right now.
So "64" is just in the end the endpoint’s maximum answer-label set. Most questions use only two or three of those rows. And, yeah, some calibration required. WIP on that