I built something like this just last month, using a few of the same models, but I used ThreeJS's Three-Shading-Language abstraction to do it: <a href="https://three-llm.ben3d.ca/?model=qwen3.5-0.8b" rel="nofollow">https://three-llm.ben3d.ca/?model=qwen3.5-0.8b
Awesome! I tried it and after loading (which took a while as it is a large model) got 12 tokens/second and very coherent output. Great demonstration.
I would recommend adding the MiniCPM5-1B model. Surprisingly coherent for a 1B model and performs well. On my pixel 9 I get 33 tok/s on CPU. On GPU I get about 26 tok/s but prefill jumps to nearly 500.
bhouston · · focus · HN ↗
logicallee · · focus · HN ↗
sourweasel · · focus · HN ↗