Thread

MiMo v2.6

1130 points · 483 comments · volf_

syntaxing · 1790023047 · focus · HN ↗
All these new models are such tease for us folks with 128GB of shared memory. Buying another unit now to expand to 256GB is a mortgage payment but it’s getting tempting…
1. brcmthrowaway · 1790023227 · focus · HN ↗
  Is there a gamechanger around the corner to reduce DRAM requirements?
  1. stymaar · 1790023551 · focus · HN ↗
    n-gram per-layer embeddings[1][2] might be it.
    [1] <a href="https://sebastianraschka.com/llm-architecture-gallery/per-layer-embeddings/" rel="nofollow">https://sebastianraschka.com/llm-architecture-gallery/per-la...
    [2]: See DS 4.1-Flash and Qwen-3.8-Next.
    verdverm · 1790023718 · focus · HN ↗
    this is to offload VRAM to DRAM (for GP comment), and makes no difference for URAM
    zozbot234 · 1790024709 · focus · HN ↗
    You can definitely offload n-gram embeddings to storage; they're very sparsely used (only a few KB fetched per token) so this is quite effective. Loading to DRAM only becomes necessary if they are a bottleneck to overall performance (which might happen if you're doing very wide batches and everything else uses super fast VRAM/HBM).
    verdverm · 1790024874 · focus · HN ↗
    I was looking at the qwen-next-flash, and the weights would fill my OEM Spark on their own, before the n-gram. I'm unclear if offloading to disk can work here, is that what you are implying is possible?!
    girvo · 1790024933 · focus · HN ↗
    Check out eugr’s TP=1 sparkrun recipe :)
    It’s an NVFP4 quant, but it fits, and is surprisingly capable.
    verdverm · 1790025043 · focus · HN ↗
    do you have a HF link? HF search is not uncovering it for me
    (or is it somewhere else)
    verdverm · 1790026316 · focus · HN ↗
    looks like this is likely it
    <a href="https://github.com/spark-arena/eugr-recipes" rel="nofollow">https://github.com/spark-arena/eugr-recipes

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.