‹ BackHN Continuity

Thread

Show HN: Maple-Preview – Ternary 20B MoE running at 120 tok/s on a iPhone

169 points · 52 comments · edwardbzhang

  1. beautiful_apple · · focus · HN ↗
    A benchmark table comparing to Qwen 3.5 35B-A3B seems strange when Qwen 3.6 35B-A3B has been out for some time and is significantly better.

    I didn't notice the version difference when first reading the article! So this is a heads up to people like me.

    1. ricardobeat · · focus · HN ↗
      Their main comparison is 1-bit Bonsai 27B (Qwen3.6 27B) which beats A3B anyway.
      1. walrus01 · · focus · HN ↗
        Beats how? In my experience 1-bit bonsai 27B is quite "dumb" when asked a question about a lot of things, as a canned repository of static knowledge from its training dataset. I mean, I literally asked it for a 250 word description of Seattle and it hallucinated a tallest building in the city with an observation deck that doesn't exist, and didn't mention the Space Needle.

        Really basic stuff. But then again, the entire thing was running in <6GB of RAM.

        But before anyone says 1-bit bonsai 27B beats anything, please actually run it and ask it some questions about topics you already know the answer to.

        While Qwen 3.6 35B A3B in Q8 with full context capability (llama-server in no-mmap mode with 262k context will eat 47GB, so not comparable in size either) knows a great deal. The 35B-A3B can even translate multiple pages of English into Farsi and its Farsi output is not far off the quality of what Google Translate does.

        I haven&#x27;t tested something as badly quantized as 35B A3B Q2 which is somewhere around 12GB on disk. <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;unsloth&#x2F;Qwen3.6-35B-A3B-GGUF" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;unsloth&#x2F;Qwen3.6-35B-A3B-GGUF

        1. cmrx64 · · focus · HN ↗
          these tools are for manipulating and retrieving and transforming sequences of context. using it as a knowledgebase is just expecting the wrong thing.
          1. nxtfari · · focus · HN ↗
            Sure, then you should allow that they are also tools for transforming sequences of questions into answers. Language models are based on compression of information, using them as a knowledge base is entirely within capability.
            1. swiftcoder · · focus · HN ↗
              &gt; Language models are based on compression of information, using them as a knowledge base is entirely within capability

              It&#x27;s also the aspect of LLMs that degrades fastest with quantisation. You can&#x27;t reasonably expect accurate knowledge of everything in the world in a few gigabytes.

              Which means that small models need to be conditioned to rely more heavily on tools to fetch accurate information, and ideally not try and generate facts purely based on their (extremely lossy) internal knowledge

            2. seba_dos1 · · focus · HN ↗
              It&#x27;s kinda like lossily compressing the entire content of a CD-DA disc to fit onto a 3.5&quot; floppy disk and then complaining that it doesn&#x27;t sound very nice.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.