Thread

1. JPLeRouzic · 1789537712 · focus · HN ↗

   The author says:

   "It used Voronoi Cells and reduced most math to 8- and 16-bit integer calculations with one or two single-precision floating-point calculations."

   I wonder if it can be used to speed up LLM inference.

   1. Neywiny · 1789555892 · focus · HN ↗

      That's what quantization and stuff is, basically. So yes. Been done for years on all neural net types not just LLMs.

      1. JPLeRouzic · 1789558875 · focus · HN ↗

         Please can you provide some links to intro level articles?

         1. Neywiny · 1789593780 · focus · HN ↗

            This seems worth the read but I didn&#x27;t dig too deep <a href="https:&#x2F;&#x2F;medium.com&#x2F;@sergiopr89&#x2F;float-point-quantization-the-maths-behind-it-explained-for-everyone-fc674d313d67" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;@sergiopr89&#x2F;float-point-quantization-the-...

Unofficial Hacker News client; not affiliated with Y Combinator.