Thread

1. infogulch · 1789600676 · focus · HN ↗

   So they get down from 1.58 to 1.48 bits per weight by exploiting the fact that actual weights in practice are 0 51% of the time. Neat.

   If ternary llms work out and are baked into hardware as custom silicon I bet they'll be shockingly efficient.

   1. kadushka · 1789601079 · focus · HN ↗

      By “work out” you mean no accuracy degradation? That’s a big ask - currently we can barely quantize to dynamic fp4 with small block size - still not completely lossless on all benchmarks.

      1. montroser · 1789603562 · focus · HN ↗

         Well, you could train directly at this bitrate.

         1. brookst · 1789652644 · focus · HN ↗

            Not an expert, but doesn’t that produce lower quality results, the same way a 1MP image isn’t lower quality than a 20mp image downscaled to 1MP? (Everything else equal)

Unofficial Hacker News client; not affiliated with Y Combinator.