Thread

Can gzip be a language model?

414 points · 165 comments · networked

tromp · 1790064453 · focus · HN ↗
I'm more interested in the converse question: how well does an LLM perform as a compressor, compared to gzip (ignoring its insanely lower speed)?
1. gkbrk · 1790064714 · focus · HN ↗
  Top contestant in the Hutter Prize uses a neural network for compression. So fair to say, LLMs would perform pretty well compared to gzip.
  1. computably · 1790066098 · focus · HN ↗
    Even ignoring speed per GP, the Hutter Prize's metric includes the size of the decompressor. LLMs would be disqualified for being larger than 1GB.
    Ohentis · 1790109937 · focus · HN ↗
    Hutter prize does have speed restrictions. If it did not, LLMs would win even with counting the size of the model (which is the most reasonable choice imo) as per the main benchmark: <a href="https://www.mattmahoney.net/dc/text.html" rel="nofollow">https://www.mattmahoney.net/dc/text.html.
    computably · 1790984130 · focus · HN ↗
    Huh? The main benchmark is compressed size + program size. The uncompressed size of enwik9 is defined as 1GB, so no "L"LM qualifies.

Unofficial Hacker News client; not affiliated with Y Combinator.