I'm more interested in the converse question: how well does an LLM perform as a compressor, compared to gzip (ignoring its insanely lower speed)?
Top contestant in the Hutter Prize uses a neural network for compression. So fair to say, LLMs would perform pretty well compared to gzip.
And the hutter prize disallows GPU's. If you allow use of a powerful GPU, you can do quite a bit better.
tromp · · focus · HN ↗
gkbrk · · focus · HN ↗
londons_explore · · focus · HN ↗