Thread

1. tromp · 1790064453 · focus · HN ↗

   I'm more interested in the converse question: how well does an LLM perform as a compressor, compared to gzip (ignoring its insanely lower speed)?

   1. alienbaby · 1790100374 · focus · HN ↗

      can they be considered to have compressed the entirety of their training data into their weights?

      1. nomel · 1790104035 · focus · HN ↗

         If you consider them lossy compression, then yes.

         The goal would be to find the minimum model that, with a fixed seed, would exactly reproduce your text.

         1. anothereng · 1790109344 · focus · HN ↗

            So I could hide information in a model basically

            1. nomel · 1790120269 · focus · HN ↗

               I think this would be the exact opposite. This model would be optimized to only output your information.

               1. anothereng · 1790266434 · focus · HN ↗

                  thats what I meant. For a given input give me the output I want without anyone knowing why that is there

Unofficial Hacker News client; not affiliated with Y Combinator.