‹ BackHN Continuity

Thread

Can gzip be a language model?

414 points · 165 comments · networked

  1. Culonavirus · · focus · HN ↗
    This tracks perfectly with Winrar being more profitable than OpenAI... coincidence? I think not!
    1. wolfi1 · · focus · HN ↗
      winrar is profitable? sure? well, on the other hand, they sure don't make losses
      1. shezi · · focus · HN ↗
        They are a German GmbH and must publicly state their financials: <a href="https:&#x2F;&#x2F;www.northdata.de&#x2F;win%C2%B7rar%20GmbH,%20Berlin&#x2F;Amtsgericht%20Charlottenburg%20(Berlin)%20HRB%20109885%20B" rel="nofollow">https:&#x2F;&#x2F;www.northdata.de&#x2F;win%C2%B7rar%20GmbH,%20Berlin&#x2F;Amtsg...

        Looks pretty profitable to me.

        1. amiga386 · · focus · HN ↗
          They&#x27;re one of the few companies that actually manage to sell &quot;boxed software&quot; (i.e. has not changed much in years but new customers keep buying it)

          That said, Windows users should use 7-Zip. Better compression format, unpacks more kinds of archives

          1. xxs · · focus · HN ↗
            &gt; Windows users should use 7-Zip

            Please no - no native zstd support. NanaZip is the better option (it&#x27;s a different build of 7-zip) and it&#x27;s available at windows store.

            &gt; Better compression format, unpacks more kinds of archives

            winrar has supported zstd for 5 years[0]

            In short - Everyone should be using zstd, and 7-zip does not support it.

            [0]: <a href="https:&#x2F;&#x2F;www.win-rar.com&#x2F;singlenewsview.html?&amp;L=0&amp;tx_ttnews%5Btt_news%5D=175&amp;cHash=aca38142108c76995eba6c674bfd0e14" rel="nofollow">https:&#x2F;&#x2F;www.win-rar.com&#x2F;singlenewsview.html?&amp;L=0&amp;tx_ttnews%5...

            1. aleph_minus_one · · focus · HN ↗
              &gt; Everyone should be using zstd

              Why?

              1. shawabawa3 · · focus · HN ↗
                It&#x27;s just the best general purpose compression algorithm, in terms of compression ratio to CPU used, for the vast majority of use cases
                1. [deleted] · · focus · HN ↗

                  [deleted]

                2. Dylan16807 · · focus · HN ↗
                  I&#x27;m not that fussed about CPU use.

                  <a href="https:&#x2F;&#x2F;github.com&#x2F;mcmilk&#x2F;7-Zip-zstd" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;mcmilk&#x2F;7-Zip-zstd

                  By these charts, if I only need 5-10 megabytes per second of compression on a single core, LZMA2 wins significantly on ratio, and still decompresses at well over 100. If I&#x27;m doing a backup, or sending&#x2F;receiving over my internet connection (which only has 2MB&#x2F;s of upload), LZMA2 easily wins. If I need speed then zstd wins.

              2. xxs · · focus · HN ↗
                It&#x27;s just this good.

                On a more realistic note: few years back, I&#x27;ve added zstd compression to our log subsystem (hand written direct buffers, native code, in-process, java). For the same CPU utilization if provides twice dense compression compared to regular [-6] gzip (the topic in the title). Zstd is =much= faster on decompression as well, and it this case - unparalleledly better as it uses twice less disk.

                zstd is &#x27;silicon valley&#x27; (the tv show) - life imitates fiction, except entirely open source

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.