‹ BackHN Continuity

Thread

Packing Binary Is Fun

40 points · 17 comments · BurnerBurner

  1. ErikHuisman · · focus · HN ↗
    I also want to be a real developer so i just GZIP the JSON to make it binary.
    1. nacozarina · · focus · HN ↗
      Yeah, I’m trying to imagine how this is superior to simply adding ZIP’d JSON support to your app.
      1. entrope · · focus · HN ↗
        It depends. My personal experience is with RINEX, a text file format for GNSS observation data. gzip compresses large sets of files by about 4x. Someone named Yuki HATANAKA came up with a compact text format (CRX) that shrinks it by about a factor of 4.1x. You can gzip CRX for a total ratio of almost 12x, or bzip3 it for a total ratio of 17x. But if you combine Hatanaka's technique (essentially higher-order delta coding) with some others, you get a binary format with a compression ratio of 19x that reads faster than the CRX text.

        When one's uncompressed RINEX files are 200 GB/day, it's worth some effort to shrink the files, especially if it means they are faster to read.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.