Thread

1. pierrebarre · 1789827094 · focus · HN ↗

   Or just use <a href="https:&#x2F;&#x2F;github.com&#x2F;Barre&#x2F;ZeroFS" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;Barre&#x2F;ZeroFS

   Works pretty awesomely with git workloads, even Linux sized trees.

   1. jeremyjh · 1789841041 · focus · HN ↗

      Do you have any benchmark data you can share?

      What you are saying sounds like an assertion that the problem described here[1] - by Vicent Marti - that packfiles aren&#x27;t contiguous blocks of data and so every fetch fans out to more fetches from the client - is not actually a problem? Or is there an innovation in zerofs that somehow solves this for git? From what I can see it offers NFS and NBD interfaces and GitHub tried and rejected both of those very early on.

      The problem he described seems consistent with known patterns from datalake storage architectures, its a big part of why columnar storage is the only game in town for most use cases.

      This is why TFA has a lot of interest here - he has changed the storage layout of git packfiles to a columnar model to address this challenge.

      [1] <a href="https:&#x2F;&#x2F;cursor.com&#x2F;blog&#x2F;git-at-any-scale#git-without-packfiles" rel="nofollow">https:&#x2F;&#x2F;cursor.com&#x2F;blog&#x2F;git-at-any-scale#git-without-packfil...

      1. pierrebarre · 1789843348 · focus · HN ↗

         I’m not saying the access pattern isn’t real. I’m saying it largely doesn’t matter in practice.

         If Git repos were routinely terabytes in size, this would be a much harder problem. But the vast majority of repos are &lt;1GB, and at that scale caching dominates. Even with imperfect locality, you can cache enough of the working set that those reads don’t turn into s3 round trips.

         In a low-latency environment, I haven’t found filesystem round trips themselves to be an issue for Git workloads either.

Unofficial Hacker News client; not affiliated with Y Combinator.