Thread

Git 3.0's upcoming SHA-256 default will be a costly mistake

570 points · 536 comments · chmaynard

amluto · 1790875631 · focus · HN ↗
I don't understand why Git is not making the SHA-1 and SHA-256 modes far more compatible with each other.
SHA1-hashed objects should be able to refer to SHA-256-hashed objects, although this seems somewhat pointless.
But SHA-256-hashed objects should also be able to refer to SHA1-hashed objects, with a major caveat: if those objects themselves are part of a collision pair, then there is a genuine problem. But this is avoidable! Suppose that Linux decided to migrate to SHA-256. The upstream project could choose a pair of dates, say January 1 2027 and March 1 2027. Up to the first date, maintainers would be welcome to submit hashes of objects that are not yet in the repo but that they think they might submit later on, and, on that date, the upstream tree would finalize the list of these objects and reference it in the repo (with a new mechanism for this purpose). Effective the second date, the repo would start publishing SHA-256 commits and would never again accept a SHA1-hashed object that was not in the repo at the cutoff date or referenced as part of the Jan 1 block.
And now it would be impossible to get a new SHA1 collision in to the repo.
The only new git features needed would be:
a) actual compatibility so that a SHA-256-hashed object could reference a SHA1-hashed object
b) a new object type that's a list of allowed SHA1 hashes (or probably a tree of them) that is itself hashed with SHA-256 and a mechanism to link to one of these from a commit
c) a policy mechanism to set a repo to only allow SHA1-hashed-objects that a reachable from a preconfigured SHA-256-hashed commit
1. schacon · 1790875830 · focus · HN ↗
  Emily's talk does a pretty good job of summarizing the issues with intermixing the hashes: <a href="https://youtu.be/eJJp0RE7cd4" rel="nofollow">https://youtu.be/eJJp0RE7cd4
  1. RJIb8RBYxzAMX9u · 1790877675 · focus · HN ↗
    I skimmed the video, and I didn't quite catch that. Near the end of the video, however, she did mention that interop is in the works[0].
    In any case, even if Git 3.0 were completely incompatible, it would suck, but it's not the end of the world. You just treat it as if you were migrating from one SCM system to another. CVS -> SVN -> Perforce -> Git -> Git 3.0 -> [...] been-there-done-that. This is something that both open-source and commercial projects have had to deal with over the years.
    Or maybe it would be a repeat of Python 2.x -> 3.x. ¯\_(ツ)_/¯ With AI assistance, hopefully porting the tooling over may go a lot quicker and smoother.
    [0] <a href="https://www.youtube.com/watch?v=eJJp0RE7cd4&t=1134s" rel="nofollow">https://www.youtube.com/watch?v=eJJp0RE7cd4&t=1134s
    ba1afd89f34cb23 · 1790877912 · focus · HN ↗
    Her entire section 2 (starting at 6:12) is basically about why git does not and will never allow mixing of SHA1 and SHA256.
    The interop discussed is using copybara as a copy tool to move data from SHA1 based repos to SHA256 based repos and vice versa.
    RJIb8RBYxzAMX9u · 1790879605 · focus · HN ↗
    Thanks. I went back and re-watched that part, and her point's that "[a] tree's cryptographic strength is equal to the weakest hash algorithm anywhere in the tree," and that's fair. I also understand why Git 3.0 may not want to give user the choice, though I would still rather it be given.
    schacon · 1790880014 · focus · HN ↗
    They do give the user the choice, but the default is changing. My point is not necessarily to rip out the SHA-256 option, but simply to not make it the default. Because then people will create repos in that format that do not understand the ramifications, where the opposite should be true.
    amluto · 1790882982 · focus · HN ↗
    > "[a] tree's cryptographic strength is equal to the weakest hash algorithm anywhere in the tree"
    This is simply wrong IMO, for two reasons:
    1. The attack on SHA-1 is a collision attack. Once you have frozen a hash, you cannot attack it with existing cryptanalysis. If there were a preimage attack it would be a different story.
    2. Even if there were preimage attacks, one could freeze a mapping from SHA1 hash to SHA-256 hash.
    In fact, #2 seems like en excellent design. Objects could reference such a mapping, and a repo could disallow conflicting mappings (the mappings would only be accepted if the mapped objects are reachable from the mapping and the mapping is correct).
    angry_octet · 1791027141 · focus · HN ↗
    There were a number of unsupported assertions in the talk, and the lack of nuance in discussion made it seem like an announcement rather than a technical justification.
    schacon · 1790879334 · focus · HN ↗
    My point is not that it's the end of the world (or the end of Git), but that it will be painful and unclear and confusing to lots of people. That would be fine if it made a huge difference in trust or protection, but it's the wrong way to do that.
  2. angry_octet · 1791026998 · focus · HN ↗
    They begin by saying mixing can never work but don't address the option of dual hashes... Then the interop plan is basically a secret git format that maintains both sha1 and sha256 hashes FOREVER.
    I don't see why all users wouldn't want to keep both hashes.
2. mort96 · 1790879692 · focus · HN ↗
  Hm but the date is stored inside of the commit. The only way we can know that a commit's date is authentic... is through its hash. If I can forge commits with any SHA1 hash at will, I can make a repository whose head commit has the same SHA1 as the one in torvalds: /linux but where any commit was replaced by a malicious commit with the same SHA1 and a fake date. You have no way to detect that my repo is inauthentic other than through a deep history comparison. The whole idea behind a merkle tree is that just checking the hash of the top is sufficient to know the identity of the whole tree.
  I don't know what the solution is, but I'm inclined to believe that any repo with a single SHA1 commit is as weak as a repo with all SHA1 commits.
  1. amluto · 1790883039 · focus · HN ↗
    The date that a repo receives a commit is known to that repo. And a repo can stop accepting new SHA1 objects. And a SHA256 object could have a flag that says that no SHA1 objects may ever reference it.
    mort96 · 1790885099 · focus · HN ↗
    The design of Git, as a Merkle tree, is meant to allow for use cases like this:
    * I host a mirror of the Linux git repo.
    * You download Linux from my mirror.
    * You check out a commit, say fd179f8a05be3ccae366b9b96e176b51fbe54aab, which you know is a genuine commit through some out-of-band mechanism (mailing list, GitHub web interface, a line in a Nix file, whatever).
    * You check whether the repository I gave you is legitimate or not by re-computing the hash of the commit which I claimed was fd179f8a05be3ccae366b9b96e176b51fbe54aab. If it comes out to be fd179f8a05be3ccae366b9b96e176b51fbe54aab, you know it's legitimate. If it doesn't, you know it's fake.
    This is a completely normal use of Git. People download from mirrors all the time. People rely on commit hashes to identify a specific source tree. People trust that if whatever the mirror gave them hashes to the right value, it's genuine. That way, you don't have to trust the mirror.
    If I can forge my own commits to have any hash I want, this whole model breaks down. I can replace some old commit in the repo with my own forged commit with the same hash, and when you download a copy of the Linux repo from my mirror, you'll receive a repo with malicious content, but it'll hash to the same fd179f8a05be3ccae366b9b96e176b51fbe54aab hash as a genuine repo would. This breaks the security model of Git.
    amluto · 1790888518 · focus · HN ↗
    > You check out a commit, say fd179f8a05be3ccae366b9b96e176b51fbe54aab, which you know is a genuine commit through some out-of-band mechanism
    That's a 160 bit hash, which is SHA-1, which has the security properties of SHA-1.
    Suppose you check out a commit with a given SHA-256 hash. That commit object represent the root of a tree where all the edges are hashes (and types, etc). I'm suggesting one of two designs:
    a) (Simpler but weaker) If Linus has published that commit, then he is confident that he hasn't pulled in any too-new SHA-1 hashes and that there are no collisions present in what he thinks the tree is. So, by induction on the traversal depth, there is only one actual object identified by each edge, and those objects contain the hashes of their child edges, so those hashes are all correct.
    This breaks if there is a malicious collision already in the tree.
    b) (Stronger but higher overhead and more complex) There would be an object or objects, discoverable from the root by following only SHA-256 edges, that encode a duplicate-free mapping from SHA-1 hash to SHA-256 hash. The client finds and parses that and then, as it traverses the tree, each time it reads a SHA-1 hash, it computes the SHA-1 and SHA-256 hash of the referenced object, verifies that the pair is in the mapping and also verifies that the SHA-1 hash matches what the edge requires.
    I think that (b) is genuinely cryptographically secure in the sense that, if you can construct a commit that has the same SHA-256 hash as an official upstream commit but different contents, then there is necessarily a SHA-256 collision.
    mort96 · 1790891716 · focus · HN ↗
    For A), I don't understand what the point is? I never mentioned what Linus is confident about, I talked about what you can verify when you pull from my mirror. I could replace a commit from 2010 with a malicious one
    For B), I would think this could work, but it's a completely different solution from what you proposed and what I responded to.
    amluto · 1790896682 · focus · HN ↗
    > I could replace a commit from 2010 with a malicious one
    How? Remember, there are (currently, anyway) no known SHA-1 preimage attacks.
    mort96 · 1790897878 · focus · HN ↗
    We're discussing a hypothetical situation where SHA-1 gets even more broken. From my original comment in this thread (<a href="https://news.ycombinator.com/item?id=49924179#49925367">https://news.ycombinator.com/item?id=49924179#49925367):
    > If I can forge commits with any SHA1 hash at will
    amluto · 1790898846 · focus · HN ↗
    This is a fair point.
    I think I stand by my second proposal. I also think it's absurd that, after all these years, upstream git still can't figure out a credible migration plan.
    someonebaggy · 1790941818 · focus · HN ↗
    Maybe from your mirror. The collision attack that's been discovered is not a preimage attack though. It generates two colliding objects with the same uncontrollable hash, so you can't replace an existing hashed object, you have to sneak in one half of the pair.
    More importantly we just shouldn't use your mirror if we don't trust you. If you're evil you're probably lying about all the tags and branch tips anyway.
  2. PunchyHamster · 1790898838 · focus · HN ↗
    If there is one way enforcement (i.e. there is one point where the last SHA1 commit was signed by first SHA256 commit), I think it should be safe ?
    The "commit before" might be compromised, but the git commits refer a snapshot of a tree + a list of previous commit IDs, so the "new" SHA256 commit will not have any files altered
    samus · 1790920799 · focus · HN ↗
    Every file (indirectly) referred to by a SHA256 commit using a SHA1 hash in some tree object can still be spoofed. Fixing that requires rehashing all objects and recreating all tree objects so the tree objects referred to by SHA256 commits are purely made up of object references computed by SHA256.
    gsnedders · 1790961754 · focus · HN ↗
    My reading of the docs is that when fetching/pushing from a SHA-256 repo, all objects are referred to (in the packfile fetched) by their SHA-256 names — then, when fetching, you locally compute the SHA-1 of each object for the translation table.
    Presuming there’s some validation that SHA-1 names are unique, then that should be safe — the only way I can see one could do a pre-image attack is either fetching from a SHA-1 server (because then you don’t get the SHA-256 object name), which requires a second pre-image attack on SHA-1 (known to be feasible); or by having a second pre-image attack against both SHA-1 and SHA-256 simultaneously (and SHA-256 is still believed to be secure).

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.