Thread

1. wk_end · 1789928618 · focus · HN ↗

   From a preservation perspective, this is one of the least useful decomps ever, given how committed Capcom is to making sure RE4 is ported absolutely everywhere (kidding!)

   Made using the leaked debug build and its symbols, which shows how meaningful the work the game preservation community that acquires and distributes these things is.

   OTOH this is pretty off-putting:

   > Where the compiler needed a particular source shape to reproduce a register choice or a schedule and no natural spelling was found, the construct is marked with a // COMPILER-DIFF: comment (644 of them: dead tests, empty asm("") launders and anchors, register T x asm("rN") pins, padding statements).

   To me the value of a decomp isn't reproducing the original bytes per se (we already have the original bytes after all) - it's about reconstructing the understanding of the original game as represented by human-readable source code; getting byte-for-byte is just an indication that you've gotten it right. Needing to add a bunch of slop to force the compiler to match the original output is actually just an indication that you've gotten it wrong - and it's a demonstration of the danger of Goodhart's law, especially as it applies to AI.

   1. jchw · 1789933459 · focus · HN ↗

      Making a function fully matching by virtue of hacks like this is mostly not harmful to the ability to understand the code, but is a useful tool in ensuring that the code as a whole really is matching and identical to the original. Otherwise, it's difficult to prove.

      There is also value in decomps beyond just understanding and general interest. They can also be used to make more advanced mods, better translation patches, etc. The fidelity here matters, like having asm code accessing structures makes it hard to modify structures, but any fidelity improvement beyond pure asm is very welcome.

      Of course I still prefer to try to recover the original code that caused the compiler to do what it did, but it's a really challenging problem sometimes. I've been working on decompiling code from old versions of MSVC for literally years now and you accumulate some knowledge of what things impact register allocation or the order of symbols but some of it comes from things that get fully erased from the source. Like for example, debug builds generally seem to retain symbols that aren't actually referenced anywhere, but those symbols only actually make it into an object file if they are. For functions that were only ever inlined and not actually referenced anywhere... They still wind up in the object files and thus in debug builds, despite nothing referencing them. They are also COMDAT any'd because they can appear in multiple objects legally, which means the exact object that winds up retaining it in the final linked executable is arbitrary (and the compilation flags of the object containing it, too - I bet that was fun for developers to debug.) This is incredibly useful but very challenging, needless to say. It may even be feasible to construct examples that would be legitimately infeasible to simply guess back to equivalent source, which I suspect is a major reason why until it was finally shown to be possible in larger scale projects many people wrote fully matching decomps off as a fool's errand..

      1. Pannoniae · 1789939353 · focus · HN ↗

         With /LTCG /GL I don't think it's possible to get matching (or at least it's a very tall order), the codegen is wayy too volatile for an exact match and since inlining and reg alloc work on heuristics with thresholds it really cascades. Even stuff like what order you declare your locals in or the exact frontend syntax can mess things up...

         1. jchw · 1789940449 · focus · HN ↗

            Most people are working on builds where debug information was either inadvertently or sometimes intentionally included (e.g. for beta releases sometimes debug builds with debug info would ship for sake of making things easier.) These builds usually don't have all of the normal release optimizations on. That makes it much more likely to get a match.

            I haven't tried this, but I also suspect that once you have a lot of code fully matching, it might make it possible to ratchet your way up further into builds that you don't have debug information for, that may have more aggressive compilation options. I am not sure if you would manage to get /LTCG builds fully matching even with this advantage, but it's going to be the best shot at it. You're possibly 90% of the way there already.

            1. Pannoniae · 1789941583 · focus · HN ↗

               Hey I'm not working in matching something with LTCG atm :) I'm just saying in general.

               And yes if you have a pdb / an Od build then things are much easier, I was assuming arbitrary game i.e. release binaries.

               The "knowledge laundering" approach you describe might help in reconstructing headers, class layouts and function names which is a godsend although I don't think it would be enough to get a match. Getting functionally equivalent code is muuuch easier (although there's the problem of "how do you verify that without running every function")

               1. jchw · 1789947148 · focus · HN ↗

                  Yeah, this is probably true. I've done non-matching decompilations of modern software up to a few hundred kilobytes worth of code - it is challenging but doable. I have no idea how hard it would be to get to matching with LTCG no matter where you start from. If it was genuinely not practically possible for computational reasons I would be unsurprised.

                  AI is pretty powerful for decompilation, especially because you can also just have an LLM go and start reverse engineering bits of the linker and compiler if you want. (I suspect this decomp is AI assisted if the Clauded out README is any indication.) Maybe future models will be able to come up with clever and novel ways to reduce the number of possibilities and converge faster on possible matching source codes. Or maybe not; I think Astra is the best LLMs have ever been at decompilation and yet I find LLMs frustrating and prone to getting deeply stuck in local maxima in my experimentation.

                  1. Pannoniae · 1789977813 · focus · HN ↗

                     Yeah IMO the next big thing will be a way of verifying equivalence while filtering out the "noise" differences in optimisation. Otherwise you can't really decide whether you got it correct or not.

Unofficial Hacker News client; not affiliated with Y Combinator.