Thread

1. wk_end · 1789928618 · focus · HN ↗

   From a preservation perspective, this is one of the least useful decomps ever, given how committed Capcom is to making sure RE4 is ported absolutely everywhere (kidding!)

   Made using the leaked debug build and its symbols, which shows how meaningful the work the game preservation community that acquires and distributes these things is.

   OTOH this is pretty off-putting:

   > Where the compiler needed a particular source shape to reproduce a register choice or a schedule and no natural spelling was found, the construct is marked with a // COMPILER-DIFF: comment (644 of them: dead tests, empty asm("") launders and anchors, register T x asm("rN") pins, padding statements).

   To me the value of a decomp isn't reproducing the original bytes per se (we already have the original bytes after all) - it's about reconstructing the understanding of the original game as represented by human-readable source code; getting byte-for-byte is just an indication that you've gotten it right. Needing to add a bunch of slop to force the compiler to match the original output is actually just an indication that you've gotten it wrong - and it's a demonstration of the danger of Goodhart's law, especially as it applies to AI.

   1. jchw · 1789933459 · focus · HN ↗

      Making a function fully matching by virtue of hacks like this is mostly not harmful to the ability to understand the code, but is a useful tool in ensuring that the code as a whole really is matching and identical to the original. Otherwise, it's difficult to prove.

      There is also value in decomps beyond just understanding and general interest. They can also be used to make more advanced mods, better translation patches, etc. The fidelity here matters, like having asm code accessing structures makes it hard to modify structures, but any fidelity improvement beyond pure asm is very welcome.

      Of course I still prefer to try to recover the original code that caused the compiler to do what it did, but it's a really challenging problem sometimes. I've been working on decompiling code from old versions of MSVC for literally years now and you accumulate some knowledge of what things impact register allocation or the order of symbols but some of it comes from things that get fully erased from the source. Like for example, debug builds generally seem to retain symbols that aren't actually referenced anywhere, but those symbols only actually make it into an object file if they are. For functions that were only ever inlined and not actually referenced anywhere... They still wind up in the object files and thus in debug builds, despite nothing referencing them. They are also COMDAT any'd because they can appear in multiple objects legally, which means the exact object that winds up retaining it in the final linked executable is arbitrary (and the compilation flags of the object containing it, too - I bet that was fun for developers to debug.) This is incredibly useful but very challenging, needless to say. It may even be feasible to construct examples that would be legitimately infeasible to simply guess back to equivalent source, which I suspect is a major reason why until it was finally shown to be possible in larger scale projects many people wrote fully matching decomps off as a fool's errand..

      1. StilesCrisis · 1789960101 · focus · HN ↗

         Working on a decomp hobby project now, and "100% matching" on an individual function can be so deceptive! These two functions could look identical in assembly:

         void foo() { bar(); }

         int foo(int x, float y) { return bar(x, y); }

         This is because parameter-passing and return-value handling can sometimes be done in zero instructions in PPC, if the values are already in the right registers. So you need to continually go back and reassess old functions once you learn the signatures of newer ones!

         1. jchw · 1789961796 · focus · HN ↗

            Oh yeah - absolutely. This isn't even just a PowerPC thing - it happens more generally, because usually if you're forwarding both arguments and return values for a function that has the same calling conventions, you don't really have to do much adaptation, everything was already set up for you. It hit me a ton on x86 when I first started my journey into trying to reverse engineer. It still occasionally hits me, especially when I'm relying too hard on decompiler output and not paying enough attention.

            There are plenty of examples where two very semantically different source codes can have the same output, which is really counter-intuitive when combined with the difficulty that often comes with trying to find a single source code that does match.

            This is just another reason why debug information is such a godsend; having symbol names for C++ code will usually give you most of the function signature, and type information will give you the rest. That greatly enriches the disassembly, and the automated decompilation output, and no doubt constrains the number of possible source codes that could match both the output and the debug information, somewhat alleviating this issue.

            1. StilesCrisis · 1789993341 · focus · HN ↗

               Yup. Unfortunately the project I'm working on does not have any debug info. There are not even lingering __FILE__ strings anywhere. Just picking out the TU bounds has been a chore!

               It's a huge benefit that things like the OS and SDK are known and predictable. This got me a foothold. Otherwise I would have gotten nowhere.

               I'm still impressed that this decomp has macros (fully lost in the assembly) and nearly-100% meaningful variable names and struct layouts. That's not easy!

               1. jchw · 1789994426 · focus · HN ↗

                  How in the world are you finding reasonable TU boundaries, without debug info? I've always been puzzled by this in particular.

                  1. StilesCrisis · 1789995902 · focus · HN ↗

                     Float constants are stored as data which is deduplicated on a per-TU basis. So if my TU has 1.0 and 0.0 in it, every function in that TU which uses those numbers will load from the same address. The next TU will get its own dedicated 1.0 and 0.0 constants.

                     1. jchw · 1789998751 · focus · HN ↗

                        That is pretty clever. In a particularly C++ heavy program I was able to find decent splits by starting around where the vtables were stored and searching from there, which was helpful since they also often pointed into the relevant part of the .text section. The repeating floats thing definitely seems to ring true though. For old MSVC it seems like it will emit floats even if they aren't directly referenced, if they happen to be referenced by inline functions that are not used. That adds another challenge if you want to get really close to the actual original source: figuring out what those inline functions actually are instead of just dropping a dummy that references them and is never used :) But I'd say that's extra credit, if you ever got to the point where that was the only remaining debt...

                        1. StilesCrisis · 1790001411 · focus · HN ↗

                           For sure, actually mirroring the shape of inlined code properly rather than just emitting code N times is already extra credit as far as I'm concerned. I've got enough loose ends without chasing that sort of thing!

Unofficial Hacker News client; not affiliated with Y Combinator.