Thread

1. metalspot · 1790094725 · focus · HN ↗

   I have done a fair amount of low level performance optimization with Opus 5 and its reasoning is still very poor. Like why is CRC so slow and going through loops until I ask it if is using hardware instructions and it tells me it is using its own hand coded implementation poor. Reasoning about l1/l2/l3 cache hit ratios and their implications basically throwing darts at the wall, in the wrong room. If you give it a benchmark feedback loop then it might get there eventually but still massive alpha for low level systems engineers who instinctively know how this stuff works and can now automate 99% of the grind.

   1. Analemma_ · 1790103885 · focus · HN ↗

      Are there actually any humans who can reason about things like cache performance from first principles? I know there are some people who think they can, and I suspect they're fooling themselves. The one iron principle of micro-optimization at the level of cache hits is "measure, measure, measure", you just cannot think your way to the right answer on the first try. Processors and instruction sets today are too complicated, and tips that worked on one generation might be neutral or worse on the very next revision, making all the cargo cult knowledge passed around on this topic at best useless. I'll echo one of the sibling commenters here and say that LLMs probably bullshit their way to answers on questions like this because that's what humans online do as well.

      If you give an LLM a proper testing harness and feedback loop to actually generate hypotheses, test them and revise them, I suspect it will do much better.

      1. jandrewrogers · 1790105984 · focus · HN ↗

         Yes, people can demonstrably do this with high reliability.

         Some humans carry detailed models of CPU microarchitectures in their heads, against which they can design code from first principles that will be nearly ideal on the first try. It is repeatable and verifiable. The best people can accurately predict the measured performance before writing a line of code.

         Measurement is useful in cases where the model of software and hardware interaction is materially incomplete. In most cases this is because the people writing the software have insufficient understanding of the hardware. Having a limited understanding of the hardware is a choice.

         It would be surprising if this wasn't possible.

      2. [deleted] · 1790111241 · focus · HN ↗

         [deleted]

Unofficial Hacker News client; not affiliated with Y Combinator.