There is more to code review than (automatable) detection
Thread
Unofficial Hacker News client; not affiliated with Y Combinator.
There is more to code review than (automatable) detection
Unofficial Hacker News client; not affiliated with Y Combinator.
dimbletimbers · · focus · HN ↗
atomicnumber3 · · focus · HN ↗
I'm tech lead and I basically don't review PRs, and I tell people this, with a caveat - if you can tell me what you specifically want me to review, for what specific purpose, I'm happy to!
So "can you review this bit for race conditions" is great, love it. This forces people to actually think about what in their code they should be suspicious of, if anything.
"Can you review this" [link to PR] is getting a rubber stamp because humans have never been good enough at "just spotting bugs" to make this worth it and now any LLM is better than a human.
And for the purpose of understanding - PR time is too late. I have not reviewed a PR ("for real") in a long time and yet I could tell you how every system my people have built works down to a very fine level of detail. And it's because _we talk to each other!_ We don't just chill in the same slack channel and code independently, we all value each others brains and want each others inputs because we know it will improve our product and we value what perspectives others will bring.
Trying to learn via PR is a sad substitute for real collaboration and teamwork.
regularfry · · focus · HN ↗
> "Can you review this" [link to PR] is getting a rubber stamp because humans have never been good enough at "just spotting bugs" to make this worth it and now any LLM is better than a human.
Spotting bugs is the one thing we do have evidence that code inspection is good for. But there's a massive difference between the type of code review there's good evidence for and a github-style PR review, so it's mixed but not entirely without foundation.
tonkkatonka · · focus · HN ↗
Quoting from this article: <a href="https://thenewstack.io/kill-code-review-theater/" rel="nofollow">https://thenewstack.io/kill-code-review-theater/
Ask engineers why they review code, and most will say they do it to find bugs. In 2013, Alberto Bacchelli and Christian Bird studied code review at Microsoft and classified 570 review comments. 44% of developers ranked finding defects as their top reason for reviewing. Only 14% of the actual comments were about defects. Five years later, Caitlin Sadowski and her colleagues looked at nine million reviewed changes at Google and reached the same conclusion.
regularfry · · focus · HN ↗
There are papers that show positive results for Fagan inspection (1976 onwards) and I'd put good money on there being a straight line between IBM having good results with that method and more recent devs believing that code review is good for finding bugs, despite the loss of a formal structure.
> we think that code review is good for spotting bugs but what it's actually good for is knowledge sharing.
It can be both! I'm not sure it is, but there's nothing contradictory in that.
> Ask engineers why they review code, and most will say they do it to find bugs. In 2013, Alberto Bacchelli and Christian Bird studied code review at Microsoft and classified 570 review comments. 44% of developers ranked finding defects as their top reason for reviewing. Only 14% of the actual comments were about defects. Five years later, Caitlin Sadowski and her colleagues looked at nine million reviewed changes at Google and reached the same conclusion.
Proportion of comments tells you nothing about whether it's a good technique for finding bugs. It just tells you what the spread of comments is across the different purposes for which the channel is being used.