There is a great blog post [0] that sohl dickstein had a couple years ago, about what happens when the metric being optimized is only a proxy for the thing you want to maximize — what happens eventually is “overfitting”. The original metric (which we cannot directly optimize for) actually starts to get worse.
I believe publishing papers and citations are those proxy metrics.
I don't like objective truth as a concept, because it makes truth a matter of belief and easily turns into just another name for social truth. If you believe that something is objectively true but new evidence challenges that view, people will often ignore the evidence, because it goes against their established views.
Science, and research in general, should mostly be concerned with effective truth. The best objective-ish explanations so far, but something that can always be challenged with new evidence. The difference to objective/social concepts of truth is that you should always be willing to question the assumptions, beliefs, and reasoning ability you and people who agree with you have. If you discover something that goes against established beliefs, you should always start by assuming that you made a mistake somewhere. And if other people discover something like that, and it looks like they are seriously questioning the discovery, you should also take it seriously.
smath · · focus · HN ↗
I believe publishing papers and citations are those proxy metrics.
[0] <a href="https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart.html" rel="nofollow">https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart....
Loquebantur · · focus · HN ↗
This isn't just "misaligned rewards", it's academia pretending to reward one thing when really it does something else entirely.
As the author observes, objective truth is no longer socially rewarded. Pretense is all that matters and accepted as "social truth".
jltsiren · · focus · HN ↗
Science, and research in general, should mostly be concerned with effective truth. The best objective-ish explanations so far, but something that can always be challenged with new evidence. The difference to objective/social concepts of truth is that you should always be willing to question the assumptions, beliefs, and reasoning ability you and people who agree with you have. If you discover something that goes against established beliefs, you should always start by assuming that you made a mistake somewhere. And if other people discover something like that, and it looks like they are seriously questioning the discovery, you should also take it seriously.
glitchc · · focus · HN ↗