Thread

1. ratelimitsteve · 1789664939 · focus · HN ↗

   If 10,000 people guess 10,000 fair coin flips each one of them will get more guesses right than any of the others, one of them will get fewer guesses right than any of the others, and the gulf between the two is likely to be over 4 standard deviations wide. I'm certain that I, being an untutored schmuck from Pittsburgh and having thought of this almost immediately after reading about this contest, cannot be the first person to realize this is a potential problem for a forecasting contest. But I can't find anything they've done to mitigate that problem. Can anyone clue me in?

   1. cman1444 · 1789665693 · focus · HN ↗

      I don't understand your analogy. Are you just suggesting that luck plays too large a role in this contest? Clearly there is some "skill" or ability factor because AI's have been scoring higher and higher each year. Also, they make reference to superforecaster humans, who are presumably consistently better at forecasting than their peers.

      1. ratelimitsteve · 1789762493 · focus · HN ↗

         i'm not looking at the guesses, i'm looking at the distribution of guesser success rates. assuming random distribution, counterintuitively enough, you would expect some guessers to appear much better or much worse than others. i know i'm not the first person to think of this, so i'm asking what's been done to mitigate it because i can't find anything. if you expect x% of guesses to be within two std devs of the mean that means you can expect 100-x% to be outside that, even without any guessers actually being better at guessing than any of the others.

         1. chumzygood · 1789870740 · focus · HN ↗

            [dead]

Unofficial Hacker News client; not affiliated with Y Combinator.