I tested Jev with a fair die 400 times without telling it the die result. The true probability of face 1 is 1/6, but Jev always chose face 1 and the probability it returned was about 83%. I also tested with a fair coin 200 times and got 0.92 probability.
I did several tests and I think Jev is good at problems with a correct answer but weak at problems about actual probabilities whose answers can't be known at all.
Write-up: "Jev Does Not Play Dice"
<a href="https://kantahayashiai.github.io/posts/jev-does-not-play-dice/" rel="nofollow">https://kantahayashiai.github.io/posts/jev-does-not-play-dic...
kantahayashi · · focus · HN ↗
I did several tests and I think Jev is good at problems with a correct answer but weak at problems about actual probabilities whose answers can't be known at all.
Write-up: "Jev Does Not Play Dice" <a href="https://kantahayashiai.github.io/posts/jev-does-not-play-dice/" rel="nofollow">https://kantahayashiai.github.io/posts/jev-does-not-play-dic...
lesam · · focus · HN ↗
kantahayashi · · focus · HN ↗
From TypeSafe docs:
"Higher probability should correspond to a greater chance that the answer is correct."
"Outcomes assigned a probability of 0.2 should occur about 20% of the time."
<a href="https://docs.typesafe.ai/introduction/machine-learning-primer" rel="nofollow">https://docs.typesafe.ai/introduction/machine-learning-prime...