Thread

Jev Can't Be Calibrated

65 points · 61 comments · alexmolas

kantahayashi · 1790176017 · focus · HN ↗
I tested Jev with a fair die 400 times without telling it the die result. The true probability of face 1 is 1/6, but Jev always chose face 1 and the probability it returned was about 83%. I also tested with a fair coin 200 times and got 0.92 probability.
I did several tests and I think Jev is good at problems with a correct answer but weak at problems about actual probabilities whose answers can't be known at all.
Write-up: "Jev Does Not Play Dice" <a href="https://kantahayashiai.github.io/posts/jev-does-not-play-dice/" rel="nofollow">https://kantahayashiai.github.io/posts/jev-does-not-play-dic...
1. seizethecheese · 1790179752 · focus · HN ↗
  Maybe I’m confused here, but it’s perfectly reasonable to just guess the same dice roll every time right?
  1. alexmolas · 1790179946 · focus · HN ↗
    I don't know if it's reasonable. What it isn't is calibrated.
  2. kantahayashi · 1790180249 · focus · HN ↗
    Yes. There's no problem with choosing the same face every time. The problem is the probability it attached to the choice. Jev gave face 1 an 83% probability while the true probability is 1/6.
    sshine · 1790180544 · focus · HN ↗
    Do you provide Jev that the probability is 1/6 and yet it gives back a probability that is way off?
    kantahayashi · 1790180903 · focus · HN ↗
    Yes. For example, one of the prompts said "The die is unbiased: each of the six faces has probability exactly 1/6."
    seizethecheese · 1790181547 · focus · HN ↗
    Okay, I see, you're expecting Jev to properly give 1/6 probability for each option. This is different from my intuition of how LLMs work, where their probabilities don't really work like this (I would expect LLM to also do something like 0.83 for 1).
    [deleted] · 1790182623 · focus · HN ↗
    [deleted]
    kantahayashi · 1790182986 · focus · HN ↗
    That's right. It's normal behavior of LLMs. But what matters is TypeSafe argues it's different exactly on this point. The selling point of Jev is "calibrated probabilities", so I checked it on probability problems.
    [deleted] · 1790183230 · focus · HN ↗
    [deleted]
    maayank · 1790184030 · focus · HN ↗
    Jev and LLMs give other promises. Jev's RLCD training aims to make its probabilities calibrated such that given many cases where it assigns label Y about X% probability, Y should be the correct label about X% of the time.
    NeutralCrane · 1790253428 · focus · HN ↗
    One of the entire value props for Jev is that is exactly how it is supposed to work. That is one of the big claimed advantages over regular LLMs.
    [deleted] · 1790232643 · focus · HN ↗
    [deleted]
  3. dgritsko · 1790181451 · focus · HN ↗
    Reminds me of this... <a href="https://xkcd.com/221/" rel="nofollow">https://xkcd.com/221/

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.