‹ BackHN Continuity

Thread

The Two MMLU Scores: What a Benchmark Name Does Not Fix

8 points · 3 comments · gmays

  1. bbstats · · focus · HN ↗
    "What a ____ does not ____" is painfully Claude-Coded (no pun intended)
    1. entrope · · focus · HN ↗
      So is "does [not] fix [attribute]". The opening dot list has several others, such as "X, Y, Z stay open", "the claim carries its ___", "___ is a statement about ___ and says nothing about ___", and repeating a pair of numbers followed quickly by the difference between them. (It's +0.009!)

      And I have to think hard to guess what it probably meant by "Comparability is a property of the reference the results are traceable to, not of the number". I don't think "[X and Y] are different scopes by canonical bytes" even makes sense.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.