‹ BackHN Continuity

Thread

The Two MMLU Scores: What a Benchmark Name Does Not Fix

6 points · 0 comments · gmays

Loading the complete thread in the background. This saved snapshot is available now. Refresh

  1. bbstats · · focus · HN ↗
    "What a ____ does not ____" is painfully Claude-Coded (no pun intended)
    1. entrope · · focus · HN ↗
      So is "does [not] fix [attribute]". The opening dot list has several others, such as "X, Y, Z stay open", "the claim carries its ___", "___ is a statement about ___ and says nothing about ___", and repeating a pair of numbers followed quickly by the difference between them. (It's +0.009!)

      And I have to think hard to guess what it probably meant by "Comparability is a property of the reference the results are traceable to, not of the number". I don't think "[X and Y] are different scopes by canonical bytes" even makes sense.

  2. mpalmer · · focus · HN ↗
    I genuinely have no idea what this content is supposed to convey. Even the TL;DR lead-in is inscrutable. If there is one nice thing to say about this, I honestly can't find it.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.