<a href="https://frontierharness.org/" rel="nofollow">https://frontierharness.org/ allegedly this is on the Pareto frontier, though I don't know how good of a benchmark this really is. it seems to focus on one-shot type tasks, whereas the real utility of one harness over another seems to make itself known in long running tasks.
also this is a very limited static snapshot with one model as backend, I wish there were more consistently refreshed and diversified harness benchmarks.
i've seen "Pareto frontier" literally 500 thousand times in the last two weeks or so, and maybe once or twice before that. Can someone please explain what happened recently
In case people need a refresher on what the Pareto frontier is, this article does a great job explaining it (with Mario Kart!): <a href="https://www.mayerowitz.io/blog/mario-meets-pareto" rel="nofollow">https://www.mayerowitz.io/blog/mario-meets-pareto
I would guess that the article came out because the term was getting popular in AI/LLM circles for measuring LLM cost-for-performance metrics. It was probably riding the meme up the wave, rather than fundamentally triggering it.
scuppernong · · focus · HN ↗
also this is a very limited static snapshot with one model as backend, I wish there were more consistently refreshed and diversified harness benchmarks.
cdelsolar · · focus · HN ↗
zinoc · · focus · HN ↗
pu_pe · · focus · HN ↗
thethirdone · · focus · HN ↗
[0]: <a href="https://trends.google.com/trends/explore?date=all&q=pareto%20frontier" rel="nofollow">https://trends.google.com/trends/explore?date=all&q=pareto%2...
[1]: <a href="https://books.google.com/ngrams/graph?content=pareto+frontier" rel="nofollow">https://books.google.com/ngrams/graph?content=pareto+frontie...
dualvariable · · focus · HN ↗