<a href="https://frontierharness.org/" rel="nofollow">https://frontierharness.org/ allegedly this is on the Pareto frontier, though I don't know how good of a benchmark this really is. it seems to focus on one-shot type tasks, whereas the real utility of one harness over another seems to make itself known in long running tasks.
also this is a very limited static snapshot with one model as backend, I wish there were more consistently refreshed and diversified harness benchmarks.
i've seen "Pareto frontier" literally 500 thousand times in the last two weeks or so, and maybe once or twice before that. Can someone please explain what happened recently
So for me, literally I was trying to solve a problem with an AI agent and it threw the term at me. It is a real and useful concept, but my hypothesis is that a certain AI release started using and a bunch of people said "Oh this makes me sound smart/trendy" and started using it every chance they got.
We like to think we aren't token predictors, but the amount of sheer regurgitation I see from us humans makes me wonder (also "commoditize your complement", and so many others)
scuppernong · · focus · HN ↗
also this is a very limited static snapshot with one model as backend, I wish there were more consistently refreshed and diversified harness benchmarks.
cdelsolar · · focus · HN ↗
zug_zug · · focus · HN ↗
We like to think we aren't token predictors, but the amount of sheer regurgitation I see from us humans makes me wonder (also "commoditize your complement", and so many others)
JKCalhoun · · focus · HN ↗