> we ask them to stop testing advanced mathematical problems on proprietary models.
Maybe I'm alone on this, but for some reason these sorts of requests strike me as akin to gatekeeping how someone should breathe air. It's math... the numbers and symbols are just out there in the platonic realm available for anyone to do as they like with them. It's patently absurd to request other people to stop.
Ensuring credit where credit is due? That's fine. If your model incorporates the efforts of many others, then it's reasonable to request acknowledgement of everyone who contributed (even indirectly). But that's not what the request states — presumably their ask subsumes any advanced ML model, including those that weren't trained on a giant corpus of text.
To quote Noam Brown, "Our main focus is shipping great models so everyone can use them to make discoveries of their own" [1]. Spending 15 million dollars of compute with an internal model to blitz and scoop a resolution of Navier-Stokes that someone is already on track to resolve (or, from my understanding of why OpenAI did this, had already resolved, as staff at OpenAI stated their attempt was prompted by rumors of a resolution by Anthropic) is in complete opposition to this stated claim. That act is what this portion of the recommendation is in response to: OpenAI perpetually holding out internal models and tooling, using them to solve important problems in math, and thus themselves holding a monopoly on certain aspects of mathematics. Why is it absurd to ask people who are in a position to do an obviously damaging act to not do so, especially when those same people were the ones who asked for your advice in the first place and claimed to not want to do said act previously?
Frontier labs aren't under any obligation to release every model they develop. Either way, I don't see much evidence that they're withholding them in perpetuity. Open models aren't far behind so there's no incentive for that.
Difficult math problems are useful benchmarks and milestones. It's well worth throwing money at solving a problem if it drives competition and improves models significantly. The benefits become available to everyone, including the thousands of professional mathematicians who can use them to become more productive.
> Open models aren't far behind so there's no incentive for that.
"But inference costs play a massive role here too, which I think is still the biggest bottleneck in AI development. Running high-impact models for math, medicine, or programming requires a ton of compute. Throw in the politicization of everything AI-related, and it feels like model privatization is just the tip of the iceberg."
throwaway713 · · focus · HN ↗
Maybe I'm alone on this, but for some reason these sorts of requests strike me as akin to gatekeeping how someone should breathe air. It's math... the numbers and symbols are just out there in the platonic realm available for anyone to do as they like with them. It's patently absurd to request other people to stop.
Ensuring credit where credit is due? That's fine. If your model incorporates the efforts of many others, then it's reasonable to request acknowledgement of everyone who contributed (even indirectly). But that's not what the request states — presumably their ask subsumes any advanced ML model, including those that weren't trained on a giant corpus of text.
isotypic · · focus · HN ↗
[1] <a href="https://x.com/polynoamial/status/2093451221273387477" rel="nofollow">https://x.com/polynoamial/status/2093451221273387477
fasterik · · focus · HN ↗
Difficult math problems are useful benchmarks and milestones. It's well worth throwing money at solving a problem if it drives competition and improves models significantly. The benefits become available to everyone, including the thousands of professional mathematicians who can use them to become more productive.
parisbs · · focus · HN ↗