> we ask them to stop testing advanced mathematical problems on proprietary models.
Maybe I'm alone on this, but for some reason these sorts of requests strike me as akin to gatekeeping how someone should breathe air. It's math... the numbers and symbols are just out there in the platonic realm available for anyone to do as they like with them. It's patently absurd to request other people to stop.
Ensuring credit where credit is due? That's fine. If your model incorporates the efforts of many others, then it's reasonable to request acknowledgement of everyone who contributed (even indirectly). But that's not what the request states — presumably their ask subsumes any advanced ML model, including those that weren't trained on a giant corpus of text.
The problem with proprietary models is that you don't get to poke inside and see what it is doing and how it arrives at its answer, which is precisely what mathematicians do. Mathematical understanding derives less from any particular result than the insights and methods that pave the road to results. Whenever a theorem is proved, researchers seek to unpack the proof and get inside the author's mind to learn their ways of thinking.
LLM generated results might benefit mathematical understanding if people can inspect their intermediate reasoning traces to discover erroneous human biases or patterns that they might have previously overlooked. Otherwise, the results might as well be produced by oracles.
> The problem with proprietary models is that you don't get to poke inside and see what it is doing and how it arrives at its answer, which is precisely what mathematicians do
Uh, a hypothetical fully open source model would have the exact same problem, because LLMs rely on emergent phenomena and no one understands why they work.
throwaway713 · · focus · HN ↗
Maybe I'm alone on this, but for some reason these sorts of requests strike me as akin to gatekeeping how someone should breathe air. It's math... the numbers and symbols are just out there in the platonic realm available for anyone to do as they like with them. It's patently absurd to request other people to stop.
Ensuring credit where credit is due? That's fine. If your model incorporates the efforts of many others, then it's reasonable to request acknowledgement of everyone who contributed (even indirectly). But that's not what the request states — presumably their ask subsumes any advanced ML model, including those that weren't trained on a giant corpus of text.
curt15 · · focus · HN ↗
LLM generated results might benefit mathematical understanding if people can inspect their intermediate reasoning traces to discover erroneous human biases or patterns that they might have previously overlooked. Otherwise, the results might as well be produced by oracles.
semiquaver · · focus · HN ↗
Uh, a hypothetical fully open source model would have the exact same problem, because LLMs rely on emergent phenomena and no one understands why they work.