Thread

1. ForgotMyUUID · 1789802300 · focus · HN ↗

   I’m reminded of that famous debate between Poincaré and Hilbert at the International Congress of Mathematicians in Paris in 1900. It was then that everyone decided to follow Hilbert’s path, and proof came to be valued more than intuition. I think modern math at school and at applied university kind of lost this intuitive part.

   I try to teach my students that mathematics is, first and foremost, a very precise language of communication. It’s sometimes amusing to ask those who don’t like math to do without it entirely, just to see how much harder it becomes to describe the things around them.

   Second thing I tell them, formulas are the essence of mechanisms in their purest form. And in this form, they’re much easier to grasp and mentally manipulate. It always amused me, after taking a mechanics course, to imagine that for any formula, you could visualize a mechanism or process that implements it.

   And third thing, I suppose, the ability to verify one’s own statements as proof. Although, of course, mathematicians would probably tear me apart here for my heresy:sorry, I’m not a mathematician, but an engineer. You can make mistakes by using incorrect assumptions, but at some point, analysis itself will show you that you were mistaken. There’s a wonderful book, How to Prove It by Daniel Velleman, which provides an introduction to proof for the uninitiated like me. I really enjoyed it.

   1. bananaflag · 1789810853 · focus · HN ↗

      As a math prof, I care(d) much more about proof than intuition, not because proof is more important, but exactly because intuition is (I'm a bit Chesterton-ish here haha). You cannot do proof without intuition hence, if you emphasize proof, intuition will take care of itself. Whereas if you emphasize intuition, students won't have any idea of what a rigorous proof should be.

      1. fidotron · 1789821275 · focus · HN ↗

         Surely this implies these LLM generated proofs require the LLMs to have mathematical intuition . . . and honestly I don't think many people believe that, and rightly so, certainly not in the way Poincaré was on about.

         Maybe it's been done, but I'd like to see an LLM recreate Euclid from questioning without having seen it during training.

         1. keeda · 1789825269 · focus · HN ↗

            My hunch (or intuition, hah!) is that intuition is an instinctive mental shortcut required to navigate large problem spaces that can’t entirely fit into our heads.

            Maybe LLMs do not need intuition because they can scale their “cognitive capacity” with hardware and brute force their way through these problem spaces.

            1. fidotron · 1789831253 · focus · HN ↗

               > Maybe LLMs do not need intuition because they can scale their “cognitive capacity” with hardware and brute force their way through these problem spaces.

               My view is that is certainly true of smaller LLMs but becomes less true as they scale up.

               To quote the parent bananaflag in a sub-comment:

               > I believe the LLM weights have some internal representation of math in the same way brains do that allow them to produce proofs

               I think as the sort of spare space adjacent to pure language processing in LLMs grows the probability of the sort of reasoning bananaflag is getting at (or spatial reasoning, or anything else) emerging in that space grows enormously.

               One of the questions for AI development over the coming months or years is going to be if deliberately cultivating the architecture of those sub models for specific reasoning types beats any emergent reasoning mechanisms or not.

               1. keeda · 1789838546 · focus · HN ↗

                  Hmm we may be talking of two different interpretations of intuition here. I agree that LLM weights contain representations of abstract concepts, as a lot of prior research has shown. This surely includes Mathematical concepts.

                  But to me that is analogous to what human brains do, and a bit different from intuition. I think of intuition as “heuristics”, typically developed through experience, that may link seemingly unrelated concepts via vague, hard-to-define associations, but which let us make mental leaps (or shortcuts) while reasoning. (Maybe analogous to System 1 / 2 thinking.)

                  On the other hand, LLMs can do both: build “intuition” from patterns in data AND brute force a huge amount of potentially unrelated concepts. This gets fuzzier when we realize that even these “concepts” themselves are gleaned from patterns in data! But my point is we necessarily have to take shortcuts to scale, whereas machines can scale with hardware.

                  This is of course a layman theory! But it could explain why these models are progressing so fast.

                  1. fidotron · 1789841675 · focus · HN ↗

                     Yes, in my case "intuition" comes a lot from visualizing things spatially, manipulating them, and being able to capture their properties in equations/proofs, and it's that which is (currently) conspicuously missing when dealing with LLMs. (And may yet appear with world models).

                     With the alternate view of intuition that many of you are describing it is clear LLMs are somewhat either there or heading there now.

                     1. keeda · 1789844582 · focus · HN ↗

                        This is an intriguing observation! LLMs were famously bad at spatial reasoning, until Astra which apparently has a huge improvement. I wonder if that has any bearing on the recent jump in Mathematical performance?

                        One thing that struck me from Dario's last podcast with Dwarkesh was that he said training LLMs on a diverse set of tasks does not make them better just at those tasks, but they get better at unrelated and other tasks overall. What you described could be a concrete example of how that dynamic works!

Unofficial Hacker News client; not affiliated with Y Combinator.