Yesterday I tried to google "can the Halifax Wanderers still make the CPL playoffs?"
So obviously what appears right at the top is the AI summary, which told me "they've already secured their #4 position and made the playoffs". I knew this wasn't true, and I guess I could have just scrolled down a bit further and found my answer but now I was curious.
So I said "that's not true, they're still #5, what I want to know is _could they still make the playoffs_"
It says they've got an upcoming game against Ottawa, and if they win their chances are good. That game has already taken place, so I correct it again and finally I get a reasonable answer.
My question is: what's the point of the AI in the search engine if it itself isn't going to use the search engine first before answering? Like, I can't wrap my head around that. The answer is on the same page as its hallucination. It could have done a cursory look around before first hallucinating something completely false, and when corrected the first time giving me outdated information. It's meant to be A SEARCH ENGINE!
This is similar to how, not too long ago, LLM's had extreme difficulty counting the number of letters in some words. LLM's don't "think" or "reason" in the normal definition of those terms. They can do some pretty amazing things, but still screw up basic things like telling you something that is obviously wrong and contradicts the top search results.
LLM's, in their present stage of development, are sort of like a crack-addled idiot savant. Sometimes they are obviously insane, and sometimes they seem quite cogent, but you must never trust them implicitly. This may be why they are so difficult to constrain. You could give them something equivalent to the laws of robotics, but following laws requires thought processes they simply don't have.
I'm actually sort of amazed Google doesn't make people accept some kind of butt-covering EULA and post disclaimers about the inaccuracy of results before even showing you their AI's output. Are they not being sued over this kind of thing?
They literally cannot. They can detect the user’s intent to do math, and then use a different tool to do math, hopefully with the correct inputs. The LLM is not suited to giving deterministic answers to math problems.
It would be more accurate to say they can do math instantaneously without even thinking, at a level far beyond what humans can do. (I assume you're talking about doing arithmetic.)
TLDR: Astra has 8.6x better odds of doing a reasoning task without CoT than the
next best model (Fable 5.1), and can do 7.2 serial arithmetic steps in a forward
pass vs 4.1 for the next best model (Gemini 3.8 Flash/Fable 5.1)
Yeah, but as you might expect they internally represent numbers probabilistically, so there’s always a nonzero possibility of confusing the inputs or outputs of any operation. Kind of like misremembering your multiplication tables.
Heck I kinda wonder if LLMs can do math as well as they can reason, “think”, etc. ie its all just probabilistic lunacy that somehow works great, so why are we so concerned about math being wrong? It could be wrong about the color of the sky, the size of a basket ball, how much oranges weigh, etc etc.
The nice thing about math is it can easily plug into a tool, making it even less of a concern.
reasoning models can trivially do math (open up astra and ask it some undergraduate problems), but eventually break down (similar to how humans start to lose track if asked to do math without any assistance)
There needs to be a Godwin's Law for discussions about LLMs: where any criticism of LLMs exists online the likelihood of equating LLM behavior to human behavior approaches 1.
Functionally, a Markov chain shares many parallels with human cognition for the same reasons.
People don't understand what correlation is and they assume the mapping between a human brain and an LLM is 1:1 in every case where it matters, which is a religious and not scientifically based belief.
Maybe it’s just a different and in some ways better way of doing mathematics? Maybe how we think and process mathematics of physics is just but one way to do it? I’m not suggesting an LLM will prove 2+2=6 because of course that’s nonsense but maybe it can invent a new calculus?
> The LLM is not suited to giving deterministic answers to math problems.
Less so with formal mathematics proofs maybe but I think in general humans don’t provide deterministic answers to math problems or questions either. Humans get it wrong all the time and when you ask a human to solve a problem they may solve it in a different way than before.
Are you sure it honoured your stipulation of "without external help"? For all we know, it hacked its way into Wolfram Alpha and got the result from there.
Arithmetic is well within the capabilities of even small local models: <a href="https://i.imgur.com/21tzGlN.png" rel="nofollow">https://i.imgur.com/21tzGlN.png
Not really, since it wasn't a fresh context with a fresh question. I just told it it's wrong in the same chat session and it corrected it there.
LLMs can in fact do arithmetic, just not reliably owing to how numbers are represented probabilistically: <a href="https://arxiv.org/abs/2410.21272" rel="nofollow">https://arxiv.org/abs/2410.21272
That is a conflation of LLMs (which have clear limitations) and complex harnesses of which an LLM is one component.
I think it is clear that future AI may incorporate an LLM as a component but the current concept of LLMs are a transitional form that will give way to more capable composite models.
No it isn't. Even without any harness at all, modern LLMs are better at maths than the majority of undergraduate students in mathematics. Seriously, we need to face facts, not just comforting ourselves with what they were like a year ago.
It doesn't input and output letters. It's like if it communicated with a version of sign language that's closer to written English. It's using all the same words but it has an arbitrary signal or two for each one.
It can't spell for garbage because the tokenizer hides the real spellings from the LLM.
Even if this was a math question that he asked, this is live audio input/output from a multimodal model. It is designed for rapid responses. This is not even remotely related to what we are talking about.
LLMs cannot do math. They can generate tool calls as text that allow them to drive programs and proof agents. Compare and contrast this against human brains who can do math in the same context without needing external tools. We don't need to bring a calculator to count the letters in a sentence. It is a different neural machinery.
LLMs are bizarrely good at non-tool-assisted math these days. They can multiply multiple digit numbers without reasoning! I can’t do that. I’d love to understand better how the LLMs do this.
Hugsbox · · focus · HN ↗
So obviously what appears right at the top is the AI summary, which told me "they've already secured their #4 position and made the playoffs". I knew this wasn't true, and I guess I could have just scrolled down a bit further and found my answer but now I was curious.
So I said "that's not true, they're still #5, what I want to know is _could they still make the playoffs_"
It says they've got an upcoming game against Ottawa, and if they win their chances are good. That game has already taken place, so I correct it again and finally I get a reasonable answer.
My question is: what's the point of the AI in the search engine if it itself isn't going to use the search engine first before answering? Like, I can't wrap my head around that. The answer is on the same page as its hallucination. It could have done a cursory look around before first hallucinating something completely false, and when corrected the first time giving me outdated information. It's meant to be A SEARCH ENGINE!
beloch · · focus · HN ↗
LLM's, in their present stage of development, are sort of like a crack-addled idiot savant. Sometimes they are obviously insane, and sometimes they seem quite cogent, but you must never trust them implicitly. This may be why they are so difficult to constrain. You could give them something equivalent to the laws of robotics, but following laws requires thought processes they simply don't have.
I'm actually sort of amazed Google doesn't make people accept some kind of butt-covering EULA and post disclaimers about the inaccuracy of results before even showing you their AI's output. Are they not being sued over this kind of thing?
VCFundedGenYer · · focus · HN ↗
fasterik · · focus · HN ↗
mbgerring · · focus · HN ↗
versteegen · · focus · HN ↗
GaggiX · · focus · HN ↗
dcrazy · · focus · HN ↗
einichi · · focus · HN ↗
unshavedyak · · focus · HN ↗
The nice thing about math is it can easily plug into a tool, making it even less of a concern.
hodgehog11 · · focus · HN ↗
demibabs · · focus · HN ↗
5.6 on Instant mode can knock out 3 digit multiplication just fine.
vanuatu · · focus · HN ↗
krapp · · focus · HN ↗
Forgeties79 · · focus · HN ↗
vanuatu · · focus · HN ↗
krapp · · focus · HN ↗
People don't understand what correlation is and they assume the mapping between a human brain and an LLM is 1:1 in every case where it matters, which is a religious and not scientifically based belief.
vanuatu · · focus · HN ↗
ericmay · · focus · HN ↗
> The LLM is not suited to giving deterministic answers to math problems.
Less so with formal mathematics proofs maybe but I think in general humans don’t provide deterministic answers to math problems or questions either. Humans get it wrong all the time and when you ask a human to solve a problem they may solve it in a different way than before.
queenkjuul · · focus · HN ↗
<a href="https://x.com/maksym_andr/status/2100364212207837560" rel="nofollow">https://x.com/maksym_andr/status/2100364212207837560
Dylan16807 · · focus · HN ↗
ericmay · · focus · HN ↗
But even if you have 5x6 memorized it's not deterministic that you answer 30. It's just highly probable.
tjwebbnorfolk · · focus · HN ↗
fasterik · · focus · HN ↗
<a href="https://chatgpt.com/share/6ab9a0da-fdd0-83e8-a62d-f0cdeb54db48" rel="nofollow">https://chatgpt.com/share/6ab9a0da-fdd0-83e8-a62d-f0cdeb54db...
I'm sure it still makes mistakes, but saying it can't do arithmetic is just false.
tremon · · focus · HN ↗
jmillikin · · focus · HN ↗
guelo · · focus · HN ↗
[dead]
Xirdus · · focus · HN ↗
dcrazy · · focus · HN ↗
Xirdus · · focus · HN ↗
Xirdus · · focus · HN ↗
amluto · · focus · HN ↗
dcrazy · · focus · HN ↗
Isamu · · focus · HN ↗
I think it is clear that future AI may incorporate an LLM as a component but the current concept of LLMs are a transitional form that will give way to more capable composite models.
hodgehog11 · · focus · HN ↗
dwaite · · focus · HN ↗
spartanatreyu · · focus · HN ↗
<a href="https://www.youtube.com/watch?v=iTyLHDRhwJg" rel="nofollow">https://www.youtube.com/watch?v=iTyLHDRhwJg
Dylan16807 · · focus · HN ↗
spartanatreyu · · focus · HN ↗
Dylan16807 · · focus · HN ↗
And it can count fine. It doesn't know how to spell.
spartanatreyu · · focus · HN ↗
Counting most certainly is math.
Can you define or explain counting without also referencing/defining/explaining a mathematics concept?
> And it can count fine. It doesn't know how to spell.
It's the other way around, it could spell, it couldn't count the letters.
Dylan16807 · · focus · HN ↗
It can't spell for garbage because the tokenizer hides the real spellings from the LLM.
hodgehog11 · · focus · HN ↗
okanat · · focus · HN ↗
fenomas · · focus · HN ↗
amluto · · focus · HN ↗