I disagree. I haven't finished the article yet, but it smacks of arguing for mechanism over result.
If by some mechanism other than what a gatekeeper would call 'reasoning', a machine produces outputs that approach indistinguishable from 'well reasoned', the argument that it didn't get there by reasoning is, well, not useful at the very least.
Your last sentence is identical to your first sentence in your other comment. How can we be sure that you are reasoning and not just a stochastic parrot?
This comparison between fly and intelligence is worthy of the most vulgar bar talk
> it can't have superhuman reasoning
no they don't, we still die of cancer, there's no global deployed autonomous driving and food production driven by super intelligent ais and I'm not walking on mars thanks to gravitational elevators
Comparisons to how humans reason beg the question: is the way humans do it the only way?
Lots if these arguments are similar to birds saying "Jets don't flap their wings so they aren't even flying."
The arguments about reasoning are even shakier because they usually rely on totally unproven assertions about human reasoning. At least we know birds flap their wings.
Copying biological systems isn’t always the best way to build machines. The article compares AlphaGo’s policy network and value network to system 1 and system 2 thinking in humans.
You are almost there in terms of getting their point, so I will explain.
Birds existed since forever ago in nature, and they fly by flapping their wings. Then planes got invented, and they fly using a very different mechanism (that doesn't involve flapping wings).
The point made by the grandparent comment: saying "LLMs don't actually reason, because the underlying mechanism they use is different from how humans reason" feels about the same as "planes don't actually fly, because the underlying mechanism they use is different from how birds fly".
And a magician making a coin "disappear" doesn't mean that magic is real. I see no way we can call it thinking without a goal (other than computing the next token).
This is the problem I have. People are built to be fooled by this stuff - to see something that's not there. Pareidolia but with language. It's such a compulsion that the people who build LLM's see that the same non thing there too.
When I stumble around trying to explain my thinking though I invariably get hit with the response, Sure, but if it's functionally identical to a coin being magically pulled out your ear, what's the difference if it is or if it isn't? The only response I have is that when the coin doesn't appear you're going to be putting yourself way further behind the starting line then if you thought from the get go that there's no such thing as magic.
> This is the problem I have. People are built to be fooled by this stuff - to see something that's not there. Pareidolia but with language.
I really like this metaphor, it hits home that just because you sense something is human-like doesn't make it so. We're wired to respond to those sorts of things.
> When I stumble around trying to explain my thinking
I had similar difficulties expressing my thoughts in an organized way. The video below from Richard Sutton (the father of Reinforcement Learning [which LLMs use extensively in training]), helped bring order and firm up some of my thoughts and theories.
>I invariably get hit with the response, Sure, but if it's functionally identical to a coin being magically pulled out your ear, what's the difference if it is or if it isn't?
Not to spoil the video, but it isn't at all functionally equivalent. LLMs are an imperfect and somewhat randomized simulation of what the LLM thinks an average person might say in response to a question. Which is another way of saying, most people will on average get worse answers than if they worked on a problem themselves (but they will get that worse answer from the LLM comparatively quickly). Practical people respond to this positioning fairly well, and for others it still lands a bit since no one wants to think they're below average. That said, I'm still working on my positioning a bit as well!
I believe there's a confusion in this thread between what an LLM _does_ and what an LLM _is_. What is does is output a next token. What it is, is a universal function approximator ([1], i.e. a neural net).
With back probation in the neural net, there could be a full state machine being approximated inside the weight. And a state machine is the exact step-wise reasoning the author claims it needs.
coreyh14444 · · focus · HN ↗
hollowturtle · · focus · HN ↗
Retr0id · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
hollowturtle · · focus · HN ↗
archontes · · focus · HN ↗
If by some mechanism other than what a gatekeeper would call 'reasoning', a machine produces outputs that approach indistinguishable from 'well reasoned', the argument that it didn't get there by reasoning is, well, not useful at the very least.
joquarky · · focus · HN ↗
rtrgrd · · focus · HN ↗
In context: just because our LLMs don't have an explicit 'system 2' component doesnt mean it can't have superhuman reasoning
hollowturtle · · focus · HN ↗
> it can't have superhuman reasoning
no they don't, we still die of cancer, there's no global deployed autonomous driving and food production driven by super intelligent ais and I'm not walking on mars thanks to gravitational elevators
warkdarrior · · focus · HN ↗
Love to see the forever moving goalposts. One might say they're autonomously moving..
hollowturtle · · focus · HN ↗
Do we have finally reliable autonomous driving? And available outside the bay area or whatever small place compared to the rest of the world?
strbean · · focus · HN ↗
Lots if these arguments are similar to birds saying "Jets don't flap their wings so they aren't even flying."
The arguments about reasoning are even shakier because they usually rely on totally unproven assertions about human reasoning. At least we know birds flap their wings.
sh-run · · focus · HN ↗
filoleg · · focus · HN ↗
Birds existed since forever ago in nature, and they fly by flapping their wings. Then planes got invented, and they fly using a very different mechanism (that doesn't involve flapping wings).
The point made by the grandparent comment: saying "LLMs don't actually reason, because the underlying mechanism they use is different from how humans reason" feels about the same as "planes don't actually fly, because the underlying mechanism they use is different from how birds fly".
2snakes · · focus · HN ↗
infamia · · focus · HN ↗
hackeraccount · · focus · HN ↗
When I stumble around trying to explain my thinking though I invariably get hit with the response, Sure, but if it's functionally identical to a coin being magically pulled out your ear, what's the difference if it is or if it isn't? The only response I have is that when the coin doesn't appear you're going to be putting yourself way further behind the starting line then if you thought from the get go that there's no such thing as magic.
infamia · · focus · HN ↗
I really like this metaphor, it hits home that just because you sense something is human-like doesn't make it so. We're wired to respond to those sorts of things.
> When I stumble around trying to explain my thinking
I had similar difficulties expressing my thoughts in an organized way. The video below from Richard Sutton (the father of Reinforcement Learning [which LLMs use extensively in training]), helped bring order and firm up some of my thoughts and theories.
<a href="https://www.youtube.com/watch?v=21EYKqUsPfg" rel="nofollow">https://www.youtube.com/watch?v=21EYKqUsPfg
>I invariably get hit with the response, Sure, but if it's functionally identical to a coin being magically pulled out your ear, what's the difference if it is or if it isn't?
Not to spoil the video, but it isn't at all functionally equivalent. LLMs are an imperfect and somewhat randomized simulation of what the LLM thinks an average person might say in response to a question. Which is another way of saying, most people will on average get worse answers than if they worked on a problem themselves (but they will get that worse answer from the LLM comparatively quickly). Practical people respond to this positioning fairly well, and for others it still lands a bit since no one wants to think they're below average. That said, I'm still working on my positioning a bit as well!
conscion · · focus · HN ↗
I believe there's a confusion in this thread between what an LLM _does_ and what an LLM _is_. What is does is output a next token. What it is, is a universal function approximator ([1], i.e. a neural net).
With back probation in the neural net, there could be a full state machine being approximated inside the weight. And a state machine is the exact step-wise reasoning the author claims it needs.
[1]: <a href="https://en.wikipedia.org/wiki/Universal_approximation_theorem" rel="nofollow">https://en.wikipedia.org/wiki/Universal_approximation_theore...