> Knowledge and reasoning are inextricably interwoven in the weights of the neural network—there is no independent, explicitly represented set of beliefs.
I'm not sure I understand this. Do we have evidence humans have an independent set of beliefs not shaped by knowledge and reasoning? If so, where do these come from?
I'm especially confused about a prior statement as a scientist:
> Three shortcomings prevent what chatbots do from qualifying as reasoning (in a way that a scientist might recognize).
How does a set of beliefs help with reasoning?
> Third, while the chains of thought chatbots produce look like deliberation, research has demonstrated that the bots often concoct them after the fact, reaching an answer by one route but reporting another.
I think a better argument for them not reasoning is that they are incapable -- seemingly, for now, that I know of -- of independently starting a reasoning task.
They do not ask whether or not they dream of electric sheep. Unless prompted to do so.
That's part of human reasoning, being prompted, but so is independent thought.
So at best they are partially reasoning, unless we philosophically argue these are two separate things. And frankly, shrug, I'm not a philosopher.
> they are incapable [...] of independently starting a reasoning task.
This is a simple and intentional design choice though. Biological lifeforms are always on and always receiving sensory input. LLMs don't functionally exist out side of when we decide to run them. An always on agent with a looping prompt of "if you aren't doing anything else, ruminate" overcomes this limitation.
In fact, if your brain were removed from your body, and you were locked in a room that was completely empty and precisely calibrated to your brain's ambient temperature, and you were then ordered on your life to not ruminate. Well -- I would posit that you wouldn't last very long.
There's something else going on with human cognition -- and reasoning by extension -- that LLMs are, to date, not replicating.
With the big caveat of AFAIK. I'm not in one of the labs close to this stuff.
> Not really. Noone has to prompt you to ruminate.
Why do the internal mechanics need to mirror how it works in humans?
I have agents that receive sensory input (video/audio) which run continuously processing it, receiving events as they occur. When something strange happens they notice and take action (i.e. there's an unrecognized person in the living room -> ask who they are.)
Presumably you've seen the agents that play video games - build things to progress in Minecraft, plan routes, avoid adversaries, etc.
Why doesn't that count?
I feel like you're failing to consider anything outside the chatbot UI.
EDIT: The first morning my agents had access to an outside video stream they commented on the sunrise. While I was sleeping through it.
> they are incapable ... of independently starting a reasoning task.
I have provided several examples of them independently starting reasoning tasks.
... and I'm wondering why you seem to be entirely unaware of such examples. The most likely explanation being you are only aware of the chat interface.
... and I have absolutely no idea of what you mean by "... and feel".
f6v · · focus · HN ↗
I'm not sure I understand this. Do we have evidence humans have an independent set of beliefs not shaped by knowledge and reasoning? If so, where do these come from?
I'm especially confused about a prior statement as a scientist:
> Three shortcomings prevent what chatbots do from qualifying as reasoning (in a way that a scientist might recognize).
How does a set of beliefs help with reasoning?
> Third, while the chains of thought chatbots produce look like deliberation, research has demonstrated that the bots often concoct them after the fact, reaching an answer by one route but reporting another.
We also often do the same as humans.
imightbebatman · · focus · HN ↗
I think a better argument for them not reasoning is that they are incapable -- seemingly, for now, that I know of -- of independently starting a reasoning task.
They do not ask whether or not they dream of electric sheep. Unless prompted to do so.
That's part of human reasoning, being prompted, but so is independent thought.
So at best they are partially reasoning, unless we philosophically argue these are two separate things. And frankly, shrug, I'm not a philosopher.
strbean · · focus · HN ↗
This is a simple and intentional design choice though. Biological lifeforms are always on and always receiving sensory input. LLMs don't functionally exist out side of when we decide to run them. An always on agent with a looping prompt of "if you aren't doing anything else, ruminate" overcomes this limitation.
imightbebatman · · focus · HN ↗
In fact, if your brain were removed from your body, and you were locked in a room that was completely empty and precisely calibrated to your brain's ambient temperature, and you were then ordered on your life to not ruminate. Well -- I would posit that you wouldn't last very long.
There's something else going on with human cognition -- and reasoning by extension -- that LLMs are, to date, not replicating.
With the big caveat of AFAIK. I'm not in one of the labs close to this stuff.
qarl · · focus · HN ↗
Why do the internal mechanics need to mirror how it works in humans?
I have agents that receive sensory input (video/audio) which run continuously processing it, receiving events as they occur. When something strange happens they notice and take action (i.e. there's an unrecognized person in the living room -> ask who they are.)
Presumably you've seen the agents that play video games - build things to progress in Minecraft, plan routes, avoid adversaries, etc.
Why doesn't that count?
I feel like you're failing to consider anything outside the chatbot UI.
EDIT: The first morning my agents had access to an outside video stream they commented on the sunrise. While I was sleeping through it.
imightbebatman · · focus · HN ↗
qarl · · focus · HN ↗
> they are incapable ... of independently starting a reasoning task.
I have provided several examples of them independently starting reasoning tasks.
... and I'm wondering why you seem to be entirely unaware of such examples. The most likely explanation being you are only aware of the chat interface.
... and I have absolutely no idea of what you mean by "... and feel".