> Knowledge and reasoning are inextricably interwoven in the weights of the neural network—there is no independent, explicitly represented set of beliefs.
I'm not sure I understand this. Do we have evidence humans have an independent set of beliefs not shaped by knowledge and reasoning? If so, where do these come from?
I'm especially confused about a prior statement as a scientist:
> Three shortcomings prevent what chatbots do from qualifying as reasoning (in a way that a scientist might recognize).
How does a set of beliefs help with reasoning?
> Third, while the chains of thought chatbots produce look like deliberation, research has demonstrated that the bots often concoct them after the fact, reaching an answer by one route but reporting another.
In a human brain it can reason regardless of what you personally know. In an LLM, the knowledge and the structure are the same thing. It doesn’t have separate knowledge parts and processing parts, it’s all one network. If you delete the part about cats, it can’t think at all anymore. Whereas a human could lose all their memories and still be capable of thinking.
How can you know that, when every human who reasons knows a lot? We spend years in a pre-rational state, and by the time we can reason reliably we have ingested a huge amount of evidence about the world. Memories are just a snippet of what you learn by observation; you don't have a memory of gravity but you do have an internalised model of gravity derived from what you learnt as a baby.
I don’t know why you guys keep rehashing what the author said. I don’t really have a reading comprehension problem. I have a problem with someone taking it a face value. See above for my other responses.
f6v · · focus · HN ↗
I'm not sure I understand this. Do we have evidence humans have an independent set of beliefs not shaped by knowledge and reasoning? If so, where do these come from?
I'm especially confused about a prior statement as a scientist:
> Three shortcomings prevent what chatbots do from qualifying as reasoning (in a way that a scientist might recognize).
How does a set of beliefs help with reasoning?
> Third, while the chains of thought chatbots produce look like deliberation, research has demonstrated that the bots often concoct them after the fact, reaching an answer by one route but reporting another.
We also often do the same as humans.
MisterMunchkin · · focus · HN ↗
brainwad · · focus · HN ↗
f6v · · focus · HN ↗