> Knowledge and reasoning are inextricably interwoven in the weights of the neural network—there is no independent, explicitly represented set of beliefs.
I'm not sure I understand this. Do we have evidence humans have an independent set of beliefs not shaped by knowledge and reasoning? If so, where do these come from?
I'm especially confused about a prior statement as a scientist:
> Three shortcomings prevent what chatbots do from qualifying as reasoning (in a way that a scientist might recognize).
How does a set of beliefs help with reasoning?
> Third, while the chains of thought chatbots produce look like deliberation, research has demonstrated that the bots often concoct them after the fact, reaching an answer by one route but reporting another.
Philosophers like Kant, Descartes and Locke wanted to draw a distinction between a priori and a posteriori beliefs (or knowledge). I believe AI will be very much bound by these discussions. Some knowledge can be had purely by reason. Some other knowledge will require observations.
Consider a calculator, a machine capable of valid transformations wrt the semantics of arithmetic.
Does a calculator reason its way to 2 + 2 = 4? No, it merely transforms.
An LLM is simply a much more complex version of this; except that its complexity hides a lot of what is otherwise reducible to a transformation of semantic, conceptual, and inferential relations. There are lots of NLP models prior to the introduction of modern "AI" that would make these representations more clear.
However I don't believe any of the aforementioned philosophers would ever consider reason itself as a transformation, as Kant especially spent a lot of time differentiating apprehension from transition.
I’m not so sure. While I completely agree on pure transformation, I continue to believe that the most concerning aspect of neural networks is that they really strengthen the argument that consciousness is the illusion.
The interplay between deductive frameworks and inductive frameworks is still always buoyed by the black swan problem… which is a the heart of Hume’s solipsism. Kant’s pure reason never considered instinctual logical gates that come prepackaged in the mind. That seems frighteningly close to a prediction algorithm, no matter how much we wrap it in a concept of understanding. In the end, we’re just building a model with a minimal error rate.
> I continue to believe that the most concerning aspect of neural networks is that they really strengthen the argument that consciousness is the illusion.
Based on what? Just because you can model the weather using math doesn't imply at all that the weather is the product of a mathematical model itself. I don't see how reasoning is different.
We deliberately construct LLMs so that they approximate certain relations typical of reasoning. Suppose they become extraordinarily successful at doing so. What exactly have we demonstrated? That these effects can be artificially reproduced through such a mechanism. But how would it follow that the thing being reproduced must itself be nothing more than that mechanism?
Based on the face that we have zero explanation for consciousness. Based on the face that biologically neural networks are exactly a kind of neural network.
My point isn’t that we’re able to explain consciousness as an illusion. It’s that we’re able to explain consciousness-like behavior presumably without consciousness, and the infrastructure look uncomfortably like we do.
What does consciousness have to do with reasoning? You're conflating two different things.
Consciousness is the what-it's-like of sensation you have as an organism from your sensory modalities and having a body. For human, that's colors, sounds, tastes, smells, feels, feeling of balance and what not that happen in perception, hallucination, memory, imagination, inner dialog and dreams along with emotions. Each sentient organism has their own sensations and emotions making up subjective experiences that nobody else has access to except inference from behavior and language (for humans).
That's not reasoning (just the having sensations/emotions). Reasoning is an adjacent activity that can be conscious but also unconscious. And it's hard to say that colors, sounds, emotions are illusions. What does that mean? We do experience them. There is something it's like to see color which is different from what it's like to feel pain or to be in love. And none of those (presumably) are what it's like when a bat experiences echolocation (there are many examples from the animal kingdom if one takes issue with echolocation).
Western tradition has firmly put pure reasoning in a cage. It serves its purpose only as a source of hypotheses, conjecture and thought experiments. Abrahamic religions, while still not accepted sensu proprio by many, are much less mystical than the Eastern Religions. You can see the difference in the unusual creative process of Ramanujan.
Which tradition are we to follow then whilst training the AI models? Pushed beyond their limits today's models tend to hallucinate and cannot be accepted sensu proprio. Philosophers will call it speculative reasoning. And yet, some speculation can be useful in breaking new grounds. Anyway, I have no answers. Just musings.
> I continue to believe that the most concerning aspect of neural networks is that they really strengthen the argument that consciousness is the illusion.
I think it’s not a question of it existing, but to what extent does conscious can control the organism (as opposed to being a passenger). There’s some intriguing research on the topic.
f6v · · focus · HN ↗
I'm not sure I understand this. Do we have evidence humans have an independent set of beliefs not shaped by knowledge and reasoning? If so, where do these come from?
I'm especially confused about a prior statement as a scientist:
> Three shortcomings prevent what chatbots do from qualifying as reasoning (in a way that a scientist might recognize).
How does a set of beliefs help with reasoning?
> Third, while the chains of thought chatbots produce look like deliberation, research has demonstrated that the bots often concoct them after the fact, reaching an answer by one route but reporting another.
We also often do the same as humans.
leopoldj · · focus · HN ↗
nateb2022 · · focus · HN ↗
Does a calculator reason its way to 2 + 2 = 4? No, it merely transforms.
An LLM is simply a much more complex version of this; except that its complexity hides a lot of what is otherwise reducible to a transformation of semantic, conceptual, and inferential relations. There are lots of NLP models prior to the introduction of modern "AI" that would make these representations more clear.
However I don't believe any of the aforementioned philosophers would ever consider reason itself as a transformation, as Kant especially spent a lot of time differentiating apprehension from transition.
leopoldj · · focus · HN ↗
1. <a href="https://www.youtube.com/watch?v=cyhXtJN2AQQ" rel="nofollow">https://www.youtube.com/watch?v=cyhXtJN2AQQ
morkalork · · focus · HN ↗
leopoldj · · focus · HN ↗
Models are now starting to show emergent behavior that they were not explicitly trained for. This is leading to the current debate.
scoofy · · focus · HN ↗
The interplay between deductive frameworks and inductive frameworks is still always buoyed by the black swan problem… which is a the heart of Hume’s solipsism. Kant’s pure reason never considered instinctual logical gates that come prepackaged in the mind. That seems frighteningly close to a prediction algorithm, no matter how much we wrap it in a concept of understanding. In the end, we’re just building a model with a minimal error rate.
nateb2022 · · focus · HN ↗
Based on what? Just because you can model the weather using math doesn't imply at all that the weather is the product of a mathematical model itself. I don't see how reasoning is different.
We deliberately construct LLMs so that they approximate certain relations typical of reasoning. Suppose they become extraordinarily successful at doing so. What exactly have we demonstrated? That these effects can be artificially reproduced through such a mechanism. But how would it follow that the thing being reproduced must itself be nothing more than that mechanism?
scoofy · · focus · HN ↗
Based on the face that we have zero explanation for consciousness. Based on the face that biologically neural networks are exactly a kind of neural network.
My point isn’t that we’re able to explain consciousness as an illusion. It’s that we’re able to explain consciousness-like behavior presumably without consciousness, and the infrastructure look uncomfortably like we do.
nateb2022 · · focus · HN ↗
[dead]
goatlover · · focus · HN ↗
Consciousness is the what-it's-like of sensation you have as an organism from your sensory modalities and having a body. For human, that's colors, sounds, tastes, smells, feels, feeling of balance and what not that happen in perception, hallucination, memory, imagination, inner dialog and dreams along with emotions. Each sentient organism has their own sensations and emotions making up subjective experiences that nobody else has access to except inference from behavior and language (for humans).
That's not reasoning (just the having sensations/emotions). Reasoning is an adjacent activity that can be conscious but also unconscious. And it's hard to say that colors, sounds, emotions are illusions. What does that mean? We do experience them. There is something it's like to see color which is different from what it's like to feel pain or to be in love. And none of those (presumably) are what it's like when a bat experiences echolocation (there are many examples from the animal kingdom if one takes issue with echolocation).
scoofy · · focus · HN ↗
>Each sentient organism has their own sensations
This does not follow besides feeling like it should.
leopoldj · · focus · HN ↗
Which tradition are we to follow then whilst training the AI models? Pushed beyond their limits today's models tend to hallucinate and cannot be accepted sensu proprio. Philosophers will call it speculative reasoning. And yet, some speculation can be useful in breaking new grounds. Anyway, I have no answers. Just musings.
scoofy · · focus · HN ↗
f6v · · focus · HN ↗
I think it’s not a question of it existing, but to what extent does conscious can control the organism (as opposed to being a passenger). There’s some intriguing research on the topic.
f6v · · focus · HN ↗
Yes, it reasons through machine instructions in the silicone.
Can someone explain how we reason that 2+2 is 4.