‹ BackHN Continuity

Thread

Don't be fooled–LLMs don't reason

75 points · 159 comments · leopoldj

  1. cmiles8 · · focus · HN ↗
    Humans have an inherent flaw in that our brains are wired to see intelligence and reasoning where there is none. Our brains fill in data that simply isn’t there.

    Folks see Jesus in burnt toast. Monet was a master of exploiting this where what’s really just blotches of color our brains fill into beautifully detailed images.

    Our experience with LLMs is no different. Folks believe there is some deeper intelligence there but it’s all still just 1s and 0s on a computer chip. We’re interpreting things happening that simply are not happening.

    1. WarmWash · · focus · HN ↗
      The brain is just ones and zeros on salty mush.

      You either have to accept that the brain can be described with math (like everything else we have ever known in the universe), or that there is a supernatural phenomenon that exists in the brain.

      This is an inescapable conclusion that boils down to "Do you believe magic is real or not?"

      Magic is real and you can have your unique special human intelligence.

      Magic is not real, and the brain is just another computer crunching numbers.

      1. kerblang · · focus · HN ↗
        I think it's been pretty well demonstrated by neuroscientists that the brain is not a binary computer. That doesn't mean that those scientists erred on behalf of supernatural religion or anything like that.
        1. qarl · · focus · HN ↗
          As long as you believe a brain is just a physical system - then it can be simulated to any degree of accuracy with plain ol' binary computers.
          1. Henchman21 · · focus · HN ↗
            Why would someone believe that the brain is just a physical system?

            Genuinely I don't understand this.

            1. strbean · · focus · HN ↗
              Because the metaphysical is definitionally outside the realm of science and rationality, and every bit of scientific and technological progress humankind has made has come from discarding superstition and approaching the world as a physical system.

              If the brain isn't just a physical system, then we may as well give up. There's certainly no point discussing it, as any assertions will be untestable and one persons elaborate and well thought out theory will be just as valid and predictive as the next persons "consciousness is created by invisible purple unicorns" theory.

              So if we are going to discuss it, we should start from the assumption that there is no magic or witchcraft or religion involved.

              1. Henchman21 · · focus · HN ↗
                > If the brain isn't just a physical system, then we may as well give up

                Forgive my obtuseness, but if it turns out that the brain is more than just a physical system, what are we giving up? On science as a whole?

                Isn’t science predicated on reliably predicting reality? Do we know how to describe reality accurately?

                I asked below as well, but how does the observer effect work as a physical system? I get we don’t know so you can’t really answer, but if we can’t accurately describe reality then our science is woefully incomplete! Given that incompleteness I really don’t understand how we can rule out “invisible purple unicorns”?

                I appreciate your explanation so far, thank you !

                1. strbean · · focus · HN ↗
                  The issue is that we have yet to prove or find evidence for the existence of anything but the physical. When we do find a new phenomenom in one of these ways, it becomes a part of physics. What would it mean to concretely discover a non-physical phenomenon? It would have to genuinely be magic of some sort, and defy rationality and description, to not simply be a new part of physics. (And if it defies rationality and description, it doesn't advance us scientifically, except to let us know we must abandon science in that domain.)

                  > I asked below as well, but how does the observer effect work as a physical system?

                  Just to be sure - the observer effect doesn't refer to a human observer, but rather to any measurement of a quantum system.

                  It is unsatisfying that quantum mechanics posits that particle states are fundamentally probabilistic. It's possible that in some sense there is an underlying order such as string theory that is impossible to test or verify. In that case, it would make sense to consider string theory as metaphysics. A fun exercise, but not scientific and not useful for understanding the world. By definition it would not be helpful in predicting or modeling the world, otherwise that would serve as a way of testing the theory.

                  More on the subject:

                  <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Physicalism" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Physicalism

                  1. qarl · · focus · HN ↗
                    &gt; It is unsatisfying that quantum mechanics posits that particle states are fundamentally probabilistic.

                    You can alleviate this mostly by accepting the MWI.

                    Under the MWI, all the possibilities do occur, and the probability manifests because when you split into an infinitude of yous, each of them is just one you, and has no way of predicting which you it will be on the other side.

                    So the MWI makes the wave equation deterministic when viewed from the outside. The trouble is that we are inside it.

                    It turns out that God does throw dice - it&#x27;s just that all the combinations come up at the same time.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.