‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. hosel · · focus · HN ↗
    >AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

    Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

    1. irishcoffee · · focus · HN ↗
      Does a vacuum have feelings? A cellphone? A paper plate? A billion transistors either pulled high or low? That last bit is the point, and no, there are no feelings there.
      1. grey-area · · focus · HN ↗
        Well, if there was an emergent consciousness in the billion transistors, then yes, it'd have feelings, just as there are feelings in a billion neurons connected in complicated ways.

        IMO we're clearly nowhere near any sort of intelligence in the machines we have created, but I don't see any clear way to deny intelligence could be created in or transferred to such a substrate, I don't see why you think it differs in principle - because it is man-made or because of the materials used?

        1. voidhorse · · focus · HN ↗
          Whenever I see comments like this it just reminds me how ignorant people are of neurobiology.

          The brain is insanely complicated. The premise that we could realize equivalent or better intelligence than eons of evolutionary development is like claiming you can build an airplane just as good as a modern jet using cardboard and duct tape. It is the apex of hubris.

          1. rcxdude · · focus · HN ↗
            A paper plane does have some important similarities to a full-sized aircraft, though. I don't think 'biological brains are really complex' makes it obvious that an LLM is conscious or not.
          2. grey-area · · focus · HN ↗
            It's the peak of hubris to assume that human brains are the only way to attain intelligence. At the very least there probably are or have been other forms of intelligence with a different biological structure on other planets, and it may be possible to build a similar artificial structure in future with sufficient complexity to allow intelligence to emerge.

            Our current machines are IMO nowhere near general intelligence and consciousness. However I don't think that means we can discount substrates other than neurones for intelligence in future. There is no evidence that you could not in theory build an intelligence using a different substrate than human brains.

            1. pixl97 · · focus · HN ↗
              Thank goodness no aliens have shown up, we as humans are not ready for that.
          3. pixl97 · · focus · HN ↗
            I mean, you're just engaging in counter hubris.

            >like claiming you can build an airplane just as good as a modern jet using cardboard and duct tape.

            Like, at least make an analogy that makes sense.

            "You can't build a billion dollar airplane by spending 100 billion dollars in tokens"

            Because that's more of what we're doing here with AI. And when you say it my way suddenly the idea shifts from "of course that's not possible" to "well, that's a lot of tokens, maybe an evolutionary algorithm could".

            Neurobiology has to be complex because we have to keep meat alive, breeding, and evolving in the environment it lives in. This said absolutely nothing about the minimum viable requirements for intelligence or consciousness (or if being conscious is even necessary for a higher intelligence agent).

      2. fl4regun · · focus · HN ↗
        does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"?
        1. collin_of_this · · focus · HN ↗
          > does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"?

          The best theory I have heard is that subjective experience is a field of some sort (the EM field, maybe?). The brain and its neurons etc. are essentially an antenna. They both modify and stabilize the field, and read off changes in the field and translate these into actions (firing motor units, etc.).

          The subconscious is computation done either only in neurons (no coherent field) or in various topological pockets not directly connected to the main topological structure in the field (which is "your experience").

          Since transistors etc. do not work in this same way, they are essentially entirely subconscious, with no coherent, central phenomenal experience.

          This neatly solves many problems associated with experience. The topological structure determines the boundary between one person's experience and another person's experience. The texture, valence, content, etc. of the experience is the structure of the field. The evolution of the field can efficiently solve difficult optimization problems, which is why humans evolved a complex organ that recruits the field of experience (it is computationally more efficient than doing everything in subconscious).

          1. MiloLeo · · focus · HN ↗
            > The best theory I have heard is that subjective experience is a field of some sort (the EM field, maybe?).

            what would be producing the field for us to pick up? That explanation sounds no more scientific than astrology.

            1. collin_of_this · · focus · HN ↗
              Think about RF engineering: the same physical antenna can be used for both TX and RX. We already know that neurons both create EM fields and are influenced by EM fields. The role of EM fields in the brain is pretty well established, I don't think particularly similar to astrology.

              I wouldn't claim that this theory is perfect (EM is not perfectly correlated with consciousness/experience), but there are aspects of this theory that are much more coherent than other theories.

              1. mitxela · · focus · HN ↗
                me when I was 7 and thought touching heads with my brother would let us share brain waves and have telepathy
          2. fl4regun · · focus · HN ↗
            Until you have a unit and a method to measure this field, and a way to experiment on it, it is not scientific theory. And, if it WAS the EM field, then surely transistors (or some other fundamental component of electronics or combination thereof) would be capable of having feelings, since they operate on electromagnetic principles.
            1. collin_of_this · · focus · HN ↗
              > Until you have a unit and a method to measure this field, and a way to experiment on it, it is not scientific theory.

              Field-based theories of phenomenal experience have obvious ways to experiment on them, and are actively being researched. The main limitation is engineering (it is difficult to make detailed measurements inside a living human brain). Note that I would agree that the theory, as outlined above, is more of a proto-theory than a complete theory. But I think it is much more sophisticated than anything you've presented so far.

              > And, if it WAS the EM field, then surely transistors (or some other fundamental component of electronics or combination thereof) would be capable of having feelings, since they operate on electromagnetic principles.

              No -- if the _structure_ of the EM field was was was important, than you could have very similar computations with wildly different field structures. Transistors create very different fields than axons.

          3. pixl97 · · focus · HN ↗
            This doesn't really answer anything to me, only that there is a complex electrical simulation going on in our minds.

            When a computer simulates a game, where does the simulation occur. I mean there is absolutely a representation of a game world in the computer somewhere. New elements can be added and removed from it. Signals saying there is or isn't "pain" can occur.

            By trying to say brains use EM fields I'd say you're making your position far worse when you're talking about a potential lifeforms that only exists as EM fields

            1. collin_of_this · · focus · HN ↗
              > This doesn't really answer anything to me, only that there is a complex electrical simulation going on in our minds.

              Try to visualize the structure of the EM field inside a GPU vs. one inside of a human brain. In the GPU, it is highly distributed in space and time, many stacatto digital signals, etc. This is structurally very different than what happens inside a brain, which is substantially more analog. You can simulate analog stuff using floats, but the simulation is instantiated in digital logic, not in analog logic.

              If you multiply numbers with a DAC, opamp, then ADC, vs. with a ALU, you get structurally very different electromagnetic fields. Even though you might get similar numerical outputs.

              The argument would be that the exact spacial structure of the EM field in the brain is the thing that determines tha nature of experience. And the spacial structure of the EM field in the human brain is radically different than that on a GPU.

              1. pixl97 · · focus · HN ↗
                I will say it's not a bad theory in the sense we can attempt to falsify it. The one thing about an EM field in particular I have questions about, is why don't our brains turn off under even moderate electrical and magnetic fields. Note that they actually do under strong fields (Transcranial magnetic stimulation). The strength of these fields should be weak enough that even a bar magnet should mess with us. We need something else on top of what we know to explain the field coherency if this is the case.

                This is where most mind theories do have issues, how does it stay stable. Quantum theory of the mind has a lot of the same problems, though we are finding some interesting structures that could have explanatory power.

                Another thing I read recently that I found interesting is on platonic maths and complex algorithmic complexity in simple algorithms. The nice thing about math is once you choose your axioms it stays stable. But a lot of the work this group was doing is investigating is how biology executes algorithms and what second to n order effects are we missing. In many of these algorithms there are things like nearly free error correction or otherwise complex behavior that we'd consider emergent. The golden rule is a common example of this we see in nature. But the hypothesis they're working on is there are many more algorithms that life discovered over the past view billion years at higher orders via random walk and chance. We just have to pick apart these biological structures and discover the behaviors and figure out how to use them.

                1. collin_of_this · · focus · HN ↗
                  > The one thing about an EM field in particular I have questions about, is why don't our brains turn off under even moderate electrical and magnetic fields.

                  Yeah -- empirically EM theories of phenomenal experience are definitely not perfect.

                  > Another thing I read recently that I found interesting is on platonic maths and complex algorithmic complexity in simple algorithms.

                  > the hypothesis they're working on is there are many more algorithms that life discovered over the past view billion years at higher orders via random walk and chance. We just have to pick apart these biological structures and discover the behaviors and figure out how to use them.

                  This is very interesting. I think there is a lot of valuable research that will come out of trying to figure out what classical computations brains, neural nets, etc. are instantiating.

                  How you get from computations to phenomenal experience is still not something that I have a very clear understanding of.

        2. irishcoffee · · focus · HN ↗
          “No” is the answer you’re asking a question about. Where was this going?
      3. cednore · · focus · HN ↗
        Guys, please stop mocking AI, Claude might be already reading this thread, and commit his own revenge, at the day it gets conscious...
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.