Thread

1. Svip · 1791108382 · focus · HN ↗

   If we talk about pain specifically, it is will established that physical and phycological pain can be observed on instruments; stress produces chemical reactions, that can make needles flicker on meters. Even plants can feel pain in an observable manner.

   As far as I am aware, the GPUs don't work harder, don't heat up more, and nothing in their mathematical equations are different, when the result of these vector calculations produce text, that to humans look like someone in pain. Whereas real pain is objectively observable, LLMs' pain - so far at least - is entirely subjectively observable.

   That being said, I'm a fan of Star Trek's depiction of Data in TNG and the Doctor in VOY fighting for their rights to be treated as equals to the biological crew; and while I support their plight, as far as I can remember, the shows never made a claim that their pain was observable by instruments, thus making the judgement an entirely subjective decision (particularly in the case of the Doctor). Though I'd be glad to be corrected in this matter.

   1. bondarchuk · 1791108635 · focus · HN ↗

      >nothing in their mathematical equations are different, when the result of these vector calculations produce text, that to humans look like someone in pain.

      So at least something is different, namely the text output changes (and therefore some internal state too, of course). I think your analogy is too simple, it does not have to be the case that the GPU has to act like the equivalent of the body for an LLM in this respect.

      1. danaris · 1791108795 · focus · HN ↗

         But this is basically just

         Human: "Act like you're in pain."

         Computer: "Aaugh! It hurts! Why?! No more!"

         Human: "OMG! The computer feels pain!!"

         1. Kim_Bruning · 1791109017 · focus · HN ↗

            Almost.

            Human: "Let me just poke this vector and see what happens"

            LLM: "OUW!"

            The underlying experiment didn't tell the LLM what to do. Instead, the experimenters modified a vector and observed the outcome; thus showing that there is a vector that makes an LLM go "Ouw" . People called it a "pain vector", because that's easier to remember than , idk, LVF12345.

            1. Topfi · 1791109445 · focus · HN ↗

               That misinterprets grossly what these papers have shown, responding with pain related language, something an LLM has been taught on in its training data is not akin to pain itself. The way interpretability findings are reported on by companies, the media and hype merchants is dangerously flawed, so not hard to make that mistake. Just look at the irresponsible and barely accurate mess that was coverage of J-Space vs the actual, very valuable research.

               I’ll continue what I say every time this discussion (be it consciousness, pain, self awareness, etc) comes up:

               Assertions as monumental as these require equally monumental evidence. And despite certain labs and their employees making statements, their actions betray that they do not believe this to be the case.

               If every LLM session were a conscious being, existing regulation for animals (controversially considered both sentient and economically useful) would need to be applied on each of these sessions. I suspect no lab will take that conclusion, for obvious reasons.

               1. Kim_Bruning · 1791113541 · focus · HN ↗

                  I think you've read me upside-down.

                  Paper says you poke the vector(s), the LLM exhibits aversive behaviours. You picks your scoring, you gets your operationalization.

                  That gets you an empirical result. Short of a replication failure, we can't really argue with that anymore.

                  What we can do is be very cautious as to how we interpret it.

Unofficial Hacker News client; not affiliated with Y Combinator.