‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. hosel · · focus · HN ↗
    >AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

    Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

    1. pton_xd · · focus · HN ↗
      If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.

      The current implementation as stateless matrix multiplication... yeah there's nothing going on there.

      1. myrmidon · · focus · HN ↗
        Is the ability to form and retain long-term memories a necessary precondition for rights in humans though?

        Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).

        1. qlte · · focus · HN ↗
          This argument would be difficult to reconcile if we didn't have billions of examples of humans forming memories plus modern evolutionary biology to trivially demonstrate such cases are rare pathologies but still human beings.
        2. joshheitzman · · focus · HN ↗
          > Is the ability to form and retain long-term memories a necessary precondition for rights in humans though?

          For a species / class of being yes it is necessary. Some members of that species / class being exceptions does not make the generalization invalid.

          Without the ability to form and retain long-term memories we wouldn't have developed language and abstract concepts like rights.

          Whether people like it or not, rights aren't natural or god-given. Rights are only enforced or protected by the tip of a spear (or point of a gun, or hover of a drone). There are no rights in nature and they are no where to be found when social order breaks down.

          Animals can't make war for their rights, so they only get the rights we deign to give them. It's not nothing, but its not the same. There is no question to me that my dogs are conscious, but its not at the same level as humans.

          There is no question for me that LLMs lack consciousness as they are stateless.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.