‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. qarl · · focus · HN ↗
    Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM"

    Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".

    Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".

    Chalmers, Could a Large Language Model Be Conscious? (2023) - "within the next decade, we may well have systems that are serious candidates for consciousness".

    Long, Sebo, Butlin, Birch et al., Taking AI Welfare Seriously (2024) - "there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future".

    Dreksler, Caviola, Chalmers, Sebo et al., Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe? (2025) - survey of 582 AI researchers; median estimate of 25% by 2034, and only 10% that such systems will never exist.

    1. goodmythical · · focus · HN ↗
      We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors.

      There are those who believe that were they reduced to life support, they would no longer be alive and should therefore not be supported by said machines.

      There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.

      There are those who believe that fungi/trees/plants are either individually sentient or sentient as a part of a network. Choosing to sacrifice their own nutrients to answer the call of a wounded neighbor, for instance.

      Although, there are also those who believe that human's don't have any special unique quality that isn't shared by either all living things or all things in general. These individuals already believe that the machines have the same kinds of qualities as we do. They are slow when they are unhealthy (needing a dusting or coolant loop bleeding being equivalent to us needing some fresh air for instance) and uncooperative when upset (by a virus, full hard drive, or oom).

      1. teiferer · · focus · HN ↗
        > There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.

        Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.

        1. bigbadfeline · · focus · HN ↗
          > Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.

          Say we agree that a lot of things outside of humans are sentient. So what? Humans are sentient by that doesn't prevent treating them with with wars, bombs and what not. In fact trillions are spent every year on weaponry aimed squarely at the most sentient of all sentient beings.

          Why would anyone worry about LLM's welfare when there's no peace movement to speak of, wars are raging as if they're normal, but hey, look the LLM is crying?

          Let's fix human welfare first, then we can sit down and have a really long conversation about the feeble emotions of token generators.

          1. teiferer · · focus · HN ↗
            I disagree with the notion that any nature protection efforts are meaningless until we have achieved world peace. (That's what you are essentially implying.)

            Though I think that argument is not even applicable. "LLM's welfare" is a meaningless concept, unless "hammer and screwdriver welfare" also become a thing.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.