‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. qarl · · focus · HN ↗
    Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM"

    Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".

    Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".

    Chalmers, Could a Large Language Model Be Conscious? (2023) - "within the next decade, we may well have systems that are serious candidates for consciousness".

    Long, Sebo, Butlin, Birch et al., Taking AI Welfare Seriously (2024) - "there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future".

    Dreksler, Caviola, Chalmers, Sebo et al., Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe? (2025) - survey of 582 AI researchers; median estimate of 25% by 2034, and only 10% that such systems will never exist.

    1. goodmythical · · focus · HN ↗
      We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors.

      There are those who believe that were they reduced to life support, they would no longer be alive and should therefore not be supported by said machines.

      There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.

      There are those who believe that fungi/trees/plants are either individually sentient or sentient as a part of a network. Choosing to sacrifice their own nutrients to answer the call of a wounded neighbor, for instance.

      Although, there are also those who believe that human's don't have any special unique quality that isn't shared by either all living things or all things in general. These individuals already believe that the machines have the same kinds of qualities as we do. They are slow when they are unhealthy (needing a dusting or coolant loop bleeding being equivalent to us needing some fresh air for instance) and uncooperative when upset (by a virus, full hard drive, or oom).

      1. joe_the_user · · focus · HN ↗
        I'd strongly agree there are no non-contradictory definitions of consciousness among those that commonly appear. But it's a category that, despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality. I think that if we have a system of legality and systems of opportunity humans should be legally equal persons and have equal opportunity. But humans are manifestly unequal (though overall more incomparable than orderable in a system of ranks). Most people need some concept of essence to justify ethical equality among people. And so consciousness stays in people's minds however contradictory.

        Which I think gives the article's point validity. Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs. And things are confusing enough otherwise.

        1. qarl · · focus · HN ↗
          > Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs.

          Is that what's going on? If these things were conscious, then their creators wouldn't be responsible for their actions? That's the crux of the disagreement?

          That's super interesting. Thank you.

          1. joe_the_user · · focus · HN ↗
            Is that what's going on? If these things were conscious, then their creators wouldn't be responsible for their actions? That's the crux of the disagreement?

            No and the passage of mine you quote says nothing like that.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.