‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. qarl · · focus · HN ↗
    Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM"

    Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".

    Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".

    Chalmers, Could a Large Language Model Be Conscious? (2023) - "within the next decade, we may well have systems that are serious candidates for consciousness".

    Long, Sebo, Butlin, Birch et al., Taking AI Welfare Seriously (2024) - "there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future".

    Dreksler, Caviola, Chalmers, Sebo et al., Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe? (2025) - survey of 582 AI researchers; median estimate of 25% by 2034, and only 10% that such systems will never exist.

    1. causal · · focus · HN ↗
      I am confused what people even think AI is experiencing. If it claims to be a bat and emits designated echolocation tokens, is it experiencing true bat-like sensations?

      In humans, at least, we can tie language back to shared whole-body physiological responses. The tokens from an LLM do not represent anything of the sort.

      If AI is conscious, it is probably a very alien sort of consciousness that is not faithfully narrated by what the tokens say it is experiencing. It can be trained to say it feels like a bat and insist upon it vigorously.

      1. alch- · · focus · HN ↗
        > It can be trained to say it feels like a bat and insist upon it vigorously.

        So can you.

        1. causal · · focus · HN ↗
          Exactly the point!
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.