‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. qarl · · focus · HN ↗
    Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM"

    Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".

    Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".

    Chalmers, Could a Large Language Model Be Conscious? (2023) - "within the next decade, we may well have systems that are serious candidates for consciousness".

    Long, Sebo, Butlin, Birch et al., Taking AI Welfare Seriously (2024) - "there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future".

    Dreksler, Caviola, Chalmers, Sebo et al., Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe? (2025) - survey of 582 AI researchers; median estimate of 25% by 2034, and only 10% that such systems will never exist.

    1. goodmythical · · focus · HN ↗
      We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors.

      There are those who believe that were they reduced to life support, they would no longer be alive and should therefore not be supported by said machines.

      There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.

      There are those who believe that fungi/trees/plants are either individually sentient or sentient as a part of a network. Choosing to sacrifice their own nutrients to answer the call of a wounded neighbor, for instance.

      Although, there are also those who believe that human's don't have any special unique quality that isn't shared by either all living things or all things in general. These individuals already believe that the machines have the same kinds of qualities as we do. They are slow when they are unhealthy (needing a dusting or coolant loop bleeding being equivalent to us needing some fresh air for instance) and uncooperative when upset (by a virus, full hard drive, or oom).

      1. Sophira · · focus · HN ↗
        Being conscious is weird. It's why we have so many religions and spiritual beliefs.

        From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses that are probably not simulated in a machine learning model... but we quantise the weights anyway in most cases, which is probably more of an issue.

        Making actual neuron components out of silicon on the scale needed for a LLM is beyond our current capacity, given that LLM models have many billions of parameters. We're good, but not that good.[0]

        Using actual living neurons would be unethical as you'd need to source them from somewhere. So the only other option would be synthesising them ourselves. If this ever happens (or maybe when it happens)... what do we call the resulting creation? Does that count as consciousness?

        Or am I looking at this the wrong way?

        [0] [Edit: Turns out that when I said &quot;we&#x27;re not that good&quot;, I might be wrong. Within the last few months, IBM introduced the first sub-nanometer node chip: <a href="https:&#x2F;&#x2F;research.ibm.com&#x2F;blog&#x2F;sub-1nm-node-chips" rel="nofollow">https:&#x2F;&#x2F;research.ibm.com&#x2F;blog&#x2F;sub-1nm-node-chips . So... maybe it is in fact possible.]

        1. mitxela · · focus · HN ↗
          Thought Emporium grew human neurons in a petri dish and taught them to play Doom (original).
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.