‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. hosel · · focus · HN ↗
    >AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

    Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

    1. pton_xd · · focus · HN ↗
      If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.

      The current implementation as stateless matrix multiplication... yeah there's nothing going on there.

      1. vidarh · · focus · HN ↗
        A model detached from a harness, sure. But we routinely do add persistent memory to systems incorporating LLMs.
        1. dgellow · · focus · HN ↗
          They don’t have persistent memory though. Models are static, what you have is a model that will reprocess the whole series of prompts + some data fetched from a data store. A harness is just a while loop prompting an LLM and executing tool calls, but the model is purely static
          1. vidarh · · focus · HN ↗
            The model does not. The system it runs in can. My agents have persistent memory. It is not at all clear whether or the distinction that the long-term memory is external from the model matters.
            1. chrisjj · · focus · HN ↗
              [delayed]
              1. vidarh · · focus · HN ↗
                Does your disc drive have the complexity of an LLM that shows behaviour consistent with having emotions?

                I'd consider that a prerequisite, but not sufficient, for that question to make sense.

                1. chrisjj · · focus · HN ↗
                  [delayed]
                  1. vidarh · · focus · HN ↗
                    Your second sentence does not support the claim in your first sentence; it's merely a second unsupported claim. It is also entirely irrelevant to the argument.

                    The threshold for behaviour consistent with having emotions (note: not behaviour sufficient to prove emotions) was one I purposefully set extremely low, because your comparison to a disk drive is so ludicrous, and yet your disc drive will still not clear it.

                    Do you have an actual argument?

                    1. chrisjj · · focus · HN ↗
                      [delayed]
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.