‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. hosel · · focus · HN ↗
    >AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.

    Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.

    1. pton_xd · · focus · HN ↗
      If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.

      The current implementation as stateless matrix multiplication... yeah there's nothing going on there.

      1. shawnz · · focus · HN ↗
        So if a human loses their memory and their neutral plasticity (with age for example), then they are no longer capable of experiencing suffering?
        1. staticman2 · · focus · HN ↗
          This framing is a trap. It states "Dehumanize a sick patient OR concede the AI tech bro claim about LLMs"

          It's also a rhetorical move I've seen several times here...

          1. shawnz · · focus · HN ↗
            It's not a "trap", it's the real argument which makes me hesitant to dismiss consciousness of machines. Yes, it seems silly to dismiss a sick patient as having a lesser consciousness, which is exactly the point
            1. staticman2 · · focus · HN ↗
              It seems silly to dismiss the patient because we know when healthy they are conscious. This doesn't really analogize to LLMs.
              1. mitxela · · focus · HN ↗
                So you think a dementia patient might be a p-zombie during the time they have dementia?
                1. staticman2 · · focus · HN ↗
                  > "This framing is a trap. It states "Dehumanize a sick patient OR concede the AI tech bro claim about LLMs"
            2. joshheitzman · · focus · HN ↗
              It's not a real argument. Dementia patients can have lucid periods that plainly show their long-term memory isn't gone. They're mental state is obviously degraded, but its disingenuous to say they have no long-term memory and that they are any way similar to LLMs.
              1. staticman2 · · focus · HN ↗
                Yeah I was gonna add I know there are case studies where sick humans don't consciously have access to long term memories but will still respond nonverbally to people they "recognize".
          2. qlte · · focus · HN ↗
            Yes it's seemingly the go-to rebuttal, I've lost count of how many times I've seen it with barely any variation.

            Not sure if it's a product of LessWrong or Slate Star Codex or something. I kinda suspect it might be LW, given how it superficially resembles an unanswerable philosophical paradox but in reality is about as challenging as a question posed in a 200 level philosophy course like introduction to ethics.

          3. mitxela · · focus · HN ↗
            It's debunking GGP by calling out the "proving too much fallacy"
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.