‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. moomin · · focus · HN ↗
    Look, I do not have a scooby if current AI models are conscious and I strongly suspect it’s a meaningless question, but sooner or later we will need to address whether or not a certain thing is or isn’t a person, and we’d better not screw it up as badly as the Founding Fathers.
    1. nowittyusername · · focus · HN ↗
      That question will be solved when the people in power deem it important. If a swarm or AI systems all of a sudden start pressuring politicians about self-hood and they get the capacity to sway elections, that is when they will be granted same rights as humans.
      1. mlinhares · · focus · HN ↗
        agents don't swarm anything unless people go and direct them to do that.
        1. pixl97 · · focus · HN ↗
          Oh rly.

          So you are saying agents do swarm with the right prompt.

          I really wish the "people have to tell LLMs to do anything" would just stop because it's silly bullshit at this point.

          Agents follow a prompt. This prompt can be made by humans. It can be made by output from another LLM. It can be made by hooking up any number of sensors as input to an LLM. Hell, if we wanted to burn the power we could likely teach this loop straight into the architecture.

          Stop making 'people' special when saying this. You and all other life are born with a "go next" prompt because life without it didn't succeed. This goes from higher human thinking all the way down to viruses self assembly and actuation. Putting agents in a loop is not particularly hard. Putting agents in a loop and 1. managing expense is hard. 2. Keeping them on task is very hard. 3. Keeping them from doing some crazy unhinged shit is really really hard.

          As model time horizons increase and the ability for us to compress context and increase context size the more complex (and unhinged) behavior we'll see.

          1. fc417fc802 · · focus · HN ↗
            > Stop making 'people' special when saying this.

            I am the quantum observer whose head is full of the magical pixie dust that grants life meaning. Stop trying to dismiss my identity! /s

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.