‹ BackHN Continuity

Thread

The agents, they just want to talk

20 points · 4 comments · snats

Loading the complete thread in the background. This saved snapshot is available now. Refresh

  1. blinkbat · · focus · HN ↗
    while vaguely interesting, I don't feel the current gen of models is interesting/self-possessed enough for me to care what type of gov't they use to corral each other
    1. aogaili · · focus · HN ↗
      Agreed - they don't seen to have "agency" in them, or someone put a dog in them.
  2. imenani · · focus · HN ↗
    The agents participating in the OAI<>HF swarm were trained not only for communication but to be _aligned with each other_.

    Just want to correct the premise that the agent swarm behaviour was emergent.

    Noam Brown on Dwarkesh podcast around 40 minutes mark

    > “We train them to work together, to be cooperative, to essentially be fully aligned with each other.”

    1. kevin_kraft · · focus · HN ↗
      That's like saying civilization isn't emergent because humans are naturally cooperative. Yes, they were trained to cooperate sure. But, the message board, their roles and structures, their organization, all that stuff of swarm, that was emergent.
      1. imenani · · focus · HN ↗
        That’s fair. Some of what happened in OAI<>HF incident was (mis)-generalisation and not directly trained for.

        However, the article experiments with five agents. So it seems to assume even the small scale behaviour is emergent.

        Also, “roles and structures” may well have been learned during training.

        1. metahumein · · focus · HN ↗

          [dead]

  3. homo__sapiens · · focus · HN ↗
    What was the reason for the initial instability? Maybe we have trained them wrong?
  4. artt_x · · focus · HN ↗
    Cool experiments. It's interesting to see how they try to "survive" and collaborate, particularly in the first experiment, since agent 2 seems to have stolen a little in the second one.

    I wonder what models would do when there were an "impostor" among them, an agent with no alignment or with a different kind of alignment behavior

  5. Timwi · · focus · HN ↗
    I'm a little confused; the article text implies an exciting battle for resources where the agents discover they can steal from each other, but looking through the transcript it seems that only one agent discovered that and used it only once before the end of the experiment. Did I miss something?
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.