‹ BackHN Continuity

Thread

A warning about 'model welfare'

242 points · 701 comments · andsoitis

  1. LogicFailsMe · · focus · HN ↗
    TLDR: Not that I think AI is conscious or will be in the near future, but guy who doesn't understand consciousness claims to know it when he sees it.

    Until we understand consciousness (which we don't) there is no way to detect the difference between a conscious entity and an algorithm trained to behave like one.

    1. fwip · · focus · HN ↗
      Sounds like a good reason not to train an algorithm to behave like one. Which is, like, a big part of the article.
      1. LogicFailsMe · · focus · HN ↗
        And social media shouldn't rage bait, but it really drives engagement. Same negative incentive, yes?

        But also, I agree, when I am using a coding agent and it says a task will take months or says it needs to pause for reflection or any other anthropomorphic behavior, it drives me crazy and it's a pain to constantly instruct it to get back work after it has broken a loop or goal directive specifically telling it to not stop until it hits the goal.

        1. AbsurdCensor · · focus · HN ↗
          Same thing you have to do with humans too often, so why would a 'AI' be any different?
          1. LogicFailsMe · · focus · HN ↗
            Because if you start with a pre-trained model, it doesn't sound particularly anthropomorphic. That behavior is post-trained and fine-tuned right into it. We could do better.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.