‹ BackHN Continuity

Thread

LeCun has "zero concerns" about AI wiping out humanity, recent "rogue" incidents

400 points · 756 comments · Anon84

  1. jswelker · · focus · HN ↗
    LeCun is saying AI will be totally safe as long as we have competent and well aligned corporate management.

    What could go wrong?

    1. qayxc · · focus · HN ↗
      He has a point, though. The LLMs (or any AI for that matter) can't do anything. They can't. It's a function call that ingests symbols and spits out symbols and that's it.

      100% of its actual capabilities are tied to harnesses (the actual "agent"), i.e. ordinary deterministic programs that are connected to networks or machines and enable interaction with the outside world. This part (the part that can do harmful things) is fully under human control and all the recent headlines about "agents going rogue" are - as someone (forgot who) put it - akin to strapping a weedwhacker onto a dog and letting it run wild.

      The tech itself is safe as far as real-world interactions go - the weakness lies in unchecked access to systems surrounding it. It's not safe at all when it comes to human interaction (lots of ongoing lawsuits demonstrate that), though. There is real danger here, but it has nothing to do with doomsday scenarios ala Terminator or I,Robot and more with total corporate control over the lives, perception of reality, and abilities (like critical thinking) of people.

      1. stratos123 · · focus · HN ↗
        > It's a function call that ingests symbols and spits out symbols and that's it. 100% of its actual capabilities are tied to harnesses

        This is a bad and misleading way to think about it. Note that it's trivial to make the harness that you claim capabilities are tied to (the LLM itself could write it from scratch in one shot), but no matter how good a harness you have, it won't make gemma4:e4b capable. That's because what actually gives capabilities is the LLM's intelligence - or if you prefer not using that term, the fact that the probability distributions the LLM spits out depend on the context in useful ways.

        1. qayxc · · focus · HN ↗
          I think we're talking about fundamentally different perspectives here.

          I'm not talking about what the LLM does internally. If a metaphor helps, here's one to help you understand what I was trying to get at:

          Imagine an evil genius that has no eyes and no limbs. Everything they could learn about the world is presented to them by means of some person describing it to them through words. They have no way of directly interacting with the world and rely on someone executing any action they want to take and describe the outcome to them. Now how dangerous would you say such person would be? How dangerous could they become?

          That's what I was getting at. Replace person with LLM (or any other AI system). Replace the person that communicates with an external interface (the harness) and I hope you understand. It doesn't matter whether the LLM could generate the harness by itself - it still is just a bunch of weights sitting in memory being run by an execution engine. That's what it fundamentally is, whether you like it or not. It cannot do anything on its own - and no, not even writing files. It's the execution engine that translates the numeric output into words (or images or video or audio) and the layer above (the harness) that takes that output and interprets it to execute actual actions.

          This is not about what you or I think about the internal capabilities of the model - that's irrelevant to the conversation and you can replace LLM with a random token generator and the point still stands. The model itself is incapable of performing actions - from reading files to writing files, to controlling physical machines. All that is and HAS to be done by external interfaces outside the control of the model.

          1. jswelker · · focus · HN ↗
            Yes, now say there are several major companies and an entire open source ecosystem dedicated to creating superpowerrd exoskeletons for this limbless villain. Is that cause for concern? I say yes.
            1. qayxc · · focus · HN ↗
              Sure, but that's a problem of regulating and controlling the production and rollout of superpowered exoskeletons, not an issue of of terrorising limbless villains. Because, guess what, the same dangerous tech can be used to turn squirrels into dangerous monsters, too. So you see where the actual problem is. Giving full unchecked access to systems to any random person would be considered reckless and foolish. Replace person with AI model and it's called the future...
              1. jswelker · · focus · HN ↗
                I'll gladly take my chances with the exo suit squirrel over the evil villain.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.