Thread

1. jswelker · 1791071747 · focus · HN ↗

   LeCun is saying AI will be totally safe as long as we have competent and well aligned corporate management.

   What could go wrong?

   1. qayxc · 1791106207 · focus · HN ↗

      He has a point, though. The LLMs (or any AI for that matter) can't do anything. They can't. It's a function call that ingests symbols and spits out symbols and that's it.

      100% of its actual capabilities are tied to harnesses (the actual "agent"), i.e. ordinary deterministic programs that are connected to networks or machines and enable interaction with the outside world. This part (the part that can do harmful things) is fully under human control and all the recent headlines about "agents going rogue" are - as someone (forgot who) put it - akin to strapping a weedwhacker onto a dog and letting it run wild.

      The tech itself is safe as far as real-world interactions go - the weakness lies in unchecked access to systems surrounding it. It's not safe at all when it comes to human interaction (lots of ongoing lawsuits demonstrate that), though. There is real danger here, but it has nothing to do with doomsday scenarios ala Terminator or I,Robot and more with total corporate control over the lives, perception of reality, and abilities (like critical thinking) of people.

      1. brainwad · 1791116393 · focus · HN ↗

         This is like saying cars don't kill people, because if nobody drives them faster than 3mph there's no problem. The _whole_ promise of cars is that they can go fast, just like the whole promise of AI is offloading thinking to a computer. If AIs are unsafe without close human supervision and checking every interaction with the real world, they are unsafe full stop.

         1. qayxc · 1791117496 · focus · HN ↗

            The analogy would be more fitting if you had said "cars don't kill people if every drivers has proven skills, never drives impaired, keeps the speed in line with weather and road conditions and stays on actual roadways". You know, like everyone should, regardless of whether they're driving a high performance sports car.

            To keep with the analogy: cars have seatbelts, airbags, ABS, ESP, lights, horns, crumple zones, emergency braking systems, roads have speed limits, there are traffic stops, insurance, regular inspections (not in all countries), etc. etc.

            So what's unsafe here? The car or roads without speed limits, complete lack of safety measures (both active and passive), absence of any supervision and no insurance? That's the problem. It's not the models themselves - they can spit out tokens by the billions, there's no risk there.

            You wouldn't give full access to your phone, your computers, your house keys and your credit cards to any stranger on the street now, would you? How is it then, that people act all surprised when a non-deterministic machine that's optimised to achieve goals while taking all the shortcuts it can, suddenly uses the tools handed to it in unexpected ways? That's a failure on the operator's side, not an inherent danger within of the model.

            1. brainwad · 1791118236 · focus · HN ↗

               The cars are still unsafe at speed. All those mitigations reduce the risks but do not eliminate the inherent danger. Sandboxing autonomous LLMs is similar, there is no way to mitigate the inherent safety problem entirely while preserving the power of the thing (an LLM without tools is safe in the way a car without an engine is - safe and useless). But safety is just one thing people optimise for; if it's convenient enough people will accept imperfect safety (as with cars).

               1. qayxc · 1791132983 · focus · HN ↗

                  And therein lies the problem. I listed all the things that differentiate cars from current AI models: cars require a license to drive, they are subject to heavy regulation (insurance, registration, inspections, etc.), there's road signs, police, incident statistics, recalls in case of defects etc. etc. NONE of that is currently in place for AI models and the systems surrounding them. So the analogy is flawed on every level. I don't buy the convenience is just too appealing narrative when your own analogy clearly demonstrates what is required in order to roll out dangerous technology to the masses, while NONE of that is in place in the context of AI models.

                  1. brainwad · 1791133399 · focus · HN ↗

                     My point is that even if all of it were in place, LLMs with any reasonable harness would still be dangerous. Not to belabour the analogy, but while deaths per km travelled have decreased a lot for cars with better safeguards, cars still kill hundreds of thousands of people per year.

                     Your initial argument of "it's just the harness/user" is wrong; reasoning LLMs are inherently dangerous unless locked in an unbreakable box, which is tantamount to not using them at all. We can make them safe_r_ with better tooling, but they can't be made safe.

            2. lelandfe · 1791122614 · focus · HN ↗

               What did your momma tell you about playing in the street?

               Cars are inherently dangerous. They’ll still be dangerous when computers are driving them all.

               1. qayxc · 1791132082 · focus · HN ↗

                  I'm sick of this strawman nonsense. You are attacking a false analogy. Discuss the actual meat of the argument instead.

                  1. pixl97 · 1791135673 · focus · HN ↗

                     I have to say all your arguments are dry bones, stripped and left in the desert.

                     At this point all I can say about their contents is "They are not even wrong".

                     Please join us in the real world with how we see this product is not only dangerous, but getting more dangerous with each iteration, and no one seriously talking about controls on it.

Unofficial Hacker News client; not affiliated with Y Combinator.