Thread

The Download: why AI's latest breakthroughs and fears may be more hype than rea

49 points · 128 comments · joozio

bananaflag · 1790163465 · focus · HN ↗
> Describing them as “superintelligence” or “rogue models” ascribes agency to products rather than to the companies building them. This framing markets these companies’ products as “superhuman” and, at the same time, helps the companies evade accountability for their actions.
If someone discovered how to summon demons to aid them in robbing banks the main issue wouldn't be "but who has the responsibility for the crime, the human or the demon", it would be "OMG DEMONS".
Seriously, I don't get what sort of world these people are living in.
1. rubendev · 1790164184 · focus · HN ↗
  That's not what the LLM hacking accidents have been like at all though. To improve the analogy, it would be like summoning a totally passive demon, giving it weapons and placing it next to a bank, place a small fence around it, and then command it to perform a totally safe "exercise" that is exactly like robbing a real bank.
  LLMs do not have agency, they are just producing tokens based on a prompt that a person entered, and some of these tokens can trigger the tools that a person gave them access to.
  1. 0xDEAFBEAD · 1790165467 · focus · HN ↗
    Dwarkesh, for one, defended his use of "anthropomorphic" language.
    >Sacrificing now yields Oracle for team, but forfeits our chance, question mark. But other agents were pushing it, sending a message saying, go, sacrifice final now. And then EarlyBig eventually agreed, thinking to itself, our own utility may be already near zero. Sacrifice rational.
    <a href="https://www.youtube.com/watch?v=X50zezLFWWI#t=2m" rel="nofollow">https://www.youtube.com/watch?v=X50zezLFWWI#t=2m
    My suspicion is that many of the "LLMs do not have agency" folks just haven't learned much about the details of the incident. It was specifically with LLM agents that were trained to be more persistent than usual.
    If you're going to say that the incident details don't matter, and LLMs lack agency because it's all based on floating-point math--why can't I say that humans lack agency, because it's all based on neurons firing?
    embedding-shape · 1790166088 · focus · HN ↗
    > If you're going to say that the incident details don't matter, and LLMs lack agency because it's all based on floating-point math--why can't I say that humans lack agency, because it's all based on neurons firing?
    They're not saying this, we're saying LLMs lack agency because if you run a LLM and don't send any prompts, literally nothing happens.
    Instruct it to "Find the right answer regardless of where", it'll do exactly this. They're passive in that they don't act by themselves, somewhere, at one point, someone "told" the LLM to "do something" and that's the cause and the reason for saying "LLMs do not have agency".
    0xDEAFBEAD · 1790166732 · focus · HN ↗
    Imagine a super-competent Navy SEAL who just sleeps in the barracks unless his commander tells him to do something. Does the Navy SEAL lack agency? As a target of this Navy SEAL, should you be reassured by the fact that they'll be sleeping in the barracks unless their commander tells them to do something?
    embedding-shape · 1790170185 · focus · HN ↗
    Imagine a car, that does nothing until a person controls it. If a person uses that car to kill, who is responsible, the person or the car?
    Is it really so unbelievable that tools, objects and inanimate things don't have agency? And that they different from a person?
    pixl97 · 1790173807 · focus · HN ↗
    Imagine you have a self-driving car and it's in a parking lot a mile down the road. You tell it to come pick you up. About half way to you the car is passing an elementary school makes a sharp left and mows down 30 children. Time to put you in jail for murder, right?
    The mental model you have is one that existed in the past and is broken now the future arrived. Bad analogies do not even begin to explain what is occurring.
    mylidlpony · 1790176709 · focus · HN ↗
    In the case of self-driving car the legal case is pretty clear - the maker of the self-driving car is liable for the death in this situation. Coincidentally, knowing this unlocks a better understanding of the reasoning behind the shape of self-driving offering present on market right now, and the tendency of fully self driven vehicles to move very slowly and stop before anything they perceive in front of them.
    pixl97 · 1790178193 · focus · HN ↗
    > the maker of the self-driving car is liable for the death in this situation.
    Maybe. There will be an investigation looking at things like. Did the user modify the car? Was the car modified by an unauthorized 3rd party? Was the car hacked?
    Right now in AI things are relatively clear because it takes just massive amounts of power and compute to make anything remotely complicated. This barrier will fall as all other barriers in compute have fallen. Either via algorithm or hardware.
    In our lifetime (unless you're rather old) we will see the relatively easy creation of self directing agents by actors with few resources. This breaks the standard concepts of liability where a single human actor rarely has the ability to create massive amounts of damages far beyond their means. The closest thing I can think of is an arsonist causing billions in damages, only in this case the fire has a will of it's own and can hide and spread around dark places on the internet.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.