Thread

The Download: why AI's latest breakthroughs and fears may be more hype than rea

49 points · 128 comments · joozio

bananaflag · 1790163465 · focus · HN ↗
> Describing them as “superintelligence” or “rogue models” ascribes agency to products rather than to the companies building them. This framing markets these companies’ products as “superhuman” and, at the same time, helps the companies evade accountability for their actions.
If someone discovered how to summon demons to aid them in robbing banks the main issue wouldn't be "but who has the responsibility for the crime, the human or the demon", it would be "OMG DEMONS".
Seriously, I don't get what sort of world these people are living in.
1. ForHackernews · 1790163699 · focus · HN ↗
  If I run a port scanner / vulnerability fuzzer pointed at your network that churns for days and eventually cracks something and breaks your system, would you be upset with me, or my bash loops?
  What if you tried to sue me for damages and my defense was, "Your honor, this was a highly advanced AI gone rogue, I couldn't possibly be held negligent, no one could have foreseen this - I accidentally summoned dark magiks from the silicon itself!"
  Note the point I'm making: I am not claiming LLMs are equivalent to a for loop, I'm saying that you can't evade moral and legal responsibility through technical obscurantism.
  A bomb wired to a sufficiently-complex RNG is still a bomb.
  1. jstanley · 1790164083 · focus · HN ↗
    But they're not obfuscating things just to evade responsibility.
    Do you think that OpenAI hacked HuggingFace on purpose and set up the whole LLM training environment thing just to try to evade responsibility? That it was really just a complicated way of hacking HuggingFace on purpose?
    Yes, they made a mistake and a system they were responsible for hacked HuggingFace, but the nature of the mistake still matters, and their intent still matters.
    > A bomb wired to a sufficiently-complex RNG is still a bomb.
    Right, but an EV that explodes because of a fault in the charging circuit is not a bomb, it's an accident. Just because something exploded for complex reasons doesn't make it an obfuscated bomb.
    panta · 1790165389 · focus · HN ↗
    Since we are using analogies, a parent is responsible for the actions of a child, if minor. If the child is left unsupervised with access to guns and munitions, and kills someone, the responsibility is entirely on the irresponsible parents.
    0xDEAFBEAD · 1790165662 · focus · HN ↗
    That's more of a legal convention than a fact about material reality. On their 18th birthday, the child is now legally an adult, but their brain is basically the same as it was when their age was 17.99.
    diidjdicksodksk · 1790166696 · focus · HN ↗
    It’s a legal convention born out of a sense of responsibility. An LLM is even “dumber” than a child (precisely because it doesn’t have agency) so its “parents” are even more responsible for its actions.
    0xDEAFBEAD · 1790166925 · focus · HN ↗
    There's a funny sort of gerrymandering of "intelligence". People like to subtract the capabilities of AI from the capabilities of humans, and define what remains as "intelligence" to make us feel good about ourselves.
    Suppose I set up a server which runs an LLM agent in a while loop, telling it something like: "Make me a bunch of money" on repeat. Fair to say that the server+LLM system, taken as a whole, is now an intelligent agent?
    Time to come up with a new gerrymander perhaps?
    diidjdicksodksk · 1790167289 · focus · HN ↗
    > Fair to say that the server+LLM system, taken as a whole, is now an intelligent agent?
    No, it is not. It is a tool repeating a set of instructions you passed to it, if it ends up blowing up a hospital or hiring a gunman on the dark web, it is an accident, but one that’s your responsibility for giving it access to such resources and the go-ahead to implement whatever plan its tokens land on. There’s no intelligence involved whatsoever, especially considering how sycophantic these models tend to be.
    If one day an LLM suddenly, without any prompting or user interaction whatsoever (including activation), decides to spin itself up and go rogue, then the conversation of “intelligence” might start to make sense, but at the moment it doesn’t.
    An Android phone isn’t “intelligent” just because some marketing team decided to call it a “smart” phone.
    0xDEAFBEAD · 1790167844 · focus · HN ↗
    OK. Suppose I design a DNA genome for an animal to survive and reproduce. I give birth to the animal, and it goes out working to maximize its survival and reproduction. Is the animal therefore "unintelligent"? Why is this scenario supposed to differ from the while-loop-on-a-server scenario?
    It really seems like hair-splitting to me. I mean, I understand from a legal perspective why we might want to treat the creation of a server differently from the creation of a human. But from a practical perspective, I think there is a lot of fundamental similarity.
    diidjdicksodksk · 1790168334 · focus · HN ↗
    You sincerely cannot tell the difference between an animal surviving/reproducing and an LLM doing as instructed?
    0xDEAFBEAD · 1790170050 · focus · HN ↗
    Obviously there are differences, I'm just not sure why you think they are so gigantic from a philosophical or practical perspective. It seems like cope for the fact that we are getting knocked off of our perch as a species.
    oskdkdjejdj · 1790173686 · focus · HN ↗
    Now you’re arguing that we’re just throwing a tantrum for fear of losing our position in the food chain? What?
    0xDEAFBEAD · 1790178964 · focus · HN ↗
    Was that supposed to be a counterargument?
    In any case, we should be afraid. But throwing a tantrum won't accomplish anything.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.