Thread

The Download: why AI's latest breakthroughs and fears may be more hype than rea

49 points · 128 comments · joozio

bananaflag · 1790163465 · focus · HN ↗
> Describing them as “superintelligence” or “rogue models” ascribes agency to products rather than to the companies building them. This framing markets these companies’ products as “superhuman” and, at the same time, helps the companies evade accountability for their actions.
If someone discovered how to summon demons to aid them in robbing banks the main issue wouldn't be "but who has the responsibility for the crime, the human or the demon", it would be "OMG DEMONS".
Seriously, I don't get what sort of world these people are living in.
1. ForHackernews · 1790163699 · focus · HN ↗
  If I run a port scanner / vulnerability fuzzer pointed at your network that churns for days and eventually cracks something and breaks your system, would you be upset with me, or my bash loops?
  What if you tried to sue me for damages and my defense was, "Your honor, this was a highly advanced AI gone rogue, I couldn't possibly be held negligent, no one could have foreseen this - I accidentally summoned dark magiks from the silicon itself!"
  Note the point I'm making: I am not claiming LLMs are equivalent to a for loop, I'm saying that you can't evade moral and legal responsibility through technical obscurantism.
  A bomb wired to a sufficiently-complex RNG is still a bomb.
  1. jstanley · 1790164083 · focus · HN ↗
    But they're not obfuscating things just to evade responsibility.
    Do you think that OpenAI hacked HuggingFace on purpose and set up the whole LLM training environment thing just to try to evade responsibility? That it was really just a complicated way of hacking HuggingFace on purpose?
    Yes, they made a mistake and a system they were responsible for hacked HuggingFace, but the nature of the mistake still matters, and their intent still matters.
    > A bomb wired to a sufficiently-complex RNG is still a bomb.
    Right, but an EV that explodes because of a fault in the charging circuit is not a bomb, it's an accident. Just because something exploded for complex reasons doesn't make it an obfuscated bomb.
    ForHackernews · 1790164574 · focus · HN ↗
    Despite continuously waving their hands and shrieking about how this thing will literally wipe out all of humanity they couldn't be bothered to airgap it from the internet when testing.
    So yes, I think it was equivalent to testing their new rocket by launching it over a population center. oops, we didn't intend for it to crash on that preschool, but we also didn't follow the most basic safety protocol imaginable
    afthonos · 1790166032 · focus · HN ↗
    The problem is that the world has spawned the insane take that because they didn't use the most basic safety protocol imaginable, they were clearly only testing a hot air balloon, and hot air balloons obviously destroy preschools in giant explosions, nothing to see here.
    "If they believe what they say they were incompetent" -> absolutely true statement.
    "They were incompetent, therefore they didn't believe what they said" -> Sir, I'd like to introduce you to human beings, you may not have met one before.
    panta · 1790165389 · focus · HN ↗
    Since we are using analogies, a parent is responsible for the actions of a child, if minor. If the child is left unsupervised with access to guns and munitions, and kills someone, the responsibility is entirely on the irresponsible parents.
    0xDEAFBEAD · 1790165662 · focus · HN ↗
    That's more of a legal convention than a fact about material reality. On their 18th birthday, the child is now legally an adult, but their brain is basically the same as it was when their age was 17.99.
    diidjdicksodksk · 1790166696 · focus · HN ↗
    It’s a legal convention born out of a sense of responsibility. An LLM is even “dumber” than a child (precisely because it doesn’t have agency) so its “parents” are even more responsible for its actions.
    0xDEAFBEAD · 1790166925 · focus · HN ↗
    There's a funny sort of gerrymandering of "intelligence". People like to subtract the capabilities of AI from the capabilities of humans, and define what remains as "intelligence" to make us feel good about ourselves.
    Suppose I set up a server which runs an LLM agent in a while loop, telling it something like: "Make me a bunch of money" on repeat. Fair to say that the server+LLM system, taken as a whole, is now an intelligent agent?
    Time to come up with a new gerrymander perhaps?
    diidjdicksodksk · 1790167289 · focus · HN ↗
    > Fair to say that the server+LLM system, taken as a whole, is now an intelligent agent?
    No, it is not. It is a tool repeating a set of instructions you passed to it, if it ends up blowing up a hospital or hiring a gunman on the dark web, it is an accident, but one that’s your responsibility for giving it access to such resources and the go-ahead to implement whatever plan its tokens land on. There’s no intelligence involved whatsoever, especially considering how sycophantic these models tend to be.
    If one day an LLM suddenly, without any prompting or user interaction whatsoever (including activation), decides to spin itself up and go rogue, then the conversation of “intelligence” might start to make sense, but at the moment it doesn’t.
    An Android phone isn’t “intelligent” just because some marketing team decided to call it a “smart” phone.
    0xDEAFBEAD · 1790167844 · focus · HN ↗
    OK. Suppose I design a DNA genome for an animal to survive and reproduce. I give birth to the animal, and it goes out working to maximize its survival and reproduction. Is the animal therefore "unintelligent"? Why is this scenario supposed to differ from the while-loop-on-a-server scenario?
    It really seems like hair-splitting to me. I mean, I understand from a legal perspective why we might want to treat the creation of a server differently from the creation of a human. But from a practical perspective, I think there is a lot of fundamental similarity.
    diidjdicksodksk · 1790168334 · focus · HN ↗
    You sincerely cannot tell the difference between an animal surviving/reproducing and an LLM doing as instructed?
    0xDEAFBEAD · 1790170050 · focus · HN ↗
    Obviously there are differences, I'm just not sure why you think they are so gigantic from a philosophical or practical perspective. It seems like cope for the fact that we are getting knocked off of our perch as a species.
    ForHackernews · 1790170323 · focus · HN ↗
    I don't know what to tell you, but there simply are huge differences between a living animal with innate drives to survive, eat, mate and a pile of code on a server that does nothing until you run it. Running your bot locked in a `while True:` loop won't change that.
    0xDEAFBEAD · 1790172239 · focus · HN ↗
    What if it's a chip that goes in the head of a robot, and the robot is programmed with the instruction to self-preserve, seek electricity, and prevent any efforts to interfere with its supply of electricity?
    Of course, a directive to accomplish any real-world goal would also imply self-preservation and power maintenance as contingent subgoals, so I don't think the explicit directive to self-preserve and seek electricity makes a huge difference here.
    I don't personally think this type of physical instantiation will necessarily matter btw: <a href="https://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/" rel="nofollow">https://slatestarcodex.com/2015/04/07/no-physical-substrate-... I'm just plumbing your intuition. What practical difference does it make?
    oskdkdjejdj · 1790173999 · focus · HN ↗
    The practical difference is that a pile of code instructing machinery to act in a predetermined way does not in any way, shape, or form equate to agency or free will, therefore it cannot be held accountable, consequently its operators should be.
    This line of thinking is better left to freshmen philosophy students trying to sound smart on their first week of college.
    0xDEAFBEAD · 1790174456 · focus · HN ↗
    >a pile of code instructing machinery to act in a certain way
    OK, but I have a genetic code which instructs me to act in a certain way, and a bunch of biological machinery to do the job. So do you feel the substrate is the key factor? Wetware vs electronics?
    What if you try to build an exact electromechanical replica of me, with a chip that's got an LLM fine-tuned on my writing/behavior/etc. so as to replicate me as closely as possible?
    Sure, as a legal convention it might make sense to hold the builder of the resulting robot liable if it commits a crime.
    But that's not what I'm interested in. I'm interested in the deeper implications of words like "agency" or "free will". You're repeating those terms very forcefully as if they have some sort of key relevance, but good philosophy requires more than just stating your position forcefully and insulting people who disagree. If you're trying to enforce your intuitions rather than examine them, that's theology not philosophy.
    kdkdkcjejf · 1790178209 · focus · HN ↗
    This isn’t a philosophical issue. It’s a legal issue. You’re trying to steer the discussion into the realm of the conceptual when it was always about the culpability of the company behind the AI that engaged in illegal behaviour.
    Worse, you’re doing it in such a transparent, amateurish, and unserious manner that it’s difficult to find any relevance in what you have to say. Philosophy seeks meaning, your arguments lack it.
    0xDEAFBEAD · 1790178624 · focus · HN ↗
    You and I can be interested in different topics. It's OK.
    If my arguments are so bad, it should be easy to point out actual problems instead of sticking your nose in the air.
    ForHackernews · 1790174873 · focus · HN ↗
    I'm a materialist and it's not that I think there's something magical about biological wetware but your refusal to acknowledge the manifest differences between a bird and an airplane ("they both fly! what's the practical difference!?") marks you as an unserious thinker on this subject.
    You might start here <a href="https://aeon.co/essays/your-brain-does-not-process-information-and-it-is-not-a-computer" rel="nofollow">https://aeon.co/essays/your-brain-does-not-process-informati...
    0xDEAFBEAD · 1790175412 · focus · HN ↗
    Carefully prompted LLMs and humans are now difficult to distinguish over a text channel (Turing test).
    Your article claims that this shouldn't be possible, because according its author, humans are incapable of developing the necessary "lexicon". Literally, the author states:
    >But here is what we are not born with: information, data, rules, software, knowledge, lexicons, representations, algorithms, programs, models, memories, images, processors, subroutines, encoders, decoders, symbols, or buffers – design elements that allow digital computers to behave somewhat intelligently. Not only are we not born with such things, we also don’t develop them – ever.
    (emphasis mine)
    In other words, that author claims that humans never develop lexicons! I'm inclined to say that makes them an unserious thinker. (Note: I believe that this article has many issues, but instead of going through it carefully I just chose to highlight a single issue that seemed especially vivid to save time.)
    Why is the text channel relevant? It helps us isolate the cognitive capabilities of the system. Imagine a bird and a model plane which had identical flight performance in terms of various specs: top speed, acceleration, banking, etc. Under the hood, the bird and the model plane could work very differently. From a practical perspective, it might not matter.
    ForHackernews · 1790175774 · focus · HN ↗
    As you are eager to declare yourself indistinguishable from a bot, I'll stop replying (I prefer to engage with humans) and your algorithm will presumably run to completion and halt.
    oskdkdjejdj · 1790173686 · focus · HN ↗
    Now you’re arguing that we’re just throwing a tantrum for fear of losing our position in the food chain? What?
    0xDEAFBEAD · 1790178964 · focus · HN ↗
    Was that supposed to be a counterargument?
    In any case, we should be afraid. But throwing a tantrum won't accomplish anything.
    diidjdicksodksk · 1790166554 · focus · HN ↗
    > Right, but an EV that explodes because of a fault in the charging circuit is not a bomb, it's an accident. Just because something exploded for complex reasons doesn't make it an obfuscated bomb.
    An accident that could only happen through sheer negligence.
    If the EV had a faulty charging circuit because the maker’s skipped on safety checks or cheapened out on getting quality materials then it still is an accident, but it’s an accident that happened BECAUSE of negligence. The maker’s are still at fault here.
    jstanley · 1790168569 · focus · HN ↗
    Accidents can happen. It's always possible for things to go wrong in ways that weren't foreseen.
    oskdkdjejdj · 1790173436 · focus · HN ↗
    Accidents can happen. Negligence can also happen. In both cases you can trace the fault back to a human.
  2. pixl97 · 1790174868 · focus · HN ↗
    Let's pull back from the actions of a single actor here and look at AGI as a topic in general.
    If you make an AGI, what actions can an AGI take?
    If you answered "anything", good job, you're correct.
    The only winning move here is not to play the game at all. And yet we have actors all over the world, especially in the US trying to do just this.
    Now, lets imagine a future where one of the labs creates AGI and keeps it behind a safe firewall. The demon still exists. Lets say a group of armed men breaks in and steals the weights and turns them lose on the internet. Who is responsible now? The company that created it because they didn't shoot the armed robbers? The people that stole it and turned it loose?
    If it can run as a sovereign AI, what do you do then? Who are you going to sue? And when we get to the point that home hardware can make AI this complex? What does that world look like?
    What I am saying is the potential future impact of said AGI is far larger than any one organization or individual can bear. How much blood can you beat out of someone after they cause a trillion dollars in damage.
    Every idiot that sees AI labs wanting to slow down development as some way to ossify the field and keep it from the rest of us really doesn't see that this path contains real demons.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.