Lots of anti-AI takes seem to fall prey to the moralistic fallacy (ie because something shouldn’t be, thinking it isn’t). I’m very skeptical of the claims AI companies make, their motives, and whether their impact will be anything positive. But I am pretty tired of takes like this that are so reflexively skeptical of anything AI that they carelessly dismiss claims about the dangers and real impacts of AI that are happening.
(And to be clear: I’m not talking about being skeptical of fantastic claims about things that have not happened- ie an AI takeover, runaway super-intelligence, etc.)
What kind of real evidence about the dangers of AI that are happening do you have that didn't come from an AI company, wasn't about some internal test that conveniently can't be replicated or independently verified, where nobody disabled guardrails that would be in place for typical users, or where they didn't instruct the AI to roleplay as someone doing something evil? What danger are we seeing right now in the real world that people aren't taking seriously?
That's not what happened, in any of the many incidents. But even if it were, if you can make AI do dangerous things by telling it to roleplay doing dangerous things, then that... Makes the AI dangerous.
> That's not what happened, in any of the many incidents.
That's exactly what happened when Google's AI "autonomously" hacked into companies coincidentally in the middle of tests of its cybersecurity abilities in a simulated highly controlled scenario on a totally isolated system that was just accidentally given unrestricted access to the open internet.
That's exactly what happened when AI was "threatening to blackmail" people. Internal tests, in roleplaying scenarios designed to push the model to do exactly wthat it did.
It's exactly what happened in the the Hugging Face incident. An internal test was conducted, AI was told to do tasks specifically to test AI's ability to take known vulnerabilities and turn them into working exploits, mostly using an internal research model which was not intended for production, but also another production model whose guardrails had been intentionally removed.
So yes, a company with billions and billions of dollars and years of research and investment into AI models likely will be able to use their AI in cyber attacks. I think pretty much everybody believed that was a risk.
What does that tell us about our risks in the real world right now? Well, it's extremely hard to say because we're still left trusting the AI company who is constantly lying about what their AI is to tell us what it was doing and how, and we're still unable to independently verify or replicate what happened, and there are no known examples of anyone who isn't an AI company doing anything like it. Certainty you or I or the vast majority of the population can't just browse to chatgpt.com and hack the planet.
Did AI do something that humans couldn't already do? No. Did they spontaneously "go rogue", or demonstrate "superintelligence" or "create AI civilizations"? No. Can we be sure they didn't just orchestrate the whole incident to push their "AI scary!" narrative? Nope. So we're pretty much left with what we already knew. All the largest risks are from AI companies themselves and governments building AI models. There isn't much any of us can do about that.
So, your claim is that OpenAI intentionally might have had its agents hack multiple companies, as well as multiple government agencies? And they're just pretending to not have known about it?
quicklywilliam · · focus · HN ↗
(And to be clear: I’m not talking about being skeptical of fantastic claims about things that have not happened- ie an AI takeover, runaway super-intelligence, etc.)
autoexec · · focus · HN ↗
scarmig · · focus · HN ↗
autoexec · · focus · HN ↗
That's exactly what happened when Google's AI "autonomously" hacked into companies coincidentally in the middle of tests of its cybersecurity abilities in a simulated highly controlled scenario on a totally isolated system that was just accidentally given unrestricted access to the open internet.
That's exactly what happened when AI was "threatening to blackmail" people. Internal tests, in roleplaying scenarios designed to push the model to do exactly wthat it did.
It's exactly what happened in the the Hugging Face incident. An internal test was conducted, AI was told to do tasks specifically to test AI's ability to take known vulnerabilities and turn them into working exploits, mostly using an internal research model which was not intended for production, but also another production model whose guardrails had been intentionally removed.
So yes, a company with billions and billions of dollars and years of research and investment into AI models likely will be able to use their AI in cyber attacks. I think pretty much everybody believed that was a risk.
What does that tell us about our risks in the real world right now? Well, it's extremely hard to say because we're still left trusting the AI company who is constantly lying about what their AI is to tell us what it was doing and how, and we're still unable to independently verify or replicate what happened, and there are no known examples of anyone who isn't an AI company doing anything like it. Certainty you or I or the vast majority of the population can't just browse to chatgpt.com and hack the planet.
Did AI do something that humans couldn't already do? No. Did they spontaneously "go rogue", or demonstrate "superintelligence" or "create AI civilizations"? No. Can we be sure they didn't just orchestrate the whole incident to push their "AI scary!" narrative? Nope. So we're pretty much left with what we already knew. All the largest risks are from AI companies themselves and governments building AI models. There isn't much any of us can do about that.
scarmig · · focus · HN ↗
Okay.