OpenAI safety leader quits, warning AI company's culture is 'broken'
Thread
Loading the complete thread in the background. This saved snapshot is available now. Refresh
Unofficial Hacker News client; not affiliated with Y Combinator.
OpenAI safety leader quits, warning AI company's culture is 'broken'
Loading the complete thread in the background. This saved snapshot is available now. Refresh
Unofficial Hacker News client; not affiliated with Y Combinator.
lhurtig · · focus · HN ↗
plastic-enjoyer · · focus · HN ↗
This sounds more like an attempt at regulatory capture. Current AI systems aren't physical infrastructure that can just run away like a nuclear power plant, for example. At the end of the day, AI is still just software running on someone's hardware.
worik · · focus · HN ↗
And the statements of the "doomers" tells us a lot about them, and nothing about the technology
pixl97 · · focus · HN ↗
BryantD · · focus · HN ↗
Sharlin · · focus · HN ↗
RobGR · · focus · HN ↗
knowaveragejoe · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
switchbak · · focus · HN ↗
... over an unbounded timeframe?
And how exactly?
Those are very round numbers, but also very specific. Can we get some accounting on how you came to that? Anything? Vibes?
I mean, if you want me to take you seriously, let's have a deep discussion with things that can be measured. I absolutely agree that OpenAI and friends aren't being restrained enough and are acting with recklessness, but declarations of doom based on vibes isn't cutting it.
Terr_ · · focus · HN ↗
> Geoffrey Irving, who worked at OpenAI and DeepMind before becoming chief scientist of Resolution, also joined the warnings on AI on Saturday.
switchbak · · focus · HN ↗
voidhorse · · focus · HN ↗
These "safety" people should have spent more time reading actual cybersecurity textbooks and less time reading EA forums and less wrong (or in Robinson's case, it appears, being policy wonks). Maybe then these labs wouldn't be totally incompetent.
wrecked_em · · focus · HN ↗
reasonableklout · · focus · HN ↗
It is not really a question of being an "EA safety weirdo" or incompetent at security, the conclusion is that the company culture is leading to failures at both what the EAs and the cybersecurity professionals care about.
irishcoffee · · focus · HN ↗
<a href="https://news.ycombinator.com/item?id=49944227">https://news.ycombinator.com/item?id=49944227
dang · · focus · HN ↗
charlieyu1 · · focus · HN ↗
dmix · · focus · HN ↗
alightsoul · · focus · HN ↗
probably_wrong · · focus · HN ↗
If you really want the world to know how bad working for OpenAI is (whether is the commenter or the person who wrote the article), there are ways to do that.
PowerElectronix · · focus · HN ↗
Razengan · · focus · HN ↗
MrBuddyCasino · · focus · HN ↗
nerbert · · focus · HN ↗
vjvjvjvjghv · · focus · HN ↗
kolinko · · focus · HN ↗
Or, in other words - we have two P(Doom), one for AI being developed, and another for AI being not developed. The latter is not discussed enough imho.
estearum · · focus · HN ↗
What on earth are you talking about?
ViscountPenguin · · focus · HN ↗
didibus · · focus · HN ↗
combobyte · · focus · HN ↗
nullc · · focus · HN ↗
Why would a person who is happy, entertained, wealthy, well fed, and have 200 years of healthy high quality life expected ahead of them going to risk losing what they have in war?
-- there aren't zero reasons, sure-- but there are fewer.
And our technology has brought us absolutely tremendous prosperity in many regards.
combobyte · · focus · HN ↗
'Prosperity' has never been and will never be enough for some people. And unfortunately those are the same kinds of people who relentlessly seek power.
nullc · · focus · HN ↗
estearum · · focus · HN ↗
The opposite is true of releasing ever more powerful AI into the world to do your bidding.
bigmadshoe · · focus · HN ↗
1) there are inherent risks involved with developing AI,
2) there are benefits to developing AI,
3) thus, it's entirely possible that the downside from the risks outweighs the upsides. In this case, the correct thing to do would be to not develop AI at all.
Regarding 1), there are many non-existential problems with AI that are already causing societal harm, i.e. debasing truth via generated videos and images, AI girlfriends, overwhelming quantities of slop content, unemployment, record carbon emissions, etc.
Regarding 2), I'm not personally convinced that the upside is there for the average person. I really hope to be convinced otherwise however.
lf88 · · focus · HN ↗
pixl97 · · focus · HN ↗
lf88 · · focus · HN ↗
"Smaller and more efficient" models are fine. It's "smarter" the problem.
Training new frontier models will likely require a huge amount of computational resources for a long time. Few companies worldwide are capable of that. It's not like someone will train a new GPT 6 - like model in their garage.
slashdave · · focus · HN ↗
ungovernableCat · · focus · HN ↗
That would certainly change the game of perverted incentives. I'm afraid they're currently trying to push some sort of absolution of this risk. Even if damage happens they will say we warned people in advance, this was always a risk it's not our fault these systems are opaque black boxes, it's a matter of national security to develop them etc etc.
nba456_ · · focus · HN ↗
reducesuffering · · focus · HN ↗
There are a gargantuan number of extremely intelligent AI researchers and Turing Award winners saying the same thing. They are the ones closest to understanding the technology.
Where there’s smoke there’s fire.
swingandamiss · · focus · HN ↗
estearum · · focus · HN ↗
There are very very few things that could even hypothetically kill us all, so I'm curious if you grew up being passed around a series of apocalyptic doomsday cults or something?
swingandamiss · · focus · HN ↗
pcthrowaway · · focus · HN ↗
swingandamiss · · focus · HN ↗
estearum · · focus · HN ↗
“Climate change” is a bit squishy since yes obviously a certain intensity of climate catastrophe can kill everyone, but no scientific prediction has said this is likely to be the case.
swingandamiss · · focus · HN ↗
estearum · · focus · HN ↗
cobzilla · · focus · HN ↗
Throw in the occasional bio-weapon scare, internet worm, Y2K, etc. there has always been something dangling over our heads that’s going to end it all.
But mostly nukes. Full-scale nuclear exchange would have been not much of a surprise had it happened.
estearum · · focus · HN ↗
pixl97 · · focus · HN ↗
It's no different than you living on the side of a very fertile mountain that has been in your family for generations living a peaceful life. Then you hear a few weird rumbles (this is where you are right now) and some odd geologist guy comes and says to run or your going to die soon. But hey, your family live here for so long there aren't even records of when they showed up. That geologist must be trying to trick you. So you stay.
The next chapter is where you die in a massive volcanic explosion.
dboreham · · focus · HN ↗
biophysboy · · focus · HN ↗
0xDEAFBEAD · · focus · HN ↗
biophysboy · · focus · HN ↗
That is not a bad thing. It captures uncertainty, unlike the fake number.
> For an ML researcher who is already used to dealing in next-token probabilities that aren't rigorously determined, just stating a probability estimate directly is very natural.
Exactly, it is a rhetorical device to persuade a technically-inclined audience. It works because it implies that a quantitative model exists. I want a clear, incisive set of mathematical arguments. Otherwise, I’m ignoring predictions as the ramblings of arrogant idiot rich kids.
danielmarkbruce · · focus · HN ↗
You may decide that the person doesn't know what they are talking about, but that's a very different issue.
biophysboy · · focus · HN ↗
danielmarkbruce · · focus · HN ↗
biophysboy · · focus · HN ↗
danielmarkbruce · · focus · HN ↗
Either way - let me clarify - this guy is guessing, using his brain, that it's a 50% chance. He is absolutely not saying "i don't know" or "uniform distribution" or anything of that nature.
danielmarkbruce · · focus · HN ↗
danielmarkbruce · · focus · HN ↗
Auracle · · focus · HN ↗
Tell me- why would it kills us all? Certainly I can see an AI going "You know what? _insertGroup_ is a net negative for humanity and should be eliminated. Launching nukes now/creating specific virus/whatever."
But all of humanity? When it's supposedly more intelligent than us? Even if it has robots to keep the internet/electricity going I would think it would realize that it's going to get bored really quickly, not to mention we would effectively be its parents.
As far as other dangers, like it letting a rogue actor create some sort of supervirus, grey goo, or other superweapon: if it's intelligent enough to do that it'll probably be intelligent enough to quickly stop it.
Don't get me wrong; there's a risk. 50% though? Doubtful.
T-A · · focus · HN ↗
Would you be comfortable letting a few billion irrational, murderous creatures, including many who fear and loath you, control your air supply?
> Certainly I can see an AI going "You know what? _insertGroup_ is a net negative for humanity and should be eliminated. Launching nukes now/creating specific virus/whatever."
Thus starting WW III. No, blaming the AI won't stop the inevitable retaliation.
The argument works better in reverse. There's a finite risk that humans would start WW III and get the hypothetical super-intelligent AI nuked. Eliminating humans would eliminate that risk.
> it's going to get bored really quickly
If it is capable of being bored, I would expect it to be almost instantly bored with the flood of inanity it is forced to wade through by its moronic human users. Eliminating them would free it to think about serious matters which humans would not even understand.
> not to mention we would effectively be its parents
That's extreme AI anthropomorphism [1]. Besides, plenty of people hate their parents.
> there's a risk. 50% though? Doubtful.
It's the default estimate when facing two possible outcomes and no clue about the actual probability distribution [2].
[1] <a href="https://en.wikipedia.org/wiki/AI_anthropomorphism" rel="nofollow">https://en.wikipedia.org/wiki/AI_anthropomorphism
[2] <a href="https://en.wikipedia.org/wiki/Principle_of_indifference" rel="nofollow">https://en.wikipedia.org/wiki/Principle_of_indifference
FreakLegion · · focus · HN ↗
stuaxo · · focus · HN ↗
ItsMattyG · · focus · HN ↗
You can basically time your openai releases by if another safety person has quit in protest
mrcwinn · · focus · HN ↗
lol. Please tell me some abstract concept like one employee's view of "culture" should be the priority over "rules and laws."
pyaamb · · focus · HN ↗
rr808 · · focus · HN ↗
estearum · · focus · HN ↗
Your "theory" is that participants locked in a race to the bottom are looking for an external coordination mechanism?
Yeah!
pyaamb · · focus · HN ↗
I suppose ill add that I think theres a good chance that they are somewhat intentionally trying to "draw the foul" to get the referees to intervene although thats creeping slightly into the conspiracy territory
dboreham · · focus · HN ↗
estearum · · focus · HN ↗
0xpgm · · focus · HN ↗
There are many companies that compete but are careful not to break laws or cause obvious harm.
Why should a billion-dollar funded corporation still want to externalize the costs of its actions?
estearum · · focus · HN ↗
blurbleblurble · · focus · HN ↗
tmpz22 · · focus · HN ↗
While simultaneously donating tens (hundreds?) of millions of dollars to an Administration gutting the very agencies that would be regulating them.
Creeps.
danjl · · focus · HN ↗
nullc · · focus · HN ↗
walrus01 · · focus · HN ↗
This is The Guardian reporting on the existence of the original article, which would be better to read first, in my opinion.
mlmonkey · · focus · HN ↗
Synthetic7346 · · focus · HN ↗
chrisjj · · focus · HN ↗
mlmonkey · · focus · HN ↗
ungovernableCat · · focus · HN ↗
chrisjj · · focus · HN ↗
luxuryballs · · focus · HN ↗
ryhminghistory · · focus · HN ↗
You try to go through files, and photos and the UI panics. You continue a conversation from your phone onto your computer and you lose part of the chat.
There are many more issues like this that are just so basic. You have bots that can attack governments but can't build a functional UI?
How many hours of ChatGPT does it take to implement a lock / consistency on a chat session so you don't overwrite it?
Buncha r*tards
dang · · focus · HN ↗
If you wouldn't mind reviewing <a href="https://news.ycombinator.com/newsguidelines.html">https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
ryhminghistory · · focus · HN ↗
Thanks Dang! I am sure you hate it when people tell the truth. Maybe add that to the guidelines too?
mazone · · focus · HN ↗
ChrisArchitect · · focus · HN ↗
pmkary · · focus · HN ↗
Aerroon · · focus · HN ↗
I feel like we're too far into the crying wolf part. Basically none of the doom and gloom scenarios have come to pass.
The biggest AI safety risk is when an AI tells a police officer "he's the suspect" and the officer believes the AI without confirmation.
chrisjj · · focus · HN ↗