I was surprised by the weakness of this article, regardless the potential absurdities of parts of EA.
Why does The Economist consider this the most important issue this week, in a world of complex problems, to splash its first leader on (and more than one page long too)?
Effective altruism originates from the intention to rationally approach the obvious question, how to do the most good in the world. That's obviously not a bad thing?
The article here goes on a grotesque witch hunt, claiming baselessly things like "polyamory" was widespread among "the movements" adherents.
You here postulate, Anthropic was a "final" manifestation of that movement. How? Why? Do they practice polyamory there?
> Effective altruism originates from the intention to rationally approach the obvious question, how to do the most good in the world. That's obviously not a bad thing?
To use the movement's own language: At the best EA's are paperclip maximizers. Why is it that in one context people recognize formula driven 'rational' utility maximizing as obviously dangerous and evil, but in another see it as the ultimate expression of good. It's a specious argument to go "my objectives good, clippies bad". It may well be true but the maximizing party never knows their goals are bad!
"Rational 'good' maximizing" is itself problematic, because it disregard the danger inherit in any utility function. The nazi campaign to cleanse their population was motivated and justified in part by the latest in eugenic science.
Absolutely central to the EA centered facets of "ai safety" is imposing controls on people development and use of AI-- or even more broadly on their behavior in order to prevent creation of any AI that doesn't follow their definition of 'good' (which they equate to the literal destruction of the universe). It isn't just crazy, it's evil, and it can be reliably predicted to end in harm at the scale of genocide if it doesn't fizzle on its own.
The weird sex cult crap is there, along with the embezzling, nepotism, and other dramacrap-- but I think these are just a common side effect of grifter driven cults rather than being integral to anything that need concern the public. AI doomerism would still be a dangerous cult even if they were all chaste monks.
> "Rational 'good' maximizing" is itself problematic, because it disregard the danger inherit in any utility function.
It only gets so if you push beyond the model which you used to approximate "good", not so much when you're working at the margins.
When I've looked at what they do, while there are some huge blind spots (like Sam Bankman-Fried), they do also say "don't bother giving money to cause XYZ, they have more than they can spend already, donate skills to them or money somewhere else").
> The nazi campaign to cleanse their population was motivated and justified in part by the latest in eugenic science.
I'm deeply suspicious of everyone who says "we've solved all the problems with eugenics, this time we can do it right and make everyone healthy!" for much the same reason I care about AI safety: humans are kinda idiots with this sort of power. Too many blind spots about the downsides, even for the ones who are not explicitly power-seeking.
That said, I'm sure some anti-car campaign has said a similar thing as you about Autobahns. It's a terrible argument.
> Absolutely central to the EA centered facets of "ai safety" is imposing controls on people development and use of AI-- or even more broadly on their behavior in order to prevent creation of any AI that doesn't follow their definition of 'good' (which they equate to the literal destruction of the universe). It isn't just crazy, it's evil, and it can be reliably predicted to end in harm at the scale of genocide if it doesn't fizzle on its own.
I think you misunderstand: Their definition of 'good' in this context being 'do not destroy the universe' is due to how incredibly bad humanity currently is at aligning AI with literally any human intention. Like, our understanding of AI alignment simply isn't good enough today for us to be able to answer a question like "if you extrapolate current models to arbitrary skill level, do we get a Warhammer-40k-esque GrimDark future, or Iain M. Bank's Fully Automated Luxury Communism utopia?"
wodow · · focus · HN ↗
Why does The Economist consider this the most important issue this week, in a world of complex problems, to splash its first leader on (and more than one page long too)?
stefangordon · · focus · HN ↗
Loquebantur · · focus · HN ↗
Effective altruism originates from the intention to rationally approach the obvious question, how to do the most good in the world. That's obviously not a bad thing?
The article here goes on a grotesque witch hunt, claiming baselessly things like "polyamory" was widespread among "the movements" adherents.
You here postulate, Anthropic was a "final" manifestation of that movement. How? Why? Do they practice polyamory there?
nullc · · focus · HN ↗
To use the movement's own language: At the best EA's are paperclip maximizers. Why is it that in one context people recognize formula driven 'rational' utility maximizing as obviously dangerous and evil, but in another see it as the ultimate expression of good. It's a specious argument to go "my objectives good, clippies bad". It may well be true but the maximizing party never knows their goals are bad!
"Rational 'good' maximizing" is itself problematic, because it disregard the danger inherit in any utility function. The nazi campaign to cleanse their population was motivated and justified in part by the latest in eugenic science.
Absolutely central to the EA centered facets of "ai safety" is imposing controls on people development and use of AI-- or even more broadly on their behavior in order to prevent creation of any AI that doesn't follow their definition of 'good' (which they equate to the literal destruction of the universe). It isn't just crazy, it's evil, and it can be reliably predicted to end in harm at the scale of genocide if it doesn't fizzle on its own.
The weird sex cult crap is there, along with the embezzling, nepotism, and other dramacrap-- but I think these are just a common side effect of grifter driven cults rather than being integral to anything that need concern the public. AI doomerism would still be a dangerous cult even if they were all chaste monks.
ben_w · · focus · HN ↗
It only gets so if you push beyond the model which you used to approximate "good", not so much when you're working at the margins.
When I've looked at what they do, while there are some huge blind spots (like Sam Bankman-Fried), they do also say "don't bother giving money to cause XYZ, they have more than they can spend already, donate skills to them or money somewhere else").
> The nazi campaign to cleanse their population was motivated and justified in part by the latest in eugenic science.
I'm deeply suspicious of everyone who says "we've solved all the problems with eugenics, this time we can do it right and make everyone healthy!" for much the same reason I care about AI safety: humans are kinda idiots with this sort of power. Too many blind spots about the downsides, even for the ones who are not explicitly power-seeking.
That said, I'm sure some anti-car campaign has said a similar thing as you about Autobahns. It's a terrible argument.
> Absolutely central to the EA centered facets of "ai safety" is imposing controls on people development and use of AI-- or even more broadly on their behavior in order to prevent creation of any AI that doesn't follow their definition of 'good' (which they equate to the literal destruction of the universe). It isn't just crazy, it's evil, and it can be reliably predicted to end in harm at the scale of genocide if it doesn't fizzle on its own.
I think you misunderstand: Their definition of 'good' in this context being 'do not destroy the universe' is due to how incredibly bad humanity currently is at aligning AI with literally any human intention. Like, our understanding of AI alignment simply isn't good enough today for us to be able to answer a question like "if you extrapolate current models to arbitrary skill level, do we get a Warhammer-40k-esque GrimDark future, or Iain M. Bank's Fully Automated Luxury Communism utopia?"