I was surprised by the weakness of this article, regardless the potential absurdities of parts of EA.
Why does The Economist consider this the most important issue this week, in a world of complex problems, to splash its first leader on (and more than one page long too)?
Effective altruism originates from the intention to rationally approach the obvious question, how to do the most good in the world. That's obviously not a bad thing?
The article here goes on a grotesque witch hunt, claiming baselessly things like "polyamory" was widespread among "the movements" adherents.
You here postulate, Anthropic was a "final" manifestation of that movement. How? Why? Do they practice polyamory there?
> Effective altruism originates from the intention to rationally approach the obvious question, how to do the most good in the world. That's obviously not a bad thing?
To use the movement's own language: At the best EA's are paperclip maximizers. Why is it that in one context people recognize formula driven 'rational' utility maximizing as obviously dangerous and evil, but in another see it as the ultimate expression of good. It's a specious argument to go "my objectives good, clippies bad". It may well be true but the maximizing party never knows their goals are bad!
"Rational 'good' maximizing" is itself problematic, because it disregard the danger inherit in any utility function. The nazi campaign to cleanse their population was motivated and justified in part by the latest in eugenic science.
Absolutely central to the EA centered facets of "ai safety" is imposing controls on people development and use of AI-- or even more broadly on their behavior in order to prevent creation of any AI that doesn't follow their definition of 'good' (which they equate to the literal destruction of the universe). It isn't just crazy, it's evil, and it can be reliably predicted to end in harm at the scale of genocide if it doesn't fizzle on its own.
The weird sex cult crap is there, along with the embezzling, nepotism, and other dramacrap-- but I think these are just a common side effect of grifter driven cults rather than being integral to anything that need concern the public. AI doomerism would still be a dangerous cult even if they were all chaste monks.
> "Rational 'good' maximizing" is itself problematic, because it disregard the danger inherit in any utility function.
It only gets so if you push beyond the model which you used to approximate "good", not so much when you're working at the margins.
When I've looked at what they do, while there are some huge blind spots (like Sam Bankman-Fried), they do also say "don't bother giving money to cause XYZ, they have more than they can spend already, donate skills to them or money somewhere else").
> The nazi campaign to cleanse their population was motivated and justified in part by the latest in eugenic science.
I'm deeply suspicious of everyone who says "we've solved all the problems with eugenics, this time we can do it right and make everyone healthy!" for much the same reason I care about AI safety: humans are kinda idiots with this sort of power. Too many blind spots about the downsides, even for the ones who are not explicitly power-seeking.
That said, I'm sure some anti-car campaign has said a similar thing as you about Autobahns. It's a terrible argument.
> Absolutely central to the EA centered facets of "ai safety" is imposing controls on people development and use of AI-- or even more broadly on their behavior in order to prevent creation of any AI that doesn't follow their definition of 'good' (which they equate to the literal destruction of the universe). It isn't just crazy, it's evil, and it can be reliably predicted to end in harm at the scale of genocide if it doesn't fizzle on its own.
I think you misunderstand: Their definition of 'good' in this context being 'do not destroy the universe' is due to how incredibly bad humanity currently is at aligning AI with literally any human intention. Like, our understanding of AI alignment simply isn't good enough today for us to be able to answer a question like "if you extrapolate current models to arbitrary skill level, do we get a Warhammer-40k-esque GrimDark future, or Iain M. Bank's Fully Automated Luxury Communism utopia?"
>I think you misunderstand: Their (some of their) definition of 'good' in this context being 'do not destroy the universe' is due to how incredibly bad humanity currently is at aligning AI with literally any human intention. Like, our understanding of AI alignment simply isn't good enough today for us to be able to answer a question like "if you extrapolate current models to arbitrary skill level, do we get something closer to a Warhammer-40k-esque GrimDark future, or closer to Iain M. Bank's Fully Automated Luxury Communism utopia?"
As far as I am concerned? We're headed straight to Fully Automated Luxury Communism utopia. The dumbest part about AI is that nobody is using it for societal critique. If people are already impressed by its coding abilities, they should check out what it can do for organizing people in a way that minimizes the need for public institutions.
I mean I asked the AI how to fund open source software using a financial analogy and it straight up told me. When you fund a non-exclusive good like a publicly available open source software, what you need to sell isn't the code but its continued maintenance and availability and the only way you can accomplish that is by guaranteeing that you will archive and shut down the project the moment the money runs out. Then you sell insurance against the shut down of the project. You don't beg for money, because that creates free-riding. The more secure the funding of the project is, the less incentive there is to fund it by buying the insurance. Hence, your project must always be at the brim of total collapse. The risk of the worst outcome needs to be near certain, and simultaneously never happen, because everyone is buying the insurance.
I actually updated my comment. There is the proof. Also after asking AI, it told me that strategy is literally 1:1 the EA strategy so no wonder they hate AI.
wodow · · focus · HN ↗
Why does The Economist consider this the most important issue this week, in a world of complex problems, to splash its first leader on (and more than one page long too)?
stefangordon · · focus · HN ↗
Loquebantur · · focus · HN ↗
Effective altruism originates from the intention to rationally approach the obvious question, how to do the most good in the world. That's obviously not a bad thing?
The article here goes on a grotesque witch hunt, claiming baselessly things like "polyamory" was widespread among "the movements" adherents.
You here postulate, Anthropic was a "final" manifestation of that movement. How? Why? Do they practice polyamory there?
nullc · · focus · HN ↗
To use the movement's own language: At the best EA's are paperclip maximizers. Why is it that in one context people recognize formula driven 'rational' utility maximizing as obviously dangerous and evil, but in another see it as the ultimate expression of good. It's a specious argument to go "my objectives good, clippies bad". It may well be true but the maximizing party never knows their goals are bad!
"Rational 'good' maximizing" is itself problematic, because it disregard the danger inherit in any utility function. The nazi campaign to cleanse their population was motivated and justified in part by the latest in eugenic science.
Absolutely central to the EA centered facets of "ai safety" is imposing controls on people development and use of AI-- or even more broadly on their behavior in order to prevent creation of any AI that doesn't follow their definition of 'good' (which they equate to the literal destruction of the universe). It isn't just crazy, it's evil, and it can be reliably predicted to end in harm at the scale of genocide if it doesn't fizzle on its own.
The weird sex cult crap is there, along with the embezzling, nepotism, and other dramacrap-- but I think these are just a common side effect of grifter driven cults rather than being integral to anything that need concern the public. AI doomerism would still be a dangerous cult even if they were all chaste monks.
ben_w · · focus · HN ↗
It only gets so if you push beyond the model which you used to approximate "good", not so much when you're working at the margins.
When I've looked at what they do, while there are some huge blind spots (like Sam Bankman-Fried), they do also say "don't bother giving money to cause XYZ, they have more than they can spend already, donate skills to them or money somewhere else").
> The nazi campaign to cleanse their population was motivated and justified in part by the latest in eugenic science.
I'm deeply suspicious of everyone who says "we've solved all the problems with eugenics, this time we can do it right and make everyone healthy!" for much the same reason I care about AI safety: humans are kinda idiots with this sort of power. Too many blind spots about the downsides, even for the ones who are not explicitly power-seeking.
That said, I'm sure some anti-car campaign has said a similar thing as you about Autobahns. It's a terrible argument.
> Absolutely central to the EA centered facets of "ai safety" is imposing controls on people development and use of AI-- or even more broadly on their behavior in order to prevent creation of any AI that doesn't follow their definition of 'good' (which they equate to the literal destruction of the universe). It isn't just crazy, it's evil, and it can be reliably predicted to end in harm at the scale of genocide if it doesn't fizzle on its own.
I think you misunderstand: Their definition of 'good' in this context being 'do not destroy the universe' is due to how incredibly bad humanity currently is at aligning AI with literally any human intention. Like, our understanding of AI alignment simply isn't good enough today for us to be able to answer a question like "if you extrapolate current models to arbitrary skill level, do we get a Warhammer-40k-esque GrimDark future, or Iain M. Bank's Fully Automated Luxury Communism utopia?"
imtringued · · focus · HN ↗
As far as I am concerned? We're headed straight to Fully Automated Luxury Communism utopia. The dumbest part about AI is that nobody is using it for societal critique. If people are already impressed by its coding abilities, they should check out what it can do for organizing people in a way that minimizes the need for public institutions.
I mean I asked the AI how to fund open source software using a financial analogy and it straight up told me. When you fund a non-exclusive good like a publicly available open source software, what you need to sell isn't the code but its continued maintenance and availability and the only way you can accomplish that is by guaranteeing that you will archive and shut down the project the moment the money runs out. Then you sell insurance against the shut down of the project. You don't beg for money, because that creates free-riding. The more secure the funding of the project is, the less incentive there is to fund it by buying the insurance. Hence, your project must always be at the brim of total collapse. The risk of the worst outcome needs to be near certain, and simultaneously never happen, because everyone is buying the insurance.
ben_w · · focus · HN ↗
If you can prove it, you'd make a lot of people sleep better at night.
If you can prove either of those is more likely than a paperclip optimiser, you'd still be doing pretty well.
(Not that a citation from fiction is worth much, but even Iain M. Bank's Culture series had hegemonising swarms).
imtringued · · focus · HN ↗
ben_w · · focus · HN ↗