The framing of how we think about AI has really been cultivated by a quite homogeneous group of people. These people, like all people that belong to a sub-culture, are almost certainly prone to groupthink.
It's extremely important to see that group as such. That it is not thousands of individual perspectives but a chorus of connected/aligned people who have all read the same things and talked to the same people and are rewarded implicitly for thinking a similar way.
Many of them read HN and are offended I put them this way. But it's inescapable that we as humans have this flaw when we're surrounded by a culture.
There's another universe where we do not constantly compare AI to nukes. I bet that world has a lower P(doom).
We all use the same Internet. I would find it strange for anyone who's seriously interested in AI to have never heard of Yudkowsky or Scott Alexander or the Rationalist movement. I'd expect anyone interested in the field to have at least tried reading some of their articles over the years.
Like, if you never heard of them, were you living in a cave or something? It would be like never having heard of Y Combinator or Richard Stallman.
On the face of it, this is no more suspicious than reading the same best-selling books. Being in touch with what's going on doesn't necessarily mean people agree.
> It would be like never having heard of Y Combinator or Richard Stallman.
I work in the tech sector. At one big company I worked for several years, 90% of the times I've mentioned Y Combinator to a colleague, I've had to explain who they are. And not one person I mentioned HN to had heard of it.
Likewise, no one knew levels.fyi. Not many knew Stallman, but why should they?
This isn't strange to me. That it's strange to you is evidence that you do live in a bubble and are likely susceptible to some groupthink. (I'm sure I am too, but for different bubbles from yours.)
I've read LessWrong stuff occasionally over the last decade. It rarely resonated with me. It's not bad, but it's not that great. I do tend to question someone's analytical skills when they go all in on that site. And I will say that even in the first few years of occasionally reading it, the group think was apparent.
A while back I saw LessWrong recommended on HN in a list of similar communities that included lobsters.rs, etc. I thought the design of the site looked really nice so browsed with a open mind (I didn't make the rationalism/LW/Eliezer connection).
At the time, one of the recommended posts that was soft-stickied at the top for 1 week + was:
Which, when I finally ended up clicking, was pretty much exactly what if sounds like from the title:
A long, meandering, manifesto full of selectively chosen charts and research findings with 38 mentions of "IQ", from a guy who has been unsuccessfully pitching investors on genetically modifying a grab bag of (dubiously) intelligence-associated genes in human babies.
Apart from the overall unhingedness, the part I found most striking while reading it was the off-the-chart level of self-confidence that flipping a bunch of genes on/off with CRISPR would have more or less zero risk of complications or unpredictable interactions. In his view, all that was holding back humanity's inevitable next step up the evolutionary ladder were the narrow-minded gatekeepers in the scientific establishment clinging to their so-called "ethics" and regressive "precautionary principle".
Without anything resembling experimentation to test his plan in animal models, yet alone early stage clinical trials for safety, he was willing to roll the dice on some unlucky baby dying in childbirth or living a life of agony from chronic disease based on some cherry picked papers he found on PubMed.
Because of the baffling hubris and my overall low impression of his intellectual honesty/rigor in the write up (clearly coming from a place of seeking to justify his own pre-existing beliefs), I opened the comments section expecting/hoping this was just some local nut who would be torn apart in the comment section.
But lo and behold, the very first comment at the top:
Eliezer Yudkowsky
One of the most important projects in the world. Somebody should fund it.
My extremely uncharitable view of of these kinds of people, who now infest the American AI scene, is that they got cyber bullied by domain experts for saying the most ridiculous crap on repeat, and have therefore decided to spend their life on eliminating domain expertise.
I wouldn't go that far. But as we've seen repeatedly, people who have enough qualms with how these labs operate just leave. And the people who stay behind and rise through the ranks are the ones with either very little conscience or some bizarre world view that rationalizes everything.
Their ethical concerns were "OpenAI is not following Yudkowsky enough". Which is not necessary in line with ethics of the rest of us.
Because their idea of ethics is still "worrying a lot about sci-fi threats 1000 years after while discounting harm done now as less important".
We're talking about people that unironically think eugenics based on IQ is a good thing. I think it's very effective altruism to socially ostracize such individuals.
My view - there are a few outcomes
1. AI is never fully realized, but domain expertise is derated and society stagnates (bad ending for everyone, I think unlikely)
2. AI is never fully realized, but empowers people with domain expertise (Google wins, weird rationalist nerds go back to their LW cave and leave us alone, goodish ending)
3. AI becomes real and kills everyone (bad ending, but at least I don't have to read people claiming that the Harry Potter fanfic isn't niche)
4. AI becomes real and we live in the Culture universe (people like scam Altman get slap droned, most domain experts have the time of their life working with Minds on really interesting problems. Best ending)
The only outcome where we get swirlies is the first one, and we can take solace in the fact that all of humanity is getting swirlied with us.
No, that has the opposite effect: it's very, very common to see LWers write about having a bad time in school in late adolescence and how later finding like minded people online was powerful community bonding.
Eliezer himself first blogged at 17 about how miserable school was for him which led to him dropping out/home schooling after 8th grade and discovering the Extropians listserv. Ziz of the Zizians is another (infamous) example.
The connection is that by making humans smarter, we could sidestep the need for powerful AI or be better equipped at dealing with AI that's smarter than us.
I find the proposal in the post ridiculous too and human gene editing given current technology is a bad idea, but I think human augmentation in general is an interesting path and something that will very likely be taken more seriously in the future.
Thank you, but I didn't actually need that spelled out. I simply accepted that a community with a thought leader who failed to recognize what to me was a self-evident lack of scientific rigor just wasn't for me.
throwaway13337 · · focus · HN ↗
It's extremely important to see that group as such. That it is not thousands of individual perspectives but a chorus of connected/aligned people who have all read the same things and talked to the same people and are rewarded implicitly for thinking a similar way.
Many of them read HN and are offended I put them this way. But it's inescapable that we as humans have this flaw when we're surrounded by a culture.
There's another universe where we do not constantly compare AI to nukes. I bet that world has a lower P(doom).
skybrian · · focus · HN ↗
Like, if you never heard of them, were you living in a cave or something? It would be like never having heard of Y Combinator or Richard Stallman.
On the face of it, this is no more suspicious than reading the same best-selling books. Being in touch with what's going on doesn't necessarily mean people agree.
BeetleB · · focus · HN ↗
I work in the tech sector. At one big company I worked for several years, 90% of the times I've mentioned Y Combinator to a colleague, I've had to explain who they are. And not one person I mentioned HN to had heard of it.
Likewise, no one knew levels.fyi. Not many knew Stallman, but why should they?
This isn't strange to me. That it's strange to you is evidence that you do live in a bubble and are likely susceptible to some groupthink. (I'm sure I am too, but for different bubbles from yours.)
I've read LessWrong stuff occasionally over the last decade. It rarely resonated with me. It's not bad, but it's not that great. I do tend to question someone's analytical skills when they go all in on that site. And I will say that even in the first few years of occasionally reading it, the group think was apparent.
qlte · · focus · HN ↗
At the time, one of the recommended posts that was soft-stickied at the top for 1 week + was:
"How to Make Superbabies"
<a href="https://www.lesswrong.com/posts/DfrSZaf3JC8vJdbZL/how-to-make-superbabies" rel="nofollow">https://www.lesswrong.com/posts/DfrSZaf3JC8vJdbZL/how-to-mak...
Which, when I finally ended up clicking, was pretty much exactly what if sounds like from the title:
A long, meandering, manifesto full of selectively chosen charts and research findings with 38 mentions of "IQ", from a guy who has been unsuccessfully pitching investors on genetically modifying a grab bag of (dubiously) intelligence-associated genes in human babies.
Apart from the overall unhingedness, the part I found most striking while reading it was the off-the-chart level of self-confidence that flipping a bunch of genes on/off with CRISPR would have more or less zero risk of complications or unpredictable interactions. In his view, all that was holding back humanity's inevitable next step up the evolutionary ladder were the narrow-minded gatekeepers in the scientific establishment clinging to their so-called "ethics" and regressive "precautionary principle".
Without anything resembling experimentation to test his plan in animal models, yet alone early stage clinical trials for safety, he was willing to roll the dice on some unlucky baby dying in childbirth or living a life of agony from chronic disease based on some cherry picked papers he found on PubMed.
Because of the baffling hubris and my overall low impression of his intellectual honesty/rigor in the write up (clearly coming from a place of seeking to justify his own pre-existing beliefs), I opened the comments section expecting/hoping this was just some local nut who would be torn apart in the comment section.
But lo and behold, the very first comment at the top:
... and thus ended my brief time trying out LW.Gangway0829 · · focus · HN ↗
pvab3 · · focus · HN ↗
BeetleB · · focus · HN ↗
watwut · · focus · HN ↗
Because their idea of ethics is still "worrying a lot about sci-fi threats 1000 years after while discounting harm done now as less important".
And it matters.
eloisius · · focus · HN ↗
[dead]
ChickeNES · · focus · HN ↗
Gangway0829 · · focus · HN ↗
GPerson · · focus · HN ↗
[dead]
Gangway0829 · · focus · HN ↗
The only outcome where we get swirlies is the first one, and we can take solace in the fact that all of humanity is getting swirlied with us.
qlte · · focus · HN ↗
Eliezer himself first blogged at 17 about how miserable school was for him which led to him dropping out/home schooling after 8th grade and discovering the Extropians listserv. Ziz of the Zizians is another (infamous) example.
dumberquestions · · focus · HN ↗
I find the proposal in the post ridiculous too and human gene editing given current technology is a bad idea, but I think human augmentation in general is an interesting path and something that will very likely be taken more seriously in the future.
qlte · · focus · HN ↗