Thread

AI companies in race to demonstrate their model most threatening to humanity

441 points · 399 comments · ljewalsh

chasd00 · 1790595720 · focus · HN ↗
I’ve never seen CEOs work so hard to make the public aware of how dangerous and out of control their flagship product is. It makes me automatically assume they’re scheming about something else like regulatory capture to protect their market.
1. meowface · 1790595788 · focus · HN ↗
  Or perhaps they've been saying this for literally decades because they earnestly believe it.
  1. ulfw · 1790595960 · focus · HN ↗
    Which CEO said this for 'literally decades'? And if so why are they still pursuing it then?
    Most children have learned touching a hot stove is bad for them. Then why still do it at the age of 40?
    frabcus · 1790596885 · focus · HN ↗
    Sam Altman in 2015 - "Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity" <a href="https://blog.samaltman.com/machine-intelligence-part-1" rel="nofollow">https://blog.samaltman.com/machine-intelligence-part-1
    Elon Musk in 2014 - "We need to be super careful with AI. Potentially more dangerous than nukes" <a href="https://x.com/elonmusk/status/495759307346952192" rel="nofollow">https://x.com/elonmusk/status/495759307346952192
    Dario Amodei in 2017 - "There’s a long tail of things of varying degrees of badness that could happen. I think at the extreme end is the Nick Bostrom" <a href="https://80000hours.org/podcast/episodes/the-world-needs-ai-researchers-heres-how-to-become-one/" rel="nofollow">https://80000hours.org/podcast/episodes/the-world-needs-ai-r...
    They're still pursuing it because without an international treaty, they think it is inevitable. In the case of Dario, they want to align it just enough and be first to stop others causing harm (a vainglorious strategy, yet still a strategy). In the case of Musk and Altman, not completely clear to me.
    Google feels less important right now probably, and Demis Hassabis no longer runs Deep Mind as CEO. However, he talked about this early too, and his reason for proceeding was he was only doing narrow AI (e.g. protein folding) which is fundamentally less dangerous.
    KellyCriterion · 1790597257 · focus · HN ↗
    Wondering how many details & knowhow they had already back then when saying these quotes:
    - For Altman Id say he was already deeper in the "transformer & model landscape" in 2015?
    - Musk in 2014: Not sure how much he was aware of the current-state back then
    frabcus · 1790597569 · focus · HN ↗
    Musk was told by Demis Hassabis in 2012, see <a href="https://time.com/6310076/elon-musk-ai-walter-isaacson-biography/" rel="nofollow">https://time.com/6310076/elon-musk-ai-walter-isaacson-biogra...
    Altman read Nick Bostrom's book Superintelligence, he mentions it in this post <a href="https://blog.samaltman.com/machine-intelligence-part-1" rel="nofollow">https://blog.samaltman.com/machine-intelligence-part-1
    I also read that book in 2014, and it is how I came to understand this problem early.
    meowface · 1790607346 · focus · HN ↗
    2012 - 2015 seems to have been around the time a lot of people first started taking ASI and ASI existential risk very seriously as things that will be relevant in our lifetime, and when they decided to try to build and/or prevent it.
    thfuran · 1790598115 · focus · HN ↗
    >In the case of Musk and Altman, not completely clear to me.
    They probably just want to ensure that the first AGI is a nazi.
    meowface · 1790605899 · focus · HN ↗
    While Musk is now far-right (albeit seemingly not a Hitlerist - yet), Altman is pretty much a liberal.
    ks2048 · 1790610000 · focus · HN ↗
    > Demis Hassabis ... he was only doing narrow AI (e.g. protein folding) which is fundamentally less dangerous.
    I don't think this is true. He repeatedly says things like "I’ve been working towards AGI my whole life".
    meowface · 1790613279 · focus · HN ↗
    Correct. He - I do not think unjustly - has long had AGI as the goal. "Solve AGI, then use AGI to solve medicine, and physics, and the rest of science, and create a better world." has been his thing basically forever.
    frabcus · 1790629898 · focus · HN ↗
    I'm thinking of when he warned Musk in 2012:
    "Hassabis told him to add another potential threat to the list: artificial intelligence. Machines could become superintelligent and surpass us mere mortals, -perhaps even decide to dispose of us."
    <a href="https://time.com/6310076/elon-musk-ai-walter-isaacson-biography/" rel="nofollow">https://time.com/6310076/elon-musk-ai-walter-isaacson-biogra...
    You're right that the picture is more complicated than I wrote above. Quickly looking, it seems that Shane Legg was concerned very very early on. I can't find anything tangible about what their strategy was to avoid the danger. It looks like Hassabis had longer timescales than Legg, so perhaps just thought they had more room before it really mattered?
    frabcus · 1790597033 · focus · HN ↗
    BTW, just to be clear it is insane that they're all still pursuing it, so completely reasonable to be sceptical and come up with other reasons. This is, however, a crazy situation. Lots of other people have refused to develop it - natural selection has picked those with a galaxy brained or vain strategy involving continuing to develop it to be running the companies we have.
    meowface · 1790612665 · focus · HN ↗
    Their mindset is: the upside is likely so, so, so immense, and the probability of amazing outcomes is so much higher than the probability of catastrophic outcomes, that the gamble is worth it, and it is not in fact a pure gamble because they can (in theory) try really hard to build it really slowly and carefully.
    So even with their earnest belief that there is a > 5% chance this technology could be 1) capable of autonomously killing billions of people during this century and 2) may not be sufficiently aligned and might actually act on that capability, the EV basically still sums positive, to them.
    Not saying this is necessarily a good or moral calculus. But it is what they think. There is no conspiracy or plot or deception.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.