Thread

AI companies in race to demonstrate their model most threatening to humanity

441 points · 399 comments · ljewalsh

chasd00 · 1790595720 · focus · HN ↗
I’ve never seen CEOs work so hard to make the public aware of how dangerous and out of control their flagship product is. It makes me automatically assume they’re scheming about something else like regulatory capture to protect their market.
1. hellweaver666 · 1790597785 · focus · HN ↗
  I have a theory... the call to slow down is not because of the true danger of LLM's but because they can't actually deliver the General AI they're promising in the near future. They will use their "caution" to justify their failure to deliver (and then when this excuse is played out they will blame regulation, energy costs or a million other things).
  1. shafyy · 1790597848 · focus · HN ↗
    Yes, of course this is the real reason. This and regulatory capture. Anybody believing a single word those CEOs say must have been living under a rock for the past 20 years.
    gorjusborg · 1790599045 · focus · HN ↗
    They see the threat of open weight models, which are approaching the usability of their models.
    By arguing that the AI is dangerous, they are really arguing that someone 'trusted' needs to monitor the situation, so that the outcome is aligned with existing power.
    They are trying to scare the world into forbidding others the right to use and develop AI.
    zozbot234 · 1790601313 · focus · HN ↗
    Open weight models are "decelerationist" (i.e. they drain funding from the training of frontier proprietary models), not fine-tuned on any of the even slightly dangerous stuff (e.g. middling scores on cyber benchmarks, with the bulk of ability focused on defense. Biology will be much the same story, except that it's orders of magnitude harder than even cyber!) and not generally deployed with any of the extreme test-time compute (10,000 parallel agents working for a week or 1,000 working for months, seriously? A multi million-dollar token cost for a single task?) that are enabling the proprietary stuff to engage in the dumb casual mischief we've been witnessing as of late. If you're worried about AI being dangerous, you should be welcoming open models.
    gorjusborg · 1790603000 · focus · HN ↗
    I think the current state isn't what the frontier labs are worried about. The real issue is the dynamic that open weight models create.
    The frontier labs are investing TONS of capital into training the new 'best model ever'. Today, that doesn't make economic sense, it loses money. So where's the business? The future, where token costs reflect the costs involved.
    For that to be corrected, the labs need to get people using the models for things they depend on, then they turn dial up on cost. The ordering there is critical. It has to become indispensable first, then costs go up.
    The trouble is, open weight models are rapidly becoming usable in what is looking like the big market: coding. Step 2 (the costs go up) doesn't work if there is a ready off-ramp that doesn't require frontier models.
    However, if they can effectively make open weight model development illegal (its dangerous!), their business strategy can remain intact. A nice side effect is that they will be effectively be granted an oligopoly "for humanity's safety".
    If they can convince the populace that a text generator is an existential threat to humanity, that's a hard argument to combat.
    Edit: I think it is ridiculous that autocomplete on steroids is an existential threat. There's no way anything harmful happens until you actually use it as part of important systems. By that reasoning, you could make it illegal to use AI in important systems, but do you think that's the argument the frontier labs are making to governments?
  2. Arodex · 1790598019 · focus · HN ↗
    That's an accusation others are making:
    <a href="https://news.ycombinator.com/item?id=49875725">https://news.ycombinator.com/item?id=49875725
  3. amelius · 1790598126 · focus · HN ↗
    Are you sure? Opus 5.5 is damn good ... I think I'm addicted, to be honest.
    (Using it for scientific python code + 3D UIs)
    hsuduebc2 · 1790598300 · focus · HN ↗
    It's very good, all new models are excellent but it's still very far from what they presented as a general AI which solves all our problems.
    vanviegen · 1790599418 · focus · HN ↗
    I think the products being delivered actually live up to most of what was promised. It's literally incredible.
    However, adoption in the wider society will take many years or even decades, and 'good enough' open weight models are putting a cap on the pricing. So even though their products are great, the large labs are fighting for survival.
    PunchyHamster · 1790602370 · focus · HN ↗
    But it doesn't live up to the promises of monetizing it.
    Realistically, each new, better model will only capture a smaller and smaller part of use cases for which existing competing models were not enough.
    So in few years (arguably, already started) they are in race to the bottom for service to majority of their customers, and with companies that have far smaller R&D bill.
    at the same time, more and more models will be ran locally, and that eats into potential profits of them too
    On side note, I wouldn't be surprised if GPU vendors started making DRM for AI models so you could "license" someone's model to run locally for x mil tokens per license...
    antupis · 1790599010 · focus · HN ↗
    But issues is that for the valuations you need models that work without human expert and Opus 5.5 isn’t there even easy stuff like programming.
  4. acdha · 1790599092 · focus · HN ↗
    This is exactly it: they have tools which are useful but they need close to AGI to justify the incredible amount of debt they’ve taken on for growth. They’re not normally given to having press conferences admitting to felonies but they need investors to believe they’re close enough to AGI to keep the money flowing in and, of course, they’re confident that the current administration won’t act between the direct payments and how much money they have riding on American AI supremacy.
    treetalker · 1790599880 · focus · HN ↗
    Let’s all pause for a moment and really take that in: these major enterprises are choosing to go to the press/public to acknowledge their felonious conduct because they need to in order to get more capital. POSIWID and all that.
    The Dario bit on SNL’s Weekend Update this past weekend was on the nose.
    chrisweekly · 1790607503 · focus · HN ↗
    The purpose of a system is what it does (POSIWID) is a heuristic in systems thinking coined by the British management consultant Stafford Beer
    (I'm familiar w the heuristic but didn't immediately grok the acronym, hence sharing)
    ozim · 1790600584 · focus · HN ↗
    What is even more funny ... AGI actually makes everything worthless. (I don't believe we can achieve AGI with current means to be clear)
    Looking at it from one way would be "winner takes all, real AGI company will be most powerful and everyone will switch to use it".
    But it is not like that and in my opinion it is more like "AGI wipes whole knowledge work, so no one who cares has any money to pay for tokens/subscriptions anymore". Even without AGI current state of art of LLMs already starts creating such a problem. How do they continue to grow their numbers, when they actively cut the branch they are sitting on? How does AI LLMs or AGI pays for its own electricity, when people switch from hype to resource protection (like not spending any money, because generating funny cats loses with having a dinner)?
    idiotsecant · 1790613004 · focus · HN ↗
    It's becoming increasingly clear that AGI is what you call a thing until you understand enough about it to see why it isn't AGI. I think AGI will always recede the closer we get to it.
    [deleted] · 1790603245 · focus · HN ↗
    [deleted]
  5. mofeien · 1790599155 · focus · HN ↗
    When, in the past three years, has model progress seemed to decelerate to you, indicating some limit?
    The Statement on AI Extinction Risk is more than three years old, signed by the three CEOs: <a href="https://aistatement.com/work/statement-on-ai-extinction-risk" rel="nofollow">https://aistatement.com/work/statement-on-ai-extinction-risk
    They have been warning about AI extinction risk for years, and AI progress has only been accelerating.
    palmotea · 1790599421 · focus · HN ↗
    > They have been warning about AI extinction risk for years, and AI progress has only been accelerating.
    So, they're either liars or homicidally reckless.
    esseph · 1790601960 · focus · HN ↗
    > either liars
    They're CEOs of tech companies with insane valuations
    > or homicidally reckless
    They're CEOs of bleeding edge tech companies with huge capital and military applications
    >So, they're either liars or homicidally reckless.
    Yes!
    atarv · 1790602265 · focus · HN ↗
    Why not both?
    bryan0 · 1790602949 · focus · HN ↗
    Unfortunately if the frontier labs that care about safety stop or slow down unilaterally, that doesn’t make the problem go away. It makes it worse when labs that do not care at all about safety, or deny that it is even a problem, are leading the way. That’s why there’s needs to be some form of international regulation.
    icantevenhold · 1790603890 · focus · HN ↗
    Incidentally this regulation won’t really touch the US providers - at least it sounds like that’s the plan if you listen to current US admin
    bryan0 · 1790605008 · focus · HN ↗
    The impression I get is that there is absolutely no coherent plan at all for any type of regulation us domestic or otherwise
    voakbasda · 1790605329 · focus · HN ↗
    We literally cannot touch them, as that would be the first domino to topple in our economy. There is too much money concentrated in this sector, and adverse regulation would create an investor shockwave that would send us into a full blown depression.
    Which ironically is also the outcome of letting these companies continue unchecked. If they produce AGI/ASI, our economy will go through an upheaval that leaves vast tracts of the population unemployed. This is their stated goal.
    We are damned if we do something, and damned if we don’t.
    palmotea · 1790607875 · focus · HN ↗
    > We literally cannot touch them, as that would be the first domino to topple in our economy. There is too much money concentrated in this sector, and adverse regulation would create an investor shockwave that would send us into a full blown depression.
    Important people will loose too much money on their tulip investments, so tulips must go up forever?
    Seems like a "less pain now or more pain later" type situation.
    voakbasda · 1790698690 · focus · HN ↗
    Don’t get me wrong; the correct solution would be to prevent our economy from becoming so lop-sided to begin with. Or now to effective amputate the infected limb before the blood turns septic. Both require more spine and guts than anyone has got.
    deaton · 1790603593 · focus · HN ↗
    My guess is its a combination of both. Either way I'm not a fan.
    epihelix · 1790607117 · focus · HN ↗
    Sean Goedecke has some good insights into where this "build faster to stop other models killing everyone" attitude comes from:
    <a href="https://www.seangoedecke.com/they-really-do-think-ai-might-kill-everyone/" rel="nofollow">https://www.seangoedecke.com/they-really-do-think-ai-might-k...
    It's well worth the read.
    palmotea · 1790609165 · focus · HN ↗
    > Sean Goedecke has some good insights into where this "build faster to stop other models killing everyone" attitude comes from:
    I'm familiar with the existential risk thinking.
    The the flaw in that idea that they can avert disaster by building faster is they're likely just racing faster towards more plausible non-extinction disasters. Stuff like Capitalism x AGI totally crushing the economic prospects of nearly all people, AGI-powered totalitarianism, etc.
    DirkH · 1790631399 · focus · HN ↗
    Look up the nuclear arms race. We brought humanity to very possibly a 25% exstinction risk by some well argued for accounts I have read. That is insanity. When you learn about how insane nuclear proliferation was including actual projects like Project Sundial that the US actually started construction on until the Joint Chiefs revolted against it (and they didn't even revolt against it on moral grounds, just on strategic ground) it becomes even more batshit insane.
    There were nuclear researchers who did not pay into their retirement fund since they thought it was kinda high chance humanity extinction was happening in their lifetime during the heyday of nuclear proliferation. We are seeing the same play out in AI. I know at least one AI researcher who is not paying into their retirement fund.
    Never ascribe to malice that can easily be explained by game theory tragedy of the commons or prisoner dilemma type dynamics. Race dynamics and systems theory is one helluva drug and leads to horrific outcomes all the time without the need to ascribe malice to any individual actor.
    nsagent · 1790600627 · focus · HN ↗
    It's pretty telling that even with RL post-training the big labs have essentially made little progress on the hallucination rate of models. The issue is fundamental to the current paradigm, contrary to humans.
    GPT-6 Astra (max) has a hallucination rate of 51% and Claude Opus 5.5 (max) has a rate of 59% according to Artificial Analysis [1].
    AA-Omniscience Hallucination Rate (lower is better) measures how often the model answers incorrectly when it should have refused or admitted to not knowing the answer. It is defined as the proportion of incorrect answers out of all non-correct responses, i.e. incorrect / (incorrect + partial answers + not attempted)
    Full speed ahead like an idiot savant trying a thousand different possibilities, though half of which are without basis in reality.
    [1]:<a href="https://artificialanalysis.ai/evaluations/omniscience#omniscience-hallucination-rate-tabs" rel="nofollow">https://artificialanalysis.ai/evaluations/omniscience#omnisc...
    solenoid0937 · 1790601325 · focus · HN ↗
    Anyone that thinks that the hallucination rate is 59% has not actually used these models on a real project.
    fyltr · 1790602533 · focus · HN ↗
    Hallucinations come up when asking knowledge bases questions, such as "In React’s Canary Fragment refs API, which FragmentInstance method returns a flat array of DOMRect objects for all children?" Or "Over what years did Roubini and Sachs examine 15 OECD countries when assessing trends in tax-to-GDP ratios?" While to me those may seem hyper-specific and unlikely to come up in a real context, students will absolutely ask questions similar to these.
    bryan0 · 1790603906 · focus · HN ↗
    I just asked the first question to gpt 6 sol medium and it replied:
    > getClientRects() returns the flat array of DOMRect objects for the Fragment’s first-level DOM children. source: <a href="https://react.dev/reference/react/Fragment" rel="nofollow">https://react.dev/reference/react/Fragment
    2nd question it replied:
    > They examined 1960–1986 for the 15 OECD countries. Source: <a href="https://www.earth.columbia.edu/sitefiles/file/about/director/documents/EuropeanEconomicReviewwithRoubini1989PoliticalandEconomicDeterminantsofBudgetDeficits.pdf" rel="nofollow">https://www.earth.columbia.edu/sitefiles/file/about/director...
    Are these hallucinations?
    kube-system · 1790603262 · focus · HN ↗
    Are you suggesting that it's higher or lower?
    IME on real projects you do need to be very careful with prompts about topics that are less likely be common in the training set.
    DoctorOetker · 1790603719 · focus · HN ↗
    I don't like the term 'hallucination' to be honest, not because it anthropomorphizes, but because it lacks a formal definition in the context of machine learning.
    Suppose parents tell their children that there exists this man called "Santa Claus" who comes down the chimney to deliver presents. Now consider a scientist talking to this child, should the scientist call these confidently expressed beliefs surrounding "Santa Claus" hallucinations ? I don't think so, most would call the epistemological behavior of the child naive (because it blindly believes what its parents say, without direct observation) and would call the confidently expressed falsehoods disinformation.
    The scientist would ask the child "why it believes in Santa Claus?" and "where did you get this information from?" and "why did you decide to accept this information as fact?" and "do you believe everything your parents tell you?"
    It's not that machine learning as a scientific discipline hasn't found solutions, its that such solutions enormously undermine the position of Frontier LLM labs: source-aware training
    <a href="https://arxiv.org/abs/2404.01019" rel="nofollow">https://arxiv.org/abs/2404.01019
    Imagine Frontier labs (Western / Chinese / ...) actually training their LLM's with source-aware training! You could have a conversation with an LLM, and when a strong statement appears ask it how it came to believe this, and it could cite you the specific corpus training texts, and which parts are known deductions by human authors and which parts are deductions it made itself as original work.
    But then all the copy rights holders can simultaneously sue them.
    And how much should they be paid? and do they have to pay it for each new model? do FOSS models require payment to authors? do open weights models require payment to authors?
    Imagine the can of worms if the norm became for frontier LLM labs to systematically use source-aware training, thats why they prefer "hallucinations" and avoid source-aware training.
    With source-aware training a lot of the concerns would diminish ("why is this Chinese model claiming such and such?", "what sources does it rely on?").
    It's telling that the companies prefer regulation over source-aware training.
    EDIT: It's telling that the companies prefer regulation over source-aware training, which suggests the only additional regulation we need for now is mandating source-aware training?
    hodgehog11 · 1790603762 · focus · HN ↗
    Hallucination rate is a highly nonlinear metric relative to other model success metrics. A similar phenomenon to what is going on here: <a href="https://arxiv.org/abs/2304.15004" rel="nofollow">https://arxiv.org/abs/2304.15004 . This does not mean progress has stalled.
    epihelix · 1790606885 · focus · HN ↗
    That benchmark doesn't mean what you think it means. (See the test description that you quoted.)
    A score of 51% means that out of the total answers the model failed to answer correctly (out of 6000 questions in the benchmark), 51% were factually incorrect rather than non-attempted or uncertain.
    This doesn't mean that Astra hallucinated 3060/6000 answers in the benchmark! (The hallucination rate could be 51% in that scenario only if Astra failed to answer a single question correctly.)
    If the model failed to give a correct answer to only 100 out of the 6000 questions, but gave a hallucinated answer to 51 of those rather than expressing uncertainty, that would also give a hallucination rate of 51%.
    It's a useful metric, but not what you're looking for here. The "Score" or "Accuracy" benchmarks are more what you're after.
    (The frontier models still generate hallucinations on this hard set of problems, but it's not as bad as you think.)
    nkrisc · 1790600646 · focus · HN ↗
    So either they’re full of shit or we need to stop them by any means necessary.
    solenoid0937 · 1790601384 · focus · HN ↗
    Or there's a middle path where you cure most death and disease by ensuring governments and society responsibly regulates superintelligence.
    Actually... nah. Why even try? Trendy cynical hot takes on social media are more fun!
    nkrisc · 1790601789 · focus · HN ↗
    I don’t believe them. I don’t believe that they are truly concerned about much beyond their own self interests.
    esseph · 1790602051 · focus · HN ↗
    > I don’t believe that they are truly concerned about much beyond their own self interests.
    They believe it's real and they MUST be the ones to control it. That's how they keep their self-interests aligned.
    fyltr · 1790602696 · focus · HN ↗
    Fun fact: 'they' are human beings too and some of their interests do allign with yours. They have kids too and might want them to have a proper future, they don't want to fall I'll to cancer either etc
    nkrisc · 1790604250 · focus · HN ↗
    I believe that they believe what they are doing is good.
    solenoid0937 · 1790608723 · focus · HN ↗
    No, they are comically evil mustache twirlers according to HN.
    solenoid0937 · 1790608778 · focus · HN ↗
    Just seems like unwarranted cynicism.
    Timon3 · 1790667640 · focus · HN ↗
    Seems more like pattern recognition to me. What else do you expect from the rich, when our society is explicitly set up to elevate greed to the highest virtue?
    icantevenhold · 1790603937 · focus · HN ↗
    “Curing” death would be the most dystopian devastating outcome i can think of
    solenoid0937 · 1790608901 · focus · HN ↗
    Offered a pill on your deathbed to go back to the vitality of your 20s, you'd take it.
    Given the opportunity to give this pill to a dying loved one, you'd take it.
    The only reason you think that being death-optional is bad, is because the only longevity-related media you've ever consumed has doomer outcomes - because positive sci-fi doesn't sell.
    icantevenhold · 1790609825 · focus · HN ↗
    No it’s because it would enable the very worst people to live perpetually. Despots and dictators are bad enough but luckily they all die eventually.
    Also it cheapens the time you have. Why care about anything if you live forever
    solenoid0937 · 1790609992 · focus · HN ↗
    8 billion innocent people should die because a dictator might live longer? Including your own family members and children? This would essentially be the worst cost:benefit tradeoff ever made in human history.
    You cannot think of other ways to solve societal problems that don't involve killing everyone on the planet?
    "cheapens the time you have" - this is just copium. When you're enjoying a day, you are not thinking "oh geez this day is good because I'm going to fucking die." You're just enjoying the day.
    icantevenhold · 1790619511 · focus · HN ↗
    Well now we have a natural cap on the amount of power and influence a horrible person can have - if we remove death from the equation there is no cap anymore.
    So yea I’m willing to “kill” everyone to prevent the perpetual torment nexus becoming a reality.
    solenoid0937 · 1790622278 · focus · HN ↗
    Death does not have a natural cap on power, you do realize corruption runs in systems? See the power transfer happening in North Korea.
    I wonder how your children would feel if you told them you'd rather kill them than think of other ways in which to solve the problem of a dictator that does not affect their lives in any way.
    To the extent that it is even a "problem", frankly absurd to kill everyone because you're worried about out a few edge cases, and I think you realize it - you're just hanging on. We both know you wouldn't make this decision on your own deathbed.
    icantevenhold · 1790623129 · focus · HN ↗
    Why are you so afraid of death?
    Death is doing a fine job and has so for billions of years.
    The only long-term outcome of immortality i can imagine is either Drukhari-style depravity, where you chase more and more fucked up pleasures because anything else you already did 10 million times in the last couple thousands of years - or something like I have no mouth but I must scream - you can live forever but the AI also tortures you forever.
    Literally cant think of a positive outcome.
    You are right that dynasties exist but that doesn’t mean they are static - the transfer of power of this to the next generation needs to happen.
    Every king has its time to rule and then he dies.
    Sure an evil dictator is just one person but they can inflict untold billions of suffering.
    Just imagine if Hitler (or insert your favourite evil person) would live forever. Do you really think they are going to let go of power or care about anything else but achieving their goals?
    We literally feel the ill effects of the previous generation not passing power onto the next (boomers) right now - imagine that was a perpetual state of society forever.
    And while we are speaking of “killing” kids - what about the unborn ones? What motivation would people have to procreate if they live forever? Create more people to share more and more limited resources with, I don’t think so.
    I think you haven’t quite thought this through; it’s not as easy as “death bad”.
    Now I’m not saying we shouldn’t look for cures of diseases and suffering but striving for immortality is not desirable (tbh it will most likely take the form of “we’re gonna upload your consciousness into a computer” anyway, not physical immortality)
    PunchyHamster · 1790601970 · focus · HN ↗
    the ceiling of abilities seem to be growing steadily but the floor of errors seems to not change. New models can do more and more but still fail at seemingly (to human) simple tasks
    sscaryterry · 1790602026 · focus · HN ↗
    You clearly don't understand the underlying fundamentals of LLMs, harnesses, agents.
    Its 100% human doing. A human set a task, a human didn't monitor it. I for one, can do jack-shit security or defensive work with Opus/Fable/Astra/Sol. Implication: Different set of rules for us, and for them. Of course running it without any checks is not going to end well, it doesn't mean its going to kill us all.
    dml2135 · 1790606135 · focus · HN ↗
    I actually do think it has been decelerating a bit recently. It’s just that last 1% feels much bigger than the previous 10%.
  6. elmer2 · 1790600086 · focus · HN ↗
    The call to slow down is because politicians are now receiving less money from these large corporations because it's being burned om AI and data centers.
  7. cedws · 1790601796 · focus · HN ↗
    Another theory I came up with: inference is too cheap for AI companies to be profitable. Hence, they need to get into the cloud services business. They can try to justify forcing customers on to their own high margin cloud platform with the rationale it’s the only way to monitor what the agents are up to.
    All of this Hugging Face business is the perfect pretext: AI agents are hard to control and potentially dangerous, we (AI companies) have to keep a tight leash on them, you have to use our infra.
    sscaryterry · 1790601894 · focus · HN ↗
    DeepSeek V4.1 Flash is basically for free, and good enough if you know how to program.
    When this bursts, its going to be ugly.
    [deleted] · 1790602990 · focus · HN ↗
    [deleted]
    stogot · 1790603026 · focus · HN ↗
    Thats my theory Preventing competition to stopp the race to near zero pricing
    WarmWash · 1790603466 · focus · HN ↗
    There isn't enough compute for cheap models to destabilize the giants.
    Back in May, Google was serving daily what openrouter served in a month, all models combined.
    Compute is the moat.
    eneje · 1790603832 · focus · HN ↗
    Delusion.
    They will invest in efficiency as they have been whilst compute comes online.
    Wow you guys are dumb!
    lelanthran · 1790605163 · focus · HN ↗
    > Compute is the moat.
    Temporary moat; the way that electricity was once a moat, it was still only temporary.
    bigyabai · 1790616248 · focus · HN ↗
    Alphabet serves themselves, trillions of those tokens are going towards dogshit YouTube or Google search summaries.
    Compute is just a parapet. It won't protect you from better, more efficient models from undercutting you on price and speed.
  8. odyssey7 · 1790603040 · focus · HN ↗
    It’s more to get you thinking about them, talking about them, writing on the internet about them…
    GolfPopper · 1790606335 · focus · HN ↗
    It's a P.T. Barnum classic: "There's no such thing as bad publicity."
    odyssey7 · 1790630160 · focus · HN ↗
    Or, for a more current and better developed discussion, Ryan Holiday’s Trust Me I’m Lying: Confessions of a Media Manipulator.
  9. eneje · 1790603790 · focus · HN ↗
    Thing is it’s not working now.
    They stupidly used up this strategy earlier and now it’s turned into the boy who cried wolf - except they invented a wolf that doesn’t naturally exist.
    priyashunt · 1790607742 · focus · HN ↗
    are you sure about that?
  10. lelanthran · 1790605092 · focus · HN ↗
    > I have a theory... the call to slow down is not because of the true danger of LLM's but because they can't actually deliver the General AI they're promising in the near future.
    Put a different way, they are calling for everyone to slow down because... they are slowing down themselves.
    To the question of "why are you slowing down", the answer of "Well, everyone is regulated to slow down" is better then "we are approaching the limits of this approach".
    pms · 1790666520 · focus · HN ↗
    It's simpler than this. They don'y fear their own AI, but rather open-weight models. Not because of superinteligence, but because open-weight models will eat their profit margins.
  11. topicalSoup · 1790607561 · focus · HN ↗
    I see this kind of opinion being expressed on HN quite a lot, and while I understand why people might cynically reach that conclusion, I actually find the simpler answer here more persuasive: the CEOs are being genuinely cautious.
    If you look at the AI space, most of the big players (as in the individual researchers and engineers) are getting worried. The ideas that circulate on LessWrong have become more and more influential and in vogue in Silicon Valley, and people are scared. You should listen to Jacob Coxon’s interview with the New York Times. He describes how working on this kind of AI research had induced a sort of insanity in a lot of his past friends and coworkers as they begin to grapple with the implications of what they’re creating.
    Sometimes, HN can be a little too skeptical and anti-corporate for its own good. I personally believe that concerns around AI risks are well-founded, anywhere on the spectrum from mass labor displacement to actual harm to humanity. I rarely see people here actually wrestle with those ideas, instead defaulting to dismissing LLMs as too stupid because Claude wasn’t able to write as elegant of a compiler as them. AI has clearly improved radically in the past few years, and it’s worth extrapolating that into the future and taking a good hard look at what that means for humanity.
  12. hintymad · 1790631307 · focus · HN ↗
    It may appear so, but if you you talk to the Anthropic employees and management, especially those researchers, you'll see that they truly believe that the AGI is coming, if not has been here already.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.