Thread

AI companies in race to demonstrate their model most threatening to humanity

441 points · 399 comments · ljewalsh

chasd00 · 1790595720 · focus · HN ↗
I’ve never seen CEOs work so hard to make the public aware of how dangerous and out of control their flagship product is. It makes me automatically assume they’re scheming about something else like regulatory capture to protect their market.
1. hellweaver666 · 1790597785 · focus · HN ↗
  I have a theory... the call to slow down is not because of the true danger of LLM's but because they can't actually deliver the General AI they're promising in the near future. They will use their "caution" to justify their failure to deliver (and then when this excuse is played out they will blame regulation, energy costs or a million other things).
  1. mofeien · 1790599155 · focus · HN ↗
    When, in the past three years, has model progress seemed to decelerate to you, indicating some limit?
    The Statement on AI Extinction Risk is more than three years old, signed by the three CEOs: <a href="https://aistatement.com/work/statement-on-ai-extinction-risk" rel="nofollow">https://aistatement.com/work/statement-on-ai-extinction-risk
    They have been warning about AI extinction risk for years, and AI progress has only been accelerating.
    nsagent · 1790600627 · focus · HN ↗
    It's pretty telling that even with RL post-training the big labs have essentially made little progress on the hallucination rate of models. The issue is fundamental to the current paradigm, contrary to humans.
    GPT-6 Astra (max) has a hallucination rate of 51% and Claude Opus 5.5 (max) has a rate of 59% according to Artificial Analysis [1].
    AA-Omniscience Hallucination Rate (lower is better) measures how often the model answers incorrectly when it should have refused or admitted to not knowing the answer. It is defined as the proportion of incorrect answers out of all non-correct responses, i.e. incorrect / (incorrect + partial answers + not attempted)
    Full speed ahead like an idiot savant trying a thousand different possibilities, though half of which are without basis in reality.
    [1]:<a href="https://artificialanalysis.ai/evaluations/omniscience#omniscience-hallucination-rate-tabs" rel="nofollow">https://artificialanalysis.ai/evaluations/omniscience#omnisc...
    solenoid0937 · 1790601325 · focus · HN ↗
    Anyone that thinks that the hallucination rate is 59% has not actually used these models on a real project.
    fyltr · 1790602533 · focus · HN ↗
    Hallucinations come up when asking knowledge bases questions, such as "In React’s Canary Fragment refs API, which FragmentInstance method returns a flat array of DOMRect objects for all children?" Or "Over what years did Roubini and Sachs examine 15 OECD countries when assessing trends in tax-to-GDP ratios?" While to me those may seem hyper-specific and unlikely to come up in a real context, students will absolutely ask questions similar to these.
    bryan0 · 1790603906 · focus · HN ↗
    I just asked the first question to gpt 6 sol medium and it replied:
    > getClientRects() returns the flat array of DOMRect objects for the Fragment’s first-level DOM children. source: <a href="https://react.dev/reference/react/Fragment" rel="nofollow">https://react.dev/reference/react/Fragment
    2nd question it replied:
    > They examined 1960–1986 for the 15 OECD countries. Source: <a href="https://www.earth.columbia.edu/sitefiles/file/about/director/documents/EuropeanEconomicReviewwithRoubini1989PoliticalandEconomicDeterminantsofBudgetDeficits.pdf" rel="nofollow">https://www.earth.columbia.edu/sitefiles/file/about/director...
    Are these hallucinations?

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.