‹ BackHN Continuity

Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

  1. colingauvin · · focus · HN ↗
    While I think that it's important to do root cause analysis, we need to be very careful not to absolve blame from the people that decided to shift a significant weight of decision-making authority to an AI. An AI can't be tried in a court. There has to be a responsible human for everyone one of its actions, especially when it's killing (-people-) children.

    I cannot believe this has come to pass.

    1. shepherdjerred · · focus · HN ↗
      Taking AI out of the equation, surely these same kinds of mistakes have been made in the recent past. As in, not doing due diligence and mistakenly killing civilians

      (setting aside when civilians are purposefully harmed which is a conversation about morals rather than the capability to make correct decisions)

      What I am getting at is, has AI really made the problem worse or has the mechanism just changed?

      1. aDyslecticCrow · · focus · HN ↗
        We are increasingly seeing "Sorry out <Black box> made a mistake, the developers are fixing it" rather than any one person taking responsibility for the decision. This is a growing problem that is made worse by AI. Over-inflated trust in AI output is also documented issue; "The perfect billion dollar targeting system with a 99.9% success-rate (on its verification dataset)" just made a "fluke"; "Nothing could have stopped this tragedy"

        When it was just called a "database query" or "algorithmic estimation", it was more implied that the actual decision is left to be made and that no analysis had actually been done yet. But when AI preemptively says "I've analyzed every source of data and come to the conclusion that this ship is carrying nuclear bomb material"; it really sounds urgent doesn't it? It probably doubled down when asked about it.

        <a href="https:&#x2F;&#x2F;edition.cnn.com&#x2F;2026&#x2F;09&#x2F;18&#x2F;politics&#x2F;us-military-ai-false-intelligence-china-ship" rel="nofollow">https:&#x2F;&#x2F;edition.cnn.com&#x2F;2026&#x2F;09&#x2F;18&#x2F;politics&#x2F;us-military-ai-f...

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.