‹ BackHN Continuity

Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

  1. legitster · · focus · HN ↗
    > It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.”

    Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.

    The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.

    The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.

    1. xlayn · · focus · HN ↗
      Now, ask yourself if you want AI taking decisions around your life... it will pull numbers... someone will have a "goal" of 1000 "hits" and there you go, 2-3 months in jail working to check what went wrong...

      But don't worry, the other P AI system will ensure that the first one doesn't do mistakes...

      1. TheOtherHobbes · · focus · HN ↗
        You seem to have missed the point, which is that the decision to murder a large number of innocent people was made by humans at the top of the chain of command, and 'AI' is being used as a thin pretext for that.

        The pattern matches various other acts of senseless violence against innocents in South America.

        The message isn't "We did this accidentally. We should understand how it happened, so we can make sure it doesn't happen again.".

        It's "We do this to make the point that we can, no one can stop us, and if we're challenged we're going to lie about it."

        1. geraneum · · focus · HN ↗
          The decision was indeed made by AI and humans failed in their duty to fucking verify it. They green lit and now we have school children dead. No single component of this whole chain is defensible.
          1. lifeformed · · focus · HN ↗
            Depends on what "decision" means. If I say, "Claude, compile a list of 1000 buildings to blow up," and then skim the output briefly and then press the red button, I wouldn't say Claude "decided" anything.
            1. geraneum · · focus · HN ↗
              Of course it did! Among hundreds of thousands it decided those 1000 are the ones to blow up! No matter how we twist the meaning of “decision”.
              1. edoceo · · focus · HN ↗
                Who decided to bomb places? AI? Who decided to pick a fight? AI?
              2. ajsnigrutin · · focus · HN ↗
                We had a public tender for some 3d model of a railroad project a few years ago, three companies applied and for some reason (corruption) the most expensive one was selected to make the model.

                They blamed excel for the bad choice. Excel.

                I mean sure, you can always blame the tool, i can blame my cars rear parking sensors when i hit someone when backing up during parking, but come on... at some point, a human is responsible. Someone, some person, decided and greenlit the decision to bomb those places and should now be in jail for that.

                I mean, what's next? A building collapses, because the load bearing sections are too weak... do we blame CAD tools now?

                1. geraneum · · focus · HN ↗
                  Non of the example you’ve falsely equivocated have the word “intelligent“ in them though, do they?
                  1. joquarky · · focus · HN ↗
                    Just keep moving those goalposts while ignoring your cognitive dissonance.
                    1. geraneum · · focus · HN ↗
                      Ah you must be one of those "Guns don't kill people, people kill people".
                      1. ajsnigrutin · · focus · HN ↗
                        Great example, if someone is killed by gunfire, does the gun go to jail or does the shooter (a person) take responsibility?
                        1. geraneum · · focus · HN ↗
                          I’m assuming you’ve never heard that slogan before. Google it. It’s an interesting read.
              3. archonis · · focus · HN ↗
                If someone made a table of one hundred potential targets and rolled ten ten sided dice to pick a row to target, do we blame the decision on the dice, the table, Wizards of the Coast, or the person who decided this was a good way to choose targets?
                1. geraneum · · focus · HN ↗
                  You’re conflating who chooses the decision maker with the decision making device.

                  So AI’s output is not so different than a roll of dice? Definitely the people who decided to put in such an incapable and unreliable tool should be held accountable. No dispute in that.

            2. GoblinSlayer · · focus · HN ↗
              Then you suddenly realize you outsourced your thinking to Claude, because it's tiresome to think by yourself.
              1. 1718627440 · · focus · HN ↗
                The decision to blow up 1000 buildings, without any background check wasn't done by claude.
          2. sillyfluke · · focus · HN ↗
            You should have just kept the last two sentences.

            >The decision was indeed made by AI and humans failed in their duty to fucking verify it.

            The decision to leave the decision to AI was made by humans. It's on the moronic humans for not understanding the limitations and flaws of the software they are using and believing the marketing material without constantly verifying their results.

            How good or bad the software is an almost equally important but separate issue.

            1. geraneum · · focus · HN ↗
              > The decision to leave the decision to AI

              The decision, to make the decision to leave the decision…

              So much mental gymnastics.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.