‹ BackHN Continuity

Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

  1. legitster · · focus · HN ↗
    > It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.”

    Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.

    The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.

    The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.

    1. beloch · · focus · HN ↗
      "In the final days of February, the Trump administration demanded an overwhelming aerial assault. More than 1,000 Iranian targets were hit within the first 24 hours alone. That order compressed the time frame for finalizing targets, the officials said, instilling an urgency that — combined with gaps in underlying information about the site, cuts in civilian-protection personnel and an overreliance on artificial-intelligence technology — set the stage for the deadly error."

      -------------------

      Maven is an untested tool for rapidly selecting targets under time pressure during a war. It's reason for existing is speed. If you're in a shooting war, the ability to select good military targets faster than your enemy, but not necessarily with zero error, is an advantage.

      Planning a first strike is a different situation. Time pressure should not be a significant factor. Humans should have been doing their due diligence on every single target. If an untested tool for rapid mid-battle target selection was relied upon completely because of a hasty executive order, then the responsibility falls entirely on the executive.

      1. cgio · · focus · HN ↗
        I think that’s the important distinction. Mistakes under pressure do happen. Unnecessary pressure is not a mistake, though, it’s a crime in this context. They could start planning earlier or commit to action later.
        1. ygouzerh · · focus · HN ↗
          I feel as well that it's the strongest reason. Even without AI, time pressure statistically already increase the error rate
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.