‹ BackHN Continuity

Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

  1. jtrn · · focus · HN ↗
    Things that jumped out at me.

    If true, there was no verification of the targets selected by AI, meaning AI is the final arbiter of the deadly use of force. Another Rubicon passed.

    Trump pretends like it didn’t happen: “We might never know what really happened there” is a frustrating lack of responsibility.

    Still missing, as it always is from all of these kinds of articles, is: How common is this error rate compared to pure human evaluation and decision-making? I’m not saying I support AI-assisted killing, but why is nobody comparing it to human failure rates? If the article is true, then there were 1,000 targets hit in the first 24 hours. If the majority of the legwork and selection was done by AI, how many schools, or equivalents, would one expect to be wrongly hit if all the targeting work were done by humans only?

    1. ImPostingOnHN · · focus · HN ↗
      > If the article is true, then there were 1,000 targets hit in the first 24 hours. If the majority of the legwork and selection was done by AI, how many schools, or equivalents, would one expect to be wrongly hit if all the targeting work were done by humans only?

      An interesting thought exercise: given unlimited time (the usa was not on a deadline to initiate a first strike against iran), can a human tell a school building from a military building?

      The answer is almost certainly "yes": watch the building for 1 day, and if children come out or go in, you do not bomb it.

      1. jtrn · · focus · HN ↗
        The reverse ask is, given unlimited time could an AI tell a school building from a military building, better or worse than a human?
        1. ImPostingOnHN · · focus · HN ↗
          Worse, than a team of humans who have been doing that for decades, for whom that is literally their one job, and who have on-site recon.

          For an example, see the incident described in the article, with context that beforehand the usa had this team of people, and then the usa swapped the people out for the AI, and then this incident happened.

          1. jtrn · · focus · HN ↗
            It's a bit hard to know exactly what happened, I would think. But I get your point. My feeling at the moment is that AI can do surprisingly good work compared to humans, like 80% equality for some tasks at a fraction of the price. The problem is that if you then uncritically just replace every human, you quickly are going to realize that there are a lot of instances where you needed those extra 20% capabilities.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.