‹ BackHN Continuity

Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

  1. legitster · · focus · HN ↗
    > It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.”

    Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.

    The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.

    The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.

    1. xlayn · · focus · HN ↗
      Now, ask yourself if you want AI taking decisions around your life... it will pull numbers... someone will have a "goal" of 1000 "hits" and there you go, 2-3 months in jail working to check what went wrong...

      But don't worry, the other P AI system will ensure that the first one doesn't do mistakes...

      1. toomuchtodo · · focus · HN ↗
        Lawsuit Says UnitedHealth Used AI With a 90% Error Rate to Deny Care - <a href="https:&#x2F;&#x2F;www.yahoo.com&#x2F;news&#x2F;us&#x2F;articles&#x2F;lawsuit-says-unitedhealth-used-ai-173017952.html" rel="nofollow">https:&#x2F;&#x2F;www.yahoo.com&#x2F;news&#x2F;us&#x2F;articles&#x2F;lawsuit-says-unitedhe... - September 18th, 2026

        UnitedHealth uses AI model with 90% error rate to deny care, lawsuit alleges - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=38299921">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=38299921 - November 2023 (58 comments)

        <a href="https:&#x2F;&#x2F;litigationtracker.law.georgetown.edu&#x2F;wp-content&#x2F;uploads&#x2F;2023&#x2F;11&#x2F;Estate-of-Gene-B.-Lokken-et-al_20231114_COMPLAINT.pdf" rel="nofollow">https:&#x2F;&#x2F;litigationtracker.law.georgetown.edu&#x2F;wp-content&#x2F;uplo...

        1. teravor · · focus · HN ↗
          that smells like bullshit to be fair. if you have a system that makes errors over 90% of the time then you have a system you can leverage to make fewer errors.

          the situation would have to be very particular&#x2F;contrived for that not to be the case.

          1. j4yav · · focus · HN ↗
            I can easily build you a system that will guess the price of a stock tomorrow wrong at least 90% of the time. Even 100% wrong if you&#x27;d like. How would you use that to make fewer errors in some other system?
            1. [deleted] · · focus · HN ↗

              [deleted]

            2. 1718627440 · · focus · HN ↗
              If it is a binary choice you can just do the opposite.
              1. j4yav · · focus · HN ↗
                What’s binary choice we are discussing here and what’s the opposite that you’d do? Approve every claim that was denied by the AI, and vice versa? Or what’s the binary choice related to stock prices you’re referring to?
                1. 1718627440 · · focus · HN ↗
                  I agree, that most real world problems, are not binary choices. But in this case actually yes, just do the opposite.

                  For that reason error rates above 50% don&#x27;t make any sense. Building a system, that is reliably wrong, is as hard as building one that is reliable correct, because the systems are the same, just with the output inverted.

                  The worst you can get with a system is 50%, at which point you can just flip a coin, because that is just pure random. Any deviation from that is going to be an improvement.

                  1. [deleted] · · focus · HN ↗

                    [deleted]

                  2. j4yav · · focus · HN ↗
                    I always thought insurance claims were more complex, some things may be wholly or partially covered on a line by line basis, and there should be justification provided. In certain cases they go to mediation or even court. It doesn’t seem binary to me but maybe it works differently where you are.
                    1. 1718627440 · · focus · HN ↗
                      I said &quot;if it is&quot;, not that it would be.
                      1. j4yav · · focus · HN ↗
                        Oh, haha, sure.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.