Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

imranq · 1790105401 · focus · HN ↗
> Officials said some users expected Maven to flag stale records or contradictions in the intelligence assembled for potential targets, though it is not clear why they thought the system would do that
This is what happens when people who don't understand AI and its limitations embrace it wholeheartedly, thinking it's going to behave like The ultimate analyst. Except it's not. Its got blind spots in more places than anyone knows. And not to mention the system itself has multiple areas of failure.
1. shimman · 1790106943 · focus · HN ↗
  No, this is what happens when software developers don't understand how users actually use their products.
  This was a totally expected outcome, similar to how CV cameras arrest the wrong minorities in neighboring states 100s of miles away because it told the officer it was them; or how health insurance companies use ML algorithms to deny Americans care.
  It's going to happen in other sectors of life and it will continue killing innocent civilians.
  1. whycome · 1790124887 · focus · HN ↗
    Are you saying that letting insurance companies track your driving data will get used to do things like fine drivers directly for law breaking? Never!
    shimman · 1790130249 · focus · HN ↗
    No, I'm saying health insurance companies routinely deny American civilians life saving care because they'd make more money and routinely use similar software that also allowed 200 school age girls to be murdered. Recalling that a CEO was murdered and the person excused is being hailed as a hero over just this. Think people care about this a lot actually.
    <a href="https://www.newsweek.com/united-healthcare-ceo-shooting-ai-lawsuit-1996266" rel="nofollow">https://www.newsweek.com/united-healthcare-ceo-shooting-ai-l...
    <a href="https://www.ama-assn.org/press-center/ama-press-releases/physicians-concerned-ai-increases-prior-authorization-denials" rel="nofollow">https://www.ama-assn.org/press-center/ama-press-releases/phy...
    <a href="https://www.palmbeachpost.com/story/news/healthcare/2026/03/30/ai-is-denying-health-care-claims/88221783007/" rel="nofollow">https://www.palmbeachpost.com/story/news/healthcare/2026/03/...
    _heimdall · 1790152639 · focus · HN ↗
    For better or worse, health care providers at least in the US are for profit businesses. There's been plenty of arguments made to change that, but as it stands we shouldn't be surprised when a for profit business decides what service they provide and to whom they provide it.
    ImPostingOnHN · 1790199328 · focus · HN ↗
    Likewise, we shouldn't be surprised when someone who puts profits over other's lives gets ventillated by one of those lives in downtown New York. And what do you know, nobody was.
    _heimdall · 1790221331 · focus · HN ↗
    What so you mean by ventilated here?
    Forgive my ignorance, I just don't get the reference and don't want to assume its anything front he pandemic response to United Healthcare.
    ImPostingOnHN · 1790270396 · focus · HN ↗
    I mean the person in question had new air vents rapidly installed in their body.
    _heimdall · 1790387901 · focus · HN ↗
    Are you saying that's a good thing?
    ImPostingOnHN · 1790440199 · focus · HN ↗
    My post very unambiguously explains that is an unsurprising thing. It does not explicitly say it was a good thing, so no, "good" is not specifically what I am saying in that post.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.