‹ BackHN Continuity

Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

  1. banubis · · focus · HN ↗
    The pentagon says it was Palantir software’s fault, Palantir says it was the fault of bad data input to their software. Where’s the accountability by both organizations? How did we get to a world where military decisions leading to innocent deaths are responded to as if it was just another miscommunication implementing a B2B SaaS platform.
    1. hyperhello · · focus · HN ↗
      Small children often develop coping mechanisms for their lack of agency. It may be part of these people’s worldview since early childhood that anyone who opposes them should magically die.
      1. paimapi · · focus · HN ↗
        *anti-social coping mechanisms

        you can also develop pro-social coping mechanisms like open-mindedly talking to someone who opposes you and trying to understand their worldview because you trust that it might expand your own

        but, yeah, I doubt pro-social traits are common at the top of any hyperstrict hierarchical organization like the USAF

    2. augment_me · · focus · HN ↗
      The new world is complex and AI technology has outpaced the regulation cycle, so it's inevitable that we come to a place where you don't have accountability and can never have it because 0) the courts can't handle the case, they don't have the frameworks for it, so it takes longer time 1) you can overload the courts with slop, 2) by the time there is a decision the world has moved on and technology has moved on

      Really good time to do various crime.

      1. croon · · focus · HN ↗
        I disagree with this framing. Nothing in accountability has really changed other than our willingness to enforce it.

        Replace "AI" with any other tool or software in your scenario, and the outcome is the same. The same people and organizations are accountable, it's just that we currently live in a world without accountability in the highest places where it's most needed, and that's unrelated to AI.

        1. augment_me · · focus · HN ↗
          You disagree that there is no difference between the amount of material you can generate today compared to 10 years ago?

          Or with that the EU AI Act has been redundant and postponed in 2023 because it was drafted on ResNets, and then redundant and postponed in 2025 because it was drafted on ChatGPT 3.5, and then this year probably again because it didn't have agents included?

          Or with that other countries that getting their companies hacked by AI models are ignoring any kind of prosecution of said companies and delegating this to the US?

          1. croon · · focus · HN ↗
            That's a lot of strawmen unrelated to what I wrote.

            We're lacking in enforcement, not lagging regulation (regardless of if/that I agree it might be needed).

            If we put this under a US focus lens, then I believe the unaccountability currently would be there exactly the same if we removed this generation of AI from the timeline, yes, and that lack of enforcement is the core issue, unrelated to issues with AI.

        2. forshaper · · focus · HN ↗
          Yeah, all this was going on decades before AI.
      2. s5300 · · focus · HN ↗

        [dead]

    3. gambiting · · focus · HN ↗
      To me it's extremely obvious that this is the desired outcome - no accountability for anyone with this setup.
    4. testing22321 · · focus · HN ↗
      This is the same response they’ll have when the AI launches nukes and starts annihilating the human race. No accountability till the end.
    5. rossng · · focus · HN ↗
      You might be interested in Dan Davies' book The Unaccountability Machine[1].

      It talks a fair bit about 'accountability sinks', which are exactly these systems that (intentionally or not) essentially serve as a way to unlink responsibility from any given person and allow 'the system' to be blamed instead.

      [1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;The_Unaccountability_Machine" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;The_Unaccountability_Machine

    6. Cameri · · focus · HN ↗
      Accountability was not in the NFRs.
    7. _heimdall · · focus · HN ↗
      Military accountability, and government accountability in general, has been and will always be a problem. As long as those in charge make and enforce the rules they will always look out for themselves first.
    8. _joel · · focus · HN ↗
      That&#x27;s by design.
    9. jackb4040 · · focus · HN ↗
      Well in theory the military is deferential to a political class that is elected by and thus accountable to the people. But that is so obviously no longer the case that politicians who pretend it is are perversely less popular than ones who are openly minoritarian.

      The logical response to such a self-conscious rejection of democracy is unironically political education and beginning revolutionary political organizing in your neighborhood. But I rarely bother making that point here; HN is probably the least revolutionary-minded grouping in the world outside of tech billionaires themselves.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.