‹ BackHN Continuity

Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

  1. 63 · · focus · HN ↗
    It's easy to blame the military and the higher-ups, after all they made this choice. But more relevant to HN, this couldn't have happened without technology professionals working for Palantir.

    > A Palantir spokesperson told Bloomberg the company “is not responsible for the underlying data nor identifying intelligence deficiencies” and that there is no evidence its software was at fault.

    If you work for Palantir, you are writing software that kills people. Software that, in this case, killed 123 children. What salary is worth that?

    I encourage others who work in the public sector, whether directly or through a contractor, to read and internalize Don't Get Distracted by Caleb Hearth [0]. I think about it nearly every day.

    [0] <a href="https:&#x2F;&#x2F;calebhearth.com&#x2F;dont-get-distracted" rel="nofollow">https:&#x2F;&#x2F;calebhearth.com&#x2F;dont-get-distracted

    1. gameshot911 · · focus · HN ↗
      If I write healthcare software, and a doctor orders the wrong drug that kills the patient, am I responsible?
      1. ekzy · · focus · HN ↗
        Legally, probably not...?

        Whether you may&#x2F;should feel responsible depends on how the software works, I guess. Did it automatically recommend some drug, somewhat confidently, with gray text on gray background saying &quot;AI sometimes makes mistakes&quot;?

        So I think we should behave responsibly when making software, even more now with AI

      2. kelnos · · focus · HN ↗
        It depends. Did you write a bug that caused the wrong drug to get ordered? Did you design or implement an algorithm that recommends drugs, and the algorithm was a bad one? Did you design or implement a confusing UI flow that led to the doctor ordering the wrong drug?

        If so, I&#x27;d say yes, you share some of the responsibility. Legally, I&#x27;m sure you&#x27;d be fine, but if that were me, I&#x27;d feel responsible.

        The doctor of course is responsible too; they&#x27;re the one who didn&#x27;t double-check that it was an appropriate drug. But multiple people can share responsibility for a bad thing.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.