Thread

Pentagon says overreliance on AI contributed to missile strike on Iran school

974 points · 553 comments · devonnull

63 · 1790106616 · focus · HN ↗
It's easy to blame the military and the higher-ups, after all they made this choice. But more relevant to HN, this couldn't have happened without technology professionals working for Palantir.
> A Palantir spokesperson told Bloomberg the company “is not responsible for the underlying data nor identifying intelligence deficiencies” and that there is no evidence its software was at fault.
If you work for Palantir, you are writing software that kills people. Software that, in this case, killed 123 children. What salary is worth that?
I encourage others who work in the public sector, whether directly or through a contractor, to read and internalize Don't Get Distracted by Caleb Hearth [0]. I think about it nearly every day.
[0] <a href="https://calebhearth.com/dont-get-distracted" rel="nofollow">https://calebhearth.com/dont-get-distracted
1. PorciiVorbesc · 1790107230 · focus · HN ↗
  >It's easy to blame the military and the higher-ups
  My brother in Cthulu, they're the ones making the life or death decision that killed people, who else should be to blame?
  >But more relevant to HN, this couldn't have happened without technology professionals working for Palantir.
  If you wanna go the historical tech-tree scapegoat route then none of this would have happened if DARPA wouldn't have invented the internet, or if Thomas Edison wouldn't have invented the lightbulb, or if the fish haven't ventured onto land and evolve into us land mammal apes who invented sharpened sticks as murder objects.
  1. whatshisface · 1790107593 · focus · HN ↗
    I know, right? I'm not even sure if this is the biggest installation on Earth. If it's possible for us, it's possible for any number of private military contractors. I bet there's a similar capability in the hands of every advanced county on Earth. Anyway, shaddup, the focusing crystal's not calibrating right, and tomorrow I have present the targeting parameters to Dr. Evil.
  2. cogman10 · 1790108693 · focus · HN ↗
    Look, if Palantir had any redeeming qualities then I might agree with you. Unlike electricity which runs our computers, homes, cars, etc. What does Palantir do? It spies on people and helps governments to create "people we don't like" lists. In the absolute best case it's a "minority report" engine, but it's still just there to try and stop pre-crime (not crime).
    This isn't the case where we are talking about a company doing anything socially good. It's simply there to help authoritarian governments.
    PorciiVorbesc · 1790108894 · focus · HN ↗
    Palantir only exists because government wants and funds these types of tools. If it Palantir wouldn't exist then another dozen companies with different names would exist that would create the same Torment Nexus for the government due to inexhaustible demand for such tools.
    >It's simply there to help authoritarian governments.
    The fact that you're so close yet so far away to figuring out who's to blame still boggles my mind.
    smackeyacky · 1790113583 · focus · HN ↗
    Are you implying that the demand for these evil services absolves the suppliers from any moral responsibility? Simply because “if we don’t do this somebody else will”.
    It’s an evil service and no matter who requests it or supplies it, doing so is an evil act.
    PorciiVorbesc · 1790113861 · focus · HN ↗
    >Are you implying that the demand for these evil services absolves the suppliers from any moral responsibility?
    No, you are implying that. I'm saying that evil are (primarily) those who use it, not (just) those who make it. You can try to make whatever you want, if nobody will buy it to fund it, then you won't have money to make it, so your evil thign won't exist. Palantir only exists because governments are evil and willing to fund it to buy it and use it. If you want likes of Palantir to not exist then you need to hold your government accountable to not spend money on evil things.
    Are gun makers evil and responsible whenever someone gets killed by a murderer, or are those who pull the trigger responsible for who they kill?
    smackeyacky · 1790114243 · focus · HN ↗
    I absolutely knew you’d bring up guns and gun making but it’s not a valid comparison. If the gun companies were also involved in the actual shooting of people part that would work.
    Palantir were in no way obliged or coerced into making an evil thing, they just decided being evil for money was ok.
    PorciiVorbesc · 1790121896 · focus · HN ↗
    >I absolutely knew you’d bring up guns and gun making but it’s not a valid comparison. If the gun companies were also involved in the actual shooting of people part that would work.
    Do you have a non-Reddit-level argument on this?
    >Palantir were in no way obliged or coerced into making an evil thing, they just decided being evil for money was ok.
    You keep beating it around the bush and avoiding to address the point that if Palantir wouldn't do that, then dozens of other companies would.
    You can't expect for society and businesses to rely on morality in order for evil apps to never be made, you need the government to do that by rule of law and force behavior of companies so no Palantir ever exists.
    smackeyacky · 1790123809 · focus · HN ↗
    Personal insults only weaken your argument.
    Palantir could decline to build this and so could everybody.
    If somebody proposed to you that you could have a lucrative contract in exchange for an intrusive system that provides a bedrock for authoritarianism, you are arguing it’s perfectly ok because if not you, somebody else will and that makes it OK.
    It doesn’t.
    PorciiVorbesc · 1790125558 · focus · HN ↗
    >Palantir could decline to build this and so could everybody.
    The issue with your broke argument you keep repeating is that it only works in a fantasy setting where the world only runs on morality, where countries could also decide to not have nuclear weapons and never got to war with each other and share resources fairly, and end world hunger and all that, while my argument is based on how things work in the real world which isn't run on morality but on rule of law by force, and power corrupts, so the world is run in a corrupt fashion meaning existence of various Palantirs is virtually inevitable.
    >you are arguing it’s perfectly ok because if not you, somebody else will and that makes it OK.
    No, I never said that. You're good at pretending to misunderstand things I never claim.
    bigyabai · 1790135719 · focus · HN ↗
    There are several things that make Palantir uniquely objectionable, though:
    1) They're private-sector and have low hiring standards, which can lead to Boeing-esque quality control issues
    2) They have every incentive to fleece US taxpayers, becoming another over-fattened rice-bowl defense contractor
    3) Exporting Palantir products could create diplomatic crises and conflicts of interest undermining American security (see: NSO Group)
    We can't ever avoid spyware since Google, Microsoft and Apple already built the perfect surveillance state a decade ago. But we can prevent America from being scammed by a two-bit chump like Peter Thiel.
    _heimdall · 1790153678 · focus · HN ↗
    Would you hold that same line for individuals that built weapons, planes, bombs, etc in past wars?
    Companies that make bombs aren't forced to (usually), and they also know there's only really one use for a bomb.
    cogman10 · 1790115424 · focus · HN ↗
    > Are gun makers evil and responsible whenever someone gets killed by a murderer, or are those who pull the trigger responsible?
    Yes. There's a reason Remington decided to settle for $73M dollars with the sandyhook parents even though they didn't "pull the trigger" [1]. That's not a number you pick because you think you'll win the case.
    [1] <a href="https://abcnews.com/US/sandy-hook-families-settle-remington-marking-1st-time/story?id=82881639" rel="nofollow">https://abcnews.com/US/sandy-hook-families-settle-remington-...
    PorciiVorbesc · 1790116076 · focus · HN ↗
    >There's a reason Remington decided to settle for $73M dollars with the sandyhook parents
    It's only because of how the legal system works in the US, with ambulance chasing lawyers and where cases are decided on the emotions and societal moral panics of the people in the jury and the negative image such cases can cause to a brand, not if someone or a company is objectively right or wrong.
    Meaning it's cheaper to pay people off even if they're legally wrong, than try to prove you are right but then deal with the bad publicity of the emotionally charged mob with pitchforks and torches who blames you for their kids dying and looks bad in the media.
    Keep in mind parents of that time also blamed violent videogames like Doom for it, so it's not like they were right about who was to blame, they were just looking for retribution from anyone they could, aided by unscrupulous ambulance chasing lawyers looking to cash in.
    _heimdall · 1790153523 · focus · HN ↗
    Jury trials definitely have their flaws, but what would you propose as a better approach?
    8note · 1790116088 · focus · HN ↗
    its clearly a democratically approved legal act though. If its as evil as you suggest, the better approach would be to convince your peers that its evil and thus should be made illegal
    smackeyacky · 1790116803 · focus · HN ↗
    Again, absolving the company that built it and runs it. They weren’t coerced into doing so and must share the blame for the evil that was created.
    mistercheph · 1790152331 · focus · HN ↗
    Listen guys, i cannot be held responsible for how an evil government chooses to use my child torture machine, if I didn't make it someone else would have, you cannot blame me, I'm just great at building stuff and trying to put food on the table

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.