U.S. appeals court upholds designation of Anthropic as supply chain risk
Thread
Unofficial Hacker News client; not affiliated with Y Combinator.
U.S. appeals court upholds designation of Anthropic as supply chain risk
Unofficial Hacker News client; not affiliated with Y Combinator.
ApolloFortyNine · · focus · HN ↗
This is like a pen manufacturer not wanting their pens used to sign drone strike orders, now the military needs to have a special box of pens that don't have stipulations attached. With AI usage it would be the same thing except applied to entire product chains. It seems like it would just add more complexity to operations.
You can agree with the rules anthropic wanted, but having rules set by a private company at all that apply to the military does seem fair for the military to object to.
>The Department reasonably feared that Anthropic might manipulate Claude’s design to prevent it from performing national-security functions that the Department deems contractually authorized and necessary
Though they'd probably put the DoD on the cybersecurity whitelist today, the very idea of the claude whitelists for certain functionality already exists and is being used by them today.
smsm42 · · focus · HN ↗
Army: "OK, we will make sure not to use your AIs"
Antropic: "How dare you! We are suing! We intended for you to keep using it but do only what we want!"
Army: "..."
hardbass · · focus · HN ↗
throw310822 · · focus · HN ↗
No, they were against using it to kill people without oversight. I hope nobody thinks that there's a difference between murdering American citizens and murdering people.
hardbass · · focus · HN ↗
sfink · · focus · HN ↗
Anthropic: "We'll sell you access to our AI as long as you don't use it for war crimes or mass surveillance."
DoD: "Ok, deal."
...time passes...
DoD: "Wait, no, we want to be able to use it for war crimes too."
Anthropic: "Too late, you agreed already, and we won't sign a contract for that. You can keep using it for everything we agreed to."
DoD: "Change the terms or we will destroy your business."
Anthropic: "No."
DoD then signs contracts with OpenAI and Google that permit them to commit war crimes and mass surveillance as long as they get legal cover, and proceeds to apply as much leverage as it can to destroy Anthropic's business.
----
And for the record, Anthropic agreed to all "yucky Army things" usage that was an extension of past yucky Army things, they just did not agree to a new class of yucky Army things: namely, fully autonomous killbots. Even there, it was only "not yet, it's not ready". Your whole comment is a gross mischaracterization.
I quit using ChatGPT entirely over this incident, and that's not because I find Anthropic's position to be saintly. They agreed to a contract where the DoD could and in practice is using their AI for committing war crimes.
smsm42 · · focus · HN ↗
US Society has mechanisms for preventing and investigating war crimes. They may not be perfect, and you may not be happy about how they are working, but I doubt that "give control over it to some random guy that makes AI models" is an adequate solution for that. If Antropic decided the risk of DOD circumventing those mechanisms is too high for them to tolerate - they have full freedom not to sell to DOD. Which they used.
> I quit using ChatGPT entirely over this incident
And this is supposed to be an argument because....?
> They agreed to a contract where the DoD could and in practice is using their AI for committing war crimes.
You have a very vivid imagination. Not trying to use it to state things that could be misinterpreted as statements of fact would be nice. Not that I am new on the internet, so I know it's not going to happen.
sfink · · focus · HN ↗
> Anthropic wanted them to be the last word in every decision, and DOD could not allow that
Cite one source for Anthropic demanding to be in the loop. This was about the terms of a contract. If I'm agreeing to sell you a medicine, under the condition that you do not use it to euthanize homeless people, that does not mean I am demanding to follow you around and make you ask me whether each use is ok.
> Nobody is destroying anybody's business - pretending that Anthropic could never find any users for their AI outside DOD is plain silly.
Strawman much? The question is about the US government using its influence in a competitive environment to hamstring one company. It doesn't matter whether there are other customers. It doesn't matter whether the company can survive the attack. What matters is whether it is legal and ethical for our public resources to be used for retaliation in this way.
> US Society has mechanisms for preventing and investigating war crimes....I doubt that "give control over it to some random guy that makes AI models"
This is an invented claim. Cite a source.
>> I quit using ChatGPT entirely over this incident > > And this is supposed to be an argument because....?
The argument was the first part of the post. This part was me giving my perspective on things, in this case it was about how I don't find Anthropic's position to be good, just that OpenAI's is far worse. My post was not for you exclusively; based on what you have written, I have concluded that there is no value in debating you or attempting changing your mind.
>> They agreed to a contract where the DoD could and in practice is using their AI for committing war crimes. > > You have a very vivid imagination. Not trying to use it to state things that could be misinterpreted as statements of fact would be nice.
Ok, fair. Sticking to facts: the crucial part of the negotiation was about whether the DoD can use Anthropic's products for autonomous decisions on lethal force. Since then, it has come out that AI was used in the decision to target an elementary school for girls that killed 120 schoolchildren. (The Palantir Maven Smart System, fwiw, not Claude.) The Pentagon investigated and found that although a human was theoretically in the decision loop, in practice the oversight was minimal and ineffective. "No one remaining on those teams [the teams tasked with minimizing civilian harm] reviewed the strike plans in advance of the attack on the school, according to Bloomberg." Those teams had been gutted, with Hegseth explicitly stating that they were getting in the way.
My supposedly very vivid imagination translated this to reckless decisionmaking that led to a large number of innocent civilian casualties, aka war crimes.
smsm42 · · focus · HN ↗
sfink · · focus · HN ↗
Of course AI-less wars have always had innocent victims. I'm not even arguing whether using AI will produce more or fewer such victims, and at the moment I don't see how anyone could convincingly argue either way. The use of AI is probably a pretty minor driver behind the number of inappropriate targets.
I do argue that people (one or more humans) must be responsible for these decisions, and furthermore having a human in the loop is a necessary prerequisite for such responsibility. (A human is still responsible if there is no human in the loop of individual decisions; the people who allowed that usage were in the loop of making that decision, and thereby have responsibility.)
If we start agreeing that we can just blame the AI for any unintended casualties, then we lose most incentives to reduce such casualties. Even if an AI makes "better" decisions than a human, wars are always resource-constrained and decisions will be made based on responsibility and blame. For example, the decision might be "the AI needs too much data to make accurate decisions, gathering those data is expensive, let's give it less." We cannot allow the "cheapest" decision when push comes to shove to be the one where innocent people are killed because the perpetrators have externalized the responsibility.
In the case under discussion, we're not at the point where anyone is claiming that an AI acting alone will kill fewer unintended people than an AI+a human. The people who develop the technology are specifically saying it is not ready yet. In large part, that comes down to responsibility again -- if Anthropic were to promote that use, then they would be part of the human chain that is responsible for those deaths.
In addition, using AI for autonomous killing is ridiculously dangerous and ill-advised at the present time. They have common and unpredictable failure modes. They have biases that could change something from 99% accurate to way less due to a change that should have no impact on the decisions. When using them for programming, they'll do things like directly go against instructions (whether or not they apologize afterwards) and getting stuck in loops. What are the analogous failure modes when you give them control over lethal weapons? They are not ready.
> > The question is about the US government using its influence in a competitive environment to hamstring one company.
> No company is owed contract from the US government.
The discussion is not about the contract. It is about the supply-chain risk designation. The contract issue is separate (and real! The DoD was trying to retroactively change the terms of an existing signed contract). The DoD is free to not sign any contracts with Anthropic. It is not right, and illegal in my and many people's opinions, to misuse a tool intended for preventing harm from foreign adversaries. It was retaliation, plain and simple.
> We are getting into the territory of "70% of traffic crashes are blamed on sober drivers, therefore driving sober is a crime".
You are, for some reason. I was not. I'm only countering an argument that the particular tragedy that made a splash in the news cannot be discounted by saying "sure it's awful, but irrelevant because AI was not involved." AI was involved. We don't have to imagine hypothetical scenarios where AI was involved and something bad happened. And I'm not saying AI was the cause, or the only cause, of that tragedy.
>> My supposedly very vivid imagination translated this to reckless decisionmaking that led to a large number of innocent civilian casualties, aka war crimes.
> As I said, extensively, the reality of warmaking will inevitably lead to innocent civilian casualties.
The word "reckless" was critical to what I said, because it's part of what makes something a war crime.
Ok, I give up. You're equating war crimes and any civilian casualties. Or wait, then you go on to equate war crimes with any suffering of innocents. Why? Why are you so set on dismissing the possibility of a war crime being a thing that happens, and has an actual definition that goes well beyond innocent people suffering? You seem motivated to steamroll any obstacle to using maximal force regardless of the consequences. Why? Hegseth, is that you?
Never mind, I don't think I need to know your answer. You are a rando on the internet to me.
jsiepkes · · focus · HN ↗
smsm42 · · focus · HN ↗
SpicyLemonZest · · focus · HN ↗
smsm42 · · focus · HN ↗
SpicyLemonZest · · focus · HN ↗
SpicyLemonZest · · focus · HN ↗