Thread

U.S. appeals court upholds designation of Anthropic as supply chain risk

499 points · 900 comments · cramer4next

ApolloFortyNine · 1790356118 · focus · HN ↗
I know everyone says this is political but it actually seems like a textbook designation. Anthropic wanted to have rules on how the military used AI, the military said no and therefore doesn't want anthropic used anywhere in their supply line.
This is like a pen manufacturer not wanting their pens used to sign drone strike orders, now the military needs to have a special box of pens that don't have stipulations attached. With AI usage it would be the same thing except applied to entire product chains. It seems like it would just add more complexity to operations.
You can agree with the rules anthropic wanted, but having rules set by a private company at all that apply to the military does seem fair for the military to object to.
>The Department reasonably feared that Anthropic might manipulate Claude’s design to prevent it from performing national-security functions that the Department deems contractually authorized and necessary
Though they'd probably put the DoD on the cybersecurity whitelist today, the very idea of the claude whitelists for certain functionality already exists and is being used by them today.
1. comnetxr · 1790361804 · focus · HN ↗
  I think the better analogy is an insane nuclear power plant manager deciding it wants to buy pens to use as neutron-flux regulator rods — because after all, a pen is functionally a pencil and a pencil is made of graphite.
  Then the pen manufacturer hears about this and says “Our pens are not made of graphite and are not suitable to be used in nuclear reactors”, to which the reactor owner says “it’s fine, they fit in the graphite rod holes, and we’re just using until the next generation of pens come out which will do an even better job”, and then the pen manufacturer says “I’m not going to sell you any pens until you agree that they will be used only writing.”
  1. rayiner · 1790384063 · focus · HN ↗
    This isn't like using pens in a nuclear plant. AI obviously has tremendous military applications. And it's completely unsurprising that the U.S. military--whose entire doctrine for the last century has been about having the most advanced technology--demands to use frontier models without non-military people imposing restrictions.
    Have you seen the movie Oppenheimer? The U.S. military did not care about the moral opinions of the nuclear scientists--they were going to get nukes first. This is like Truman telling off Oppenheimer for being upset after the U.S. used the bomb: <a href="https://www.biography.com/political-figures/a44361438/why-harry-truman-didnt-like-oppenheimer-atomic-bombs" rel="nofollow">https://www.biography.com/political-figures/a44361438/why-ha... ("Blood on his hands; damn it, he hasn’t half as much blood on his hands as I have. You just don’t go around bellyaching about it,” Truman said.... He called Oppenheimer a “cry-baby scientist” and said, “I don’t want to see that son of a b–– in this office ever again.”).
    ben_w · 1790407539 · focus · HN ↗
    > AI obviously has tremendous military applications.
    Yeah, but also we can be reasonably confident they're not good enough for many of them yet:
    What would be the military equivalent of all the things on <a href="https://www.felonybench.com/" rel="nofollow">https://www.felonybench.com/ ?
    Machine learning needs a lot of examples to get good. Wars and wargames (including digital ones with unbounded self-play time) and all the textbooks together will at best, on the basis of what we see in other fields, get the equivalent of a new officer fresh out of training… and then, given current observed agentic AI behaviour, "follow orders" rather than "follow lawful orders".
    rayiner · 1790431205 · focus · HN ↗
    > Yeah, but also we can be reasonably confident they're not good enough for many of them yet
    What does “not good enough mean?” Who decides what is “good enough” and what tradeoffs are acceptable? Obviously it’s the military’s job to decide that.
    ben_w · 1790432295 · focus · HN ↗
    In the context of my comment, "not good enough" includes "committing war crimes" and "would have started a war with China if there hadn't been a human in the loop".
    We can also infer from other domains, vendbench or attempts to have an LLM play Star Craft, that these models are currently extremely mediocre at management roles.
    > Who decides what is “good enough” and what tradeoffs are acceptable? Obviously it’s the military’s job to decide that.
    In practice, in the case of the USA, Trump and Hegseth.
    The US military, so far as I can tell, are rather less thrilled with these adventures.
    rayiner · 1790437685 · focus · HN ↗
    Right—and deciding what constitutes “war crimes” isn’t within the expertise of AI developers. They can explain the abilities and limitations of their models to the military leadership. But the actual decision about how to use AI is up to the military through its chain of command. That starts with Trump as the duly elected president and Hegseth as the duly appointed civilian head of the military. They are the democratically accountable actors. And all this is being done in the open, so if the people have a problem with it they can hold it against Trump and his party.
    That decision assuredly does not belong with people like Amodei, who are not democratically accountable.
    ben_w · 1790442646 · focus · HN ↗
    > That decision assuredly does not belong with people like Amodei, who are not democratically accountable.
    Has Amodei been conscripted?
    If not, he's free to say "no", just as I'm free to refuse to work for e.g. a defence contractor if I don't like what's going on or if I don't feel competent for the role.
    He, like me, is allowed to not want deaths on his conscience if he feels his own business' best product isn't good enough.
    mayhemducks · 1790431836 · focus · HN ↗
    Oppenheimer was the guy who was pushing that opinion just as much as the military folks. In the case of AI, my understanding is that Amodei is not trying to beat the enemy to it.
    What I don't understand is why the U.S. military is turning to a for-profit corporation when they could fund and pursue the development themselves. Why are they trying to "outsource" it to industry?

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.