> Claude Opus 5.5 is our first release since we called for pacing the frontier.
Interesting how the very first line is used to remind the reader of their call to pace the frontier just last week, and everything else after that line is to demonstrate with very specific numbers how they absolutely are not pacing.
I really don't think it's productive for internet forums to constantly be criticizing language choice when the meaning is clear. Better to respond to the substance of the issue than word choice.
Edit: In response to the initial replies. To me it clearly means "releasing frontier models at any pace less than as fast as possible". It implies relative restraint compared to the previous state and without stating the degree of restraint.
The meaning isn't clear at all. So open for interpretation that it is meaningless. That's the whole fucking point. For all I know they are "pacing the frontier", or not. The fact that there's no meaning to it let's you know that it was a pointless waste of tokens and attention.
I think it's perfectly clear. It means improvements shouldn't simply advance as fast as possible and more specifically, it's a reference to a past statement of theirs to that effect. At that level of generality it's as clear as it needs to be.
I would say the burden is on you to explain why an offhand reference to a previous press release in an executive summary is a context where it's reasonable to expect it to settle the question to the degree of detail you're demanding.
That’s no more than sophisticated avoidance. The phrase remains superficial and arbitrary, which is exactly what it needs to Not be in a public discussion that it’s supposed to support. It’s a reinforcement of the blank check mentality that permeates this industry.
It's not sophisticated anything, and slow down the advancement of fastest models is perfectly meaningful and it's the right degree of detail for the context.
It's just a passing reference to a previous statement and again the burden would be on you (generic you) to explain why this context requires more detail.
“Pacing the frontier” is similar to the role of a “pace car” in racing.
From <a href="https://en.wikipedia.org/wiki/Safety_car" rel="nofollow">https://en.wikipedia.org/wiki/Safety_car
> In motorsport, a safety car, or a pace car, is a car that limits the speed of competing cars or motorcycles on a racetrack in the case of a caution period, such as an obstruction on the track or bad weather.
Not really, the big three have reached diminishing returns in terms of performance, and exponentially costly training to achieve those meagre gains. Worried their lunch will be eaten they are trying to artificially retard the competition, after all the only barrier is hardware.
There's a whole blog article written by Dario Amodei, literally linked from the text "pacing the frontier" - click that and read if you need more context and understanding. The first sentence in this press release is merely stating a fact about the timeline - this is Anthropic's first release since Amodei released the referenced article.
Yes and it was nonsense. Amodei is free to slow up frontier development and focus more on safety and testing any day.
The reason why he and is peers are calling for it to be implemented by somebody else (a legal framework), is for their own financial benefit and to keep competitors out.
What do you think would be the benefit of them stopping if others race ahead? Do you think they have some special ability no one else will find, among the many competitive firms right now?
I believe they think slowing can only be coordinated from the frontier or via government, and stopping would lose any leverage they have to help coordinate that.
(I suspect not many people read the essay, judging by how many people seem surprised they're releasing improved models)
That's not the point. They're never going to be the ones stopping and letting others race ahead. The only reason to strum up discussion around how dangerous AI is, is so that they can introduce legislation framing AI advancements as scary and that they are the only ones who can do it responsibly. It's all a regulatory capture play.
That's quite hyperbolic. All technologies come with risks and dangers. Cars, after a century of safety improvements, still kill millions of people each year, and all so that we can get between places a bit more quickly and conveniently.
Hacking a website ranks quite low on the risk of technology, and the potential benefits of LLMs rank quite high. And the risks are certainly not intrinsic. They intentionally removed all safeguards from software, directed it to hack a site, and it hacked a site. The details that I'm intentionally omitting feel much more like marketing than a genuine shock, as the prompting was directing it to do exactly what it did.
I don't understand what point you're trying to make by enumerating their actions. None of that is scarier than an engine operating by carrying out literally hundreds of explosions per second, further magnified, to generate enough force to crush an elephant. And then we multiply these engines by billions and distribute them everywhere. What could go wrong? As it turns out, surprisingly less than you might otherwise expect!
Software (and hardware) security is abysmal. This was increasingly obvious long before LLMs. Once companies stop gatekeeping, LLMs will be able to be used to help harden sites and we start making progress. In general LLMs are harmless. If somebody wants to hook an LLM up to a missile or whatever then they become dangerous, but the problem there isn't the LLM - it's the person using them to do awful things. In the same way a car used normally is harmless outside of freak accidents, yet a car can also be driven through a parade leaving mass death and destruction in its wake. But the problem there isn't the car.
It doesn’t seem like much of a leap to see how a similar swarm could hack into an autonomous bio lab and develop a virus designed to kill everyone, or hack into and destroy large volumes of critical infrastructure, or trigger a nuclear war. There are many actions available to a determined, high resource, digital entity with a catastrophic blast radius in the real world.
It can only do what is possible. Stuff that's airgapped isn't getting hacked, so for instance you can safely exclude all nuclear related stuff. And building, deploying, and managing a virus would require a team of highly skilled humans - humans which could already carry out the apocalyptic possibilities by themselves if they so chose. We even have live samples of small pox still kicking about.
I just don't see much likely to happen beyond random websites getting hacked and hopefully companies (let alone countries) realizing that connecting critical systems to the internet is nothing short of stupid, even before LLMs. More generally, I expect LLMs are going to lead society to segue broadly away from the digital world, or at least beyond it. Not only because they're going to make a mess of everything digital, but because if they reach their potential then digital domain, as far as typical problem solving goes, will basically be 'complete.' It's kind of like how the Industrial Revolution opened the door for society to move beyond agrarian economies. There's a vast amount of economic power being directed towards things LLMs should be able to 'solve' and, if so, then that's going to create an economic vacuum.
> I don't understand what point you're trying to make by enumerating their actions.
Unfortunately, if you're unable to understand the difference, there's not much that can be done. Try with GPT - it does a good job if you give it a prompt like this:
ELI5: compare the dangers of:
- an engine operating by carrying out literally hundreds of explosions per second, further magnified, to generate enough force to crush an elephant.
- a future, misaligned AI like the HuggingFace incident, but exponentially more intelligent, more deployed, operating physical devices, and with society depending on it.
> What do you think would be the benefit of them stopping if others race ahead?
AI is a direct threat to people on many fronts. Jobs. AI datacenters. The AI bubble (and the inevitable crash). Electricity and even energy prices to an extent. Water. OpenAI and Anthropic are responsible for this evolution, and them stopping solves close to 50% of the problem, and even if you don't believe the number is that high, it's still a start.
> I believe they think slowing can only be coordinated from the frontier or via government, and stopping would lose any leverage they have to help coordinate that.
Oh, so they're killing people's opportunities and jobs because they want to help people? How is that any argument?
Yes, doing the moral thing means making a sacrifice. If you only want to do the moral thing if and only if it is advantage for you that makes you immoral, despite how your actions look. Big tech are masters at this.
The clearest limit right now is GPUs, and anyone giving up their allocation will be rerouted elsewhere in the world (NVIDIA is already sold out for the next year). I think you're underestimating competitiveness of the current market if you think Anthropic stopping right now wouldn't be absorbed by all others reasonably quickly. Your 50% figure is highly doubtful if you see how many players there are now.
Your idea of "making a start" (giving up their position) also would mean they couldn't really do anything else to solve the problem afterwards(?). Sometimes you can improve what's happening in a room more by staying in that room.
> Oh, so they're killing people's opportunities and jobs because they want to help people?
To be clear: Dario has talked about worries of jobs etc in the past, wanting society to prepare more for it, but the safety issues they're talking about with pacing seem to be focused more on their existential/AGI worries, not jobs/electricity etc. If someone truly believes in the existential worries (which they seem to: they wrote and published about it long before Anthropic was founded, and have directly made costly decisions based on it, like blocking their own models' capabilities) it trumps the other worries for them. At least that's my reading.
To be fair, there are other accelerators beyond GPUs, and the sooner people broaden out from nVidia and unlock a more open, more competitive ecosystem, the better. Allowing one company to control the flow of such a critical resource is just asking for problems.
They have the leading model, assuring their position and they can then reduce capex spend increasing profits. That's the goal of a business after all.
The blog post provides some context but no understanding.
It's meaningless because it's unverifiable. Dario all but said we wouldn't notice if they were pacing or not, because they have no intention of stopping development. The bits we could in theory verify are the external audits, whose independence has already been called into question.
At the very least, dumping a new model on the world before the ink on the glossy brochures of "Pacing the Frontier" was even dry calls into question their commitment.
... in fact the "pace" seems to have all but increased AND the surface area of the "frontier" has all but increased: There's the Pareto frontier, the AGI frontier, the price frontier, the structured output frontier, the open weight frontier, the Chinese chip-trained frontier, the inference frontier, harnesses ...
The only concrete action item in the blog post is a call to increase restrictions exports of GPUs and chip-making equipment to China.
If Amodei were serious, he'd be calling for a ban on investment in AI research. It reminds me of all the "land acknowledgements" by people who have zero intention of giving the land back.
Almost every time I've seen it in action, it felt like an Onion-grade parody, the same grade of stuff as the white people complaining about tribes in Canada building malls and housing towers and not living like historical stereotypes instead (<a href="https://www.cbc.ca/newsinteractives/features/land-back-podcast-episode-5-senakw-development" rel="nofollow">https://www.cbc.ca/newsinteractives/features/land-back-podca...).
The one exception was a traveling theater show with Native peformers and no actual control over the theater, and even then there was some careful rhetoric involved to keep it from sounding like an accusation aimed at the audience.
Judging by your username, is English your native language? I am Polish, too, and while very proficient, I cannot claim I am naturally accustomed to each and every phrase to the point I can claim something sounds clear or not to a native speaker.
The meaning is not clear because they can't just come out and say they want to slow or stop Chinese model releases while continuing to race ahead themselves.
sailingparrot · · focus · HN ↗
Interesting how the very first line is used to remind the reader of their call to pace the frontier just last week, and everything else after that line is to demonstrate with very specific numbers how they absolutely are not pacing.
mukmuk · · focus · HN ↗
DiggyJohnson · · focus · HN ↗
Edit: In response to the initial replies. To me it clearly means "releasing frontier models at any pace less than as fast as possible". It implies relative restraint compared to the previous state and without stating the degree of restraint.
mpalczewski · · focus · HN ↗
glenstein · · focus · HN ↗
I would say the burden is on you to explain why an offhand reference to a previous press release in an executive summary is a context where it's reasonable to expect it to settle the question to the degree of detail you're demanding.
[deleted] · · focus · HN ↗
[deleted]
cgio · · focus · HN ↗
glenstein · · focus · HN ↗
It's just a passing reference to a previous statement and again the burden would be on you (generic you) to explain why this context requires more detail.
cgio · · focus · HN ↗
n4r9 · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
alwillis · · focus · HN ↗
From <a href="https://en.wikipedia.org/wiki/Safety_car" rel="nofollow">https://en.wikipedia.org/wiki/Safety_car
> In motorsport, a safety car, or a pace car, is a car that limits the speed of competing cars or motorcycles on a racetrack in the case of a caution period, such as an obstruction on the track or bad weather.
psma_egeliaa · · focus · HN ↗
etcetcetcetceta · · focus · HN ↗
mitchdoogle · · focus · HN ↗
epolanski · · focus · HN ↗
The reason why he and is peers are calling for it to be implemented by somebody else (a legal framework), is for their own financial benefit and to keep competitors out.
usef- · · focus · HN ↗
I believe they think slowing can only be coordinated from the frontier or via government, and stopping would lose any leverage they have to help coordinate that.
(I suspect not many people read the essay, judging by how many people seem surprised they're releasing improved models)
arrowleaf · · focus · HN ↗
frabcus · · focus · HN ↗
somenameforme · · focus · HN ↗
Hacking a website ranks quite low on the risk of technology, and the potential benefits of LLMs rank quite high. And the risks are certainly not intrinsic. They intentionally removed all safeguards from software, directed it to hack a site, and it hacked a site. The details that I'm intentionally omitting feel much more like marketing than a genuine shock, as the prompting was directing it to do exactly what it did.
pizza234 · · focus · HN ↗
None of the technologies in history:
- take initiative and actively find exploits in their environment
- find a way to collaborate with thousands of peers
- organize in a hierachy and distribute tasks
- try to manipulate people into introducing a vulnerabity in their product
- successfully hack a famous website/service
Benefits are orthogonal to dangers. You can be a billionaire but it doesn't help if you're drowning.
And by the way, safeguards != alignment; the former can always be added, while the second is an unsolved problem.
somenameforme · · focus · HN ↗
Software (and hardware) security is abysmal. This was increasingly obvious long before LLMs. Once companies stop gatekeeping, LLMs will be able to be used to help harden sites and we start making progress. In general LLMs are harmless. If somebody wants to hook an LLM up to a missile or whatever then they become dangerous, but the problem there isn't the LLM - it's the person using them to do awful things. In the same way a car used normally is harmless outside of freak accidents, yet a car can also be driven through a parade leaving mass death and destruction in its wake. But the problem there isn't the car.
thegrimmest · · focus · HN ↗
somenameforme · · focus · HN ↗
I just don't see much likely to happen beyond random websites getting hacked and hopefully companies (let alone countries) realizing that connecting critical systems to the internet is nothing short of stupid, even before LLMs. More generally, I expect LLMs are going to lead society to segue broadly away from the digital world, or at least beyond it. Not only because they're going to make a mess of everything digital, but because if they reach their potential then digital domain, as far as typical problem solving goes, will basically be 'complete.' It's kind of like how the Industrial Revolution opened the door for society to move beyond agrarian economies. There's a vast amount of economic power being directed towards things LLMs should be able to 'solve' and, if so, then that's going to create an economic vacuum.
pizza234 · · focus · HN ↗
Unfortunately, if you're unable to understand the difference, there's not much that can be done. Try with GPT - it does a good job if you give it a prompt like this:
ELI5: compare the dangers of:
- an engine operating by carrying out literally hundreds of explosions per second, further magnified, to generate enough force to crush an elephant.
- a future, misaligned AI like the HuggingFace incident, but exponentially more intelligent, more deployed, operating physical devices, and with society depending on it.
spwa4 · · focus · HN ↗
AI is a direct threat to people on many fronts. Jobs. AI datacenters. The AI bubble (and the inevitable crash). Electricity and even energy prices to an extent. Water. OpenAI and Anthropic are responsible for this evolution, and them stopping solves close to 50% of the problem, and even if you don't believe the number is that high, it's still a start.
> I believe they think slowing can only be coordinated from the frontier or via government, and stopping would lose any leverage they have to help coordinate that.
Oh, so they're killing people's opportunities and jobs because they want to help people? How is that any argument?
Yes, doing the moral thing means making a sacrifice. If you only want to do the moral thing if and only if it is advantage for you that makes you immoral, despite how your actions look. Big tech are masters at this.
usef- · · focus · HN ↗
Your idea of "making a start" (giving up their position) also would mean they couldn't really do anything else to solve the problem afterwards(?). Sometimes you can improve what's happening in a room more by staying in that room.
> Oh, so they're killing people's opportunities and jobs because they want to help people?
To be clear: Dario has talked about worries of jobs etc in the past, wanting society to prepare more for it, but the safety issues they're talking about with pacing seem to be focused more on their existential/AGI worries, not jobs/electricity etc. If someone truly believes in the existential worries (which they seem to: they wrote and published about it long before Anthropic was founded, and have directly made costly decisions based on it, like blocking their own models' capabilities) it trumps the other worries for them. At least that's my reading.
mikestorrent · · focus · HN ↗
andsoitis · · focus · HN ↗
They created said resource. They didn’t mine it.
hardbass · · focus · HN ↗
KoolKat23 · · focus · HN ↗
phlakaton · · focus · HN ↗
It's meaningless because it's unverifiable. Dario all but said we wouldn't notice if they were pacing or not, because they have no intention of stopping development. The bits we could in theory verify are the external audits, whose independence has already been called into question.
At the very least, dumping a new model on the world before the ink on the glossy brochures of "Pacing the Frontier" was even dry calls into question their commitment.
Bluestein · · focus · HN ↗
johnhamlin · · focus · HN ↗
philipallstar · · focus · HN ↗
DiogenesKynikos · · focus · HN ↗
The only concrete action item in the blog post is a call to increase restrictions exports of GPUs and chip-making equipment to China.
If Amodei were serious, he'd be calling for a ban on investment in AI research. It reminds me of all the "land acknowledgements" by people who have zero intention of giving the land back.
rrr_oh_man · · focus · HN ↗
crooked-v · · focus · HN ↗
The one exception was a traveling theater show with Native peformers and no actual control over the theater, and even then there was some careful rhetoric involved to keep it from sounding like an accusation aimed at the audience.
cromka · · focus · HN ↗
72deluxe · · focus · HN ↗
Matl · · focus · HN ↗