Yeah, thank god those models have arrived. No thanks to Anthropic and their obnoxious gatekeeping, of course. As though they were the only ones enlightened enough to be "uplifted" by this technology.
Now we can actually use this stuff to improve our own security. Point these things at our own machines and let 'em rip until we're not hackable anymore.
I wanted to pay Anthropic to do this but I couldn't. I wasn't in the super special corporation list. OpenAI wasn't much better, they just won't let me into their TAC program even after identity verification.
Thank god the chinese are out there undermining these US companies.
General intelligence has very different adversarial dynamics compared to nuclear weapons.
Explained further here:
<a href="https://news.ycombinator.com/item?id=49094348">https://news.ycombinator.com/item?id=49094348
Though there is further nuance in agentic vs non-agentic AI:
<a href="https://news.ycombinator.com/item?id=49736149">https://news.ycombinator.com/item?id=49736149
I'm legitimately interested in this argument, but simply stating that good vs evil are in equilibrium is nowhere near enough.
> There will be new adversarial games when everyone's smarter (bio/cyber offense/defense), but those games are always symmetrical in the long run
The continuation of the sentence you quoted is "and there will also be new cooperative strategies."
This is not faith, but an observation of Earth's past. If it were the case that intelligence is itself harmful, then it should not have evolved in so many species.
And even if we look at only humans, we do not see dumber populations being more prosperous. One could argue about the definition of "dumber" and "prosperous", but it is at least true in my own judgement of value, which I suspect is not too far from the average in modern society. One could also argue that bio/cyber adversarial games are qualitatively different from all past adversarial games, but you could argue the same thing for other qualitatively different games that emerged in the past, when they were new. But we're still here. So this argument hinges on burden of proof, and I think that's on Anthropic.
Also, as mentioned in that first linked comment, centralizing AI development increases a different kind of speciation risk: a small group of superintelligent humans, likely the ones currently running the top AI companies, splitting from the rest of humanity.
No one I've ever heard/read is claiming intelligence itself is harmful. You should delete that idea from your brain and stop responding to it.
> One could also argue that bio/cyber adversarial games are qualitatively different from all past adversarial games
No, one doesn't need to argue that they're different from all past adversarial games. You're a victim of survivorship bias. The games that people like to play and allow to be played are ones that equilibriate. That does not mean that all adversarial games have this feature.
There are an infinite number of games that do not equilibriate. It is incumbent upon you to demonstrate why the AI arms race is one that does.
they're quite literally trying to create a techno-priest class who are the only ones with access to the hidden knowledge of salvation (ie generation)
similar to priest classes, they warn of impending, world-consuming doom, talk up how they are uniquely positioned to interpret the sacred text (ie create models), while casting aspersions on heretics who offer a similar mode of salvation but whom they describe as being morally and ethically bankrupt (ie GLM lacks safeguards!)
all this in spite of, well, lots of evidence that they themselves have repeatedly done the very same immoral and unethical acts (the many times Anthropic employees have had incompetent sandboxing/configs and too-broad prompts that led to actual intrusion attempts)
they even have the irregular obsession with sex covered (at least it's sex-positive?). the only thing they're missing is an outfit though I guess there is this: <a href="https://x.com/Aella_Girl/status/2063798788310118655" rel="nofollow">https://x.com/Aella_Girl/status/2063798788310118655
It is garbage. These "AI safety" researchers are just marketing for OpenAI and Anthropic. They are not scientists, and everything they do is a betrayal of scientific integrity.
Ah yeah, lesswrong. The "human biodiversity", crypto scam, and polycule murder cult people. Why anyone would take those clowns seriously is a mystery to me.
The only people who seem to think that a world where only Anthropic and OpenAI can act as anointed gatekeepers to the most powerful models as the only rational path forward happen to work for these companies.
The entire world should not allow them to entrench themselves and build a business strategy around this and if open weight models and democratized access to the computing power to run them means these companies can’t exist then so be it.
I would rather watch the economy fall into a deep recession and hurt everyone to spare the entire world from this dystopian future.
Sorry Dario. You and your ilk don’t speak for humanity. Go cry on LessWrong if you feel so inclined, but people like these are the last people I would want yielding this power.
The best situation for us is the one where they exhaust themselves in unending competition. Neither the US nor China should ever be allowed to win, nor should OpenAI or Anthropic or any other individual corporation.
The second any one of them wins, oppression the likes of which we cannot even imagine will follow.
This is why this tech should not be left to the mercy of only a handful of labs. The only reasonable path forward is to leave it to the FOSS community. There will be big players of course, but the idea of locking this technology behind IP law and structural monopolies is untenable.
This also has the added benefit of the community pushing towards optimizations that increase efficiency and reduces the need for dedicated datacenters tasked with performing operations that doesn’t require it.
matheusmoreira · · focus · HN ↗
Now we can actually use this stuff to improve our own security. Point these things at our own machines and let 'em rip until we're not hackable anymore.
I wanted to pay Anthropic to do this but I couldn't. I wasn't in the super special corporation list. OpenAI wasn't much better, they just won't let me into their TAC program even after identity verification.
Thank god the chinese are out there undermining these US companies.
adev_ · · focus · HN ↗
+100.
Thanks God. these open weight models exist.
And the fact Anthropic is currently trying lobby against these models is despicable.
There is no scenario where putting the key of cybersecurity in the hands of few chosen ones is even remotely acceptable.
No government, no company, no entity should have this power.
Soon or later it will be abused (By 3 letter agency or by an insider/leak).
Delayed disclosure is dead already.
So just give the same capabilities to everybody and stop to fuck around.
estearum · · focus · HN ↗
[dead]
matheusmoreira · · focus · HN ↗
estearum · · focus · HN ↗
[dead]
matheusmoreira · · focus · HN ↗
estearum · · focus · HN ↗
txrx0000 · · focus · HN ↗
Explained further here: <a href="https://news.ycombinator.com/item?id=49094348">https://news.ycombinator.com/item?id=49094348
Though there is further nuance in agentic vs non-agentic AI: <a href="https://news.ycombinator.com/item?id=49736149">https://news.ycombinator.com/item?id=49736149
estearum · · focus · HN ↗
> There will be new adversarial games when everyone's smarter (bio/cyber offense/defense), but those games are always symmetrical in the long run
This is an article of faith, not an argument.
txrx0000 · · focus · HN ↗
This is not faith, but an observation of Earth's past. If it were the case that intelligence is itself harmful, then it should not have evolved in so many species.
And even if we look at only humans, we do not see dumber populations being more prosperous. One could argue about the definition of "dumber" and "prosperous", but it is at least true in my own judgement of value, which I suspect is not too far from the average in modern society. One could also argue that bio/cyber adversarial games are qualitatively different from all past adversarial games, but you could argue the same thing for other qualitatively different games that emerged in the past, when they were new. But we're still here. So this argument hinges on burden of proof, and I think that's on Anthropic.
Also, as mentioned in that first linked comment, centralizing AI development increases a different kind of speciation risk: a small group of superintelligent humans, likely the ones currently running the top AI companies, splitting from the rest of humanity.
estearum · · focus · HN ↗
> One could also argue that bio/cyber adversarial games are qualitatively different from all past adversarial games
No, one doesn't need to argue that they're different from all past adversarial games. You're a victim of survivorship bias. The games that people like to play and allow to be played are ones that equilibriate. That does not mean that all adversarial games have this feature.
There are an infinite number of games that do not equilibriate. It is incumbent upon you to demonstrate why the AI arms race is one that does.
paimapi · · focus · HN ↗
similar to priest classes, they warn of impending, world-consuming doom, talk up how they are uniquely positioned to interpret the sacred text (ie create models), while casting aspersions on heretics who offer a similar mode of salvation but whom they describe as being morally and ethically bankrupt (ie GLM lacks safeguards!)
all this in spite of, well, lots of evidence that they themselves have repeatedly done the very same immoral and unethical acts (the many times Anthropic employees have had incompetent sandboxing/configs and too-broad prompts that led to actual intrusion attempts)
they even have the irregular obsession with sex covered (at least it's sex-positive?). the only thing they're missing is an outfit though I guess there is this: <a href="https://x.com/Aella_Girl/status/2063798788310118655" rel="nofollow">https://x.com/Aella_Girl/status/2063798788310118655
gAI · · focus · HN ↗
Well, kinda thanks to Anthropic, what with the distillations.
nbjkln · · focus · HN ↗
gAI · · focus · HN ↗
<a href="https://www.lesswrong.com/posts/Jc9YZEmqHgocAKiaH/does-distilling-claude-carry-the-persona-with-it" rel="nofollow">https://www.lesswrong.com/posts/Jc9YZEmqHgocAKiaH/does-disti...
nbjkln · · focus · HN ↗
okasaki · · focus · HN ↗
sschueller · · focus · HN ↗
gAI · · focus · HN ↗
Quibblingeek · · focus · HN ↗
The entire world should not allow them to entrench themselves and build a business strategy around this and if open weight models and democratized access to the computing power to run them means these companies can’t exist then so be it.
I would rather watch the economy fall into a deep recession and hurt everyone to spare the entire world from this dystopian future.
Sorry Dario. You and your ilk don’t speak for humanity. Go cry on LessWrong if you feel so inclined, but people like these are the last people I would want yielding this power.
matheusmoreira · · focus · HN ↗
The second any one of them wins, oppression the likes of which we cannot even imagine will follow.
Quibblingeek · · focus · HN ↗
This also has the added benefit of the community pushing towards optimizations that increase efficiency and reduces the need for dedicated datacenters tasked with performing operations that doesn’t require it.