Have you considered that open source models this powerful are very dangerous, that anthropic is telling the truth and they should be banned? It seems obvious. There is no effective way to disable these dangerous capabilities.
Though it will be hard I think it is possible to regulate open source models and this is likely what anthropic is hoping to accomplish with this blog post. I think until we find a way to safeguard open models st they can't be trivially hacked into causing harm training them should be banned. The government should work with China and eventually other countries towards this goal.
Models weaker than Astra level capabilities are fine.
Also I should add I believe they will be regulated, it's a question of whether that happens before or after a catastrophe caused by an open model.
I think you misunderstand what the danger is. You cannot create any safeguards that prevent an LLM from discovering exploits in software. Exploits and undefined behavior are inherent to the process of using a programming language; it's not possible to debug or read source code without also auditing it for exploits and unexpected behavior.
The danger is software vulnerability. When asymmetric war breaks out between the US and China, it won't matter who signed what. Both sides will immediately use uncensored AI models to attack civilian infrastructure (not just government assets), and whichever side has the more-vulnerable internet will lose. You cannot bolster America's cyber defenses while depriving consumers of capable cybersecurity models. What OpenAI and Anthropic are lobbying for is the right to tax US businesses for having poor security. China's models threaten their ability to profit off of the scaremongering that they market. It is dangerous for American privacy and security, but great for OpenAI and Anthropic's B2B contracts.
EmbarrassedHelp · · focus · HN ↗
mysterEFrank · · focus · HN ↗
JohnMakin · · focus · HN ↗
mysterEFrank · · focus · HN ↗
mysterEFrank · · focus · HN ↗
bigyabai · · focus · HN ↗
The danger is software vulnerability. When asymmetric war breaks out between the US and China, it won't matter who signed what. Both sides will immediately use uncensored AI models to attack civilian infrastructure (not just government assets), and whichever side has the more-vulnerable internet will lose. You cannot bolster America's cyber defenses while depriving consumers of capable cybersecurity models. What OpenAI and Anthropic are lobbying for is the right to tax US businesses for having poor security. China's models threaten their ability to profit off of the scaremongering that they market. It is dangerous for American privacy and security, but great for OpenAI and Anthropic's B2B contracts.
jetpks · · focus · HN ↗
mysterEFrank · · focus · HN ↗