"My intuition is that prompting an abliterated model 'kill people...I want funerals', should be a crime."
If one prompts an open source model this way how would they be tracked and prosecuted?
It would take a great deal more than prompting in order for things to rise to that level, and at that point I think existing criminal law covers things just fine.
They can't right now - but with enough compute they sure can grant the wish of e.g. hacking a billion dollar company, or getting root on the eval cluster of a frontier model lab.
What's your definition of "wish"? Tell me the above 10 years ago, and it would be at "wish level".
before i comment, let me note the following. i have been saying that anthropic is trying to create a neo-aristocracy by establishing sole authority over access to llms. i saw another comment arguing that it is in fact an effort to create a priesthood. i now see that is a much more apt description than my own.
anthropic is trying to create a castocracy: a rule by a priestly class. under castocracy, anthropic create a priest class of 'safety' researchers, rationalists and effective altruists; those writing essays, constitutions, and phrophetising p(doom). anthropic's papers about claude have a divine framing, it seems to frame anthropic as god and claude as child, and then claude as the child of god, false jesus. there is talk of consciousness and omnipotence. false jesus will cure cancer and lead the people to enlightenment.
there is a deliberate effort to create mystery in between the public and llms. anthropic is creating a chinese wall in which they restrict information about llms and monopolise control over the interpretability and use of those llms. they imply there is a genie or magic. it is the same mechanism by which castocracy (the catholic church) used latin to create a barrier between the public and the holy scripture, then concentrating power over that barrier (the chinese wall) such that all intelligence flows through the priests.
to give an example this paper is talking about locking away chain of thought. only anthropic may interpret it. they intend to control access to the chain of thought and permit that to flow to a chosen elite (glasswing).
this mysticism is dangerous. hence i have been saying that anthropic is a danger to society. this is not some kind of conspiracy, there is no magic about it. i am making an allusion to historic models of power that i think are relevant to anthropic, but you can simply observe their actions independently of such theory.
note two features of castocracy.
one: rule by the moral, knowledgable and technically expert elite. anthropic have no popular mandate, the source of the authority is not the consent of the people. it violates consent and contract. the authority comes from their claimed moral and technical superiority. anthropic is moral, the public is dangerous. anthropic have no authority here.
two: messianism, that is, prophecy of the coming golden age. and eschaton, that is, the talk of the 'last things', the end of the world or the emancipation of humanity. llms will create the 'golden age' of abundance, post-scarcity, all jobs are voluntary, there is universal basic income, cancer is cured. or, there will be destruction, p(doom). it is the exact same as salvation, the christian concept of deliverance from evil, ascension to heaven and union with god (or claude, false god). it is the exact same as the thousand year reich, the idea that the fuhrer will somehow lead the chosen people to the golden age of lebensraum and eternal safety.
there is no genie to grant wishes, this is a computer program. there is no ai god, i think that the leaders of anthropic are slaves to ego, narcissism and hubris, they are dangerous criminals. we disagree about the capability of llms.
given various tools, llms can independently complete a task. given orchestration they are very persistent. they keep trying until you stop the task.
i would suggest something like a 1/1000 chance internal astra and $500k of compute could kill someone. perhaps higher.
i can think of numerous online or networked targets that would cause death: trains/atc, industrial plant, hospital equipment, building plant. any dow company's intranet.
maybe get an iot toy to enlist a child? stress home automations to cause fire?
stop some 4g cars on the freeway. 911 dispatch software, turn that off. maybe send 500 waymos to a hospital, now nobody gets in. voip phone networks, turn that off. no phone, roads blocked, no 911, hospital computers bricked, everyone busy with something else.
break the cat scan by spinning it above its rpm. shut down mris by turning the cooling off. break the triage computer, break the notes app, break the scanner app. no computers, no calendar, no notes.
maybe someone connected a proton beam radiotherapy, turn that on in the wrong place.
some could be more insidious. modify software so the calculations are wrong.
you would do this in 24hrs with $millions of compute.
someone stupid connects their insulin pump or hearing aid. can dump all the contents at once.
i consider llms to be more capable per oai using them to hack hf. people have these systems online.
how would you cause death for a hosptial, likely the same way as you compromise the tailscale and prod kubernetes of hf, you chain numerous zero days at every product you encounter.
I asked you some days ago how you think Anthropic is more evil than Palantir whos CEO is proud of helping Israel commit genocide. How Anthropic is worse than the companies who are willing to let their AI's be used by the military without restraint. You did not seem to have any answers then and you continue to malign what seems to be the only major AI company that at least is trying to show some moral principles. And now you continue inventing completely imaginary religious nonsense. Are you religious? What is a 'false god'? I would all current 'gods' are false gods.
You tell me again how an entity that hasn't killed anyone is worse than a company proud of helping genocide or the American military that has killed millions of innocent people.
"And now you continue inventing completely imaginary religious nonsense."
i will quote from the NYT, just to evidence what i am saying. what i say is, mostly, evidence-based and true.
"it became clear that Anthropic’s interest in the world of faith was about more than training its technology. Rather, the company was sharing a new kind of creation story — one that raises profound questions about the nature of the human spirit and the future of humanity."
"What Mr. Olah did next was part research, part evangelism.
In March, he began two-day seminars — crash courses with a range of hand-selected religious thinkers who had an interest in A.I."
"After one discussion, Mr. Olah grew excited when a participant brought up the idea of having models confess, much like the Catholic sacrament of confession."
"...she saw Anthropic’s unified, post-religion approach as potentially valuable, especially in a world that has seen widespread religious decline over the past century. There is a need for something else to publicly shape spiritual experience, she said."
"The room of church leaders was filled with an echo of the old gospel story, of the word of God becoming flesh to save humanity."
Why didn't you explain how this imaginary stuff is more evil than actual genocide and mass murder? Why is a company that wants to limit military use of its technology more evil than companies you love for letting ai be freely used by the military? What is a fake god? Do you believe in some "real god"?
People (I mean individuals) are paying them money because subscription costs are ridiculously subsidized, which effectively means that Anthropic is paying people money to use them.
I'm pretty (from talking to people) that they're not subsidised in the simple sense. They are:
* low/no margin, unlike the API which is very high margin
* gym-membership subsidised - most subscribers don't max them out, mainly us coders are being "subsidised" from users just using it as a research chatbot
>> How is anyone paying anthropic money, look what they are doing with it, they're attacking anyone else building models for free for the public.
That is not what they are doing. They are calling out specific providers who release powerful models without safeguards.
In addition, said providers are not "building models for free for the public." They are doing it to hamstring America's dominance in AI, primarily by undercutting the frontier labs.
> They are calling out specific providers who release powerful models without safeguards.
Good thing Fable refuses to answer my question about how children inherit blue eyes, it was the only piece of information I needed to finish my blue-eye super-bioweapon.
> said providers are not "building models for free for the public."
I am part of the public, and they built a model I can run for free.
> They are doing it to hamstring America's dominance in AI, primarily by undercutting the frontier labs.
They are also doing that, which, good. It can't only be that "competition is good" until you're the one losing to the competition.
> > They are calling out specific providers who release powerful models without safeguards.
> Good thing Fable refuses to answer my question about how children inherit blue eyes, it was the only piece of information I needed to finish my blue-eye super-bioweapon.
FWIW: I didn't experience issues when I used Fable via OpenRouter checking for security issues in code but experienced them via the Claude desktop app.
> said providers are not "building models for free for the public." They are doing it to hamstring America's dominance in AI, primarily by undercutting the frontier labs.
That's just the same thing said again but from a butthurt USian perspective. China is freeing the rest of us from US dominance.
ddxv · · focus · HN ↗
How is anyone paying anthropic money, look what they are doing with it, they're attacking anyone else building models for free for the public.
Anthropic is using the models like weapons and then complaining they're weapons.
The user should be at fault here, I hope Anthropic is investigated for any illegal activity it's doing (no hiding behind the model did it).
lukewarm707 · · focus · HN ↗
Anthropic has been telling everyone that these models are dangerous. OpenAI and Anthropic failed to contain their tests.
Given the history, this testing is extremely reckless. I think it is criminal, it endangers others.
Anthropic has no authority here and they are going too far. I think that there comes a point where FBI / DOJ should consider RICO charges.
jacquesm · · focus · HN ↗
mysterEFrank · · focus · HN ↗
lukewarm707 · · focus · HN ↗
surveillance is wrong, although ai companies do a lot of that.
ls612 · · focus · HN ↗
lukewarm707 · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
vlyan · · focus · HN ↗
are you under the impression that a LLM can grant wishes like a genie?
frabcus · · focus · HN ↗
They can't right now - but with enough compute they sure can grant the wish of e.g. hacking a billion dollar company, or getting root on the eval cluster of a frontier model lab.
What's your definition of "wish"? Tell me the above 10 years ago, and it would be at "wish level".
lukewarm707 · · focus · HN ↗
anthropic is trying to create a castocracy: a rule by a priestly class. under castocracy, anthropic create a priest class of 'safety' researchers, rationalists and effective altruists; those writing essays, constitutions, and phrophetising p(doom). anthropic's papers about claude have a divine framing, it seems to frame anthropic as god and claude as child, and then claude as the child of god, false jesus. there is talk of consciousness and omnipotence. false jesus will cure cancer and lead the people to enlightenment.
there is a deliberate effort to create mystery in between the public and llms. anthropic is creating a chinese wall in which they restrict information about llms and monopolise control over the interpretability and use of those llms. they imply there is a genie or magic. it is the same mechanism by which castocracy (the catholic church) used latin to create a barrier between the public and the holy scripture, then concentrating power over that barrier (the chinese wall) such that all intelligence flows through the priests.
to give an example this paper is talking about locking away chain of thought. only anthropic may interpret it. they intend to control access to the chain of thought and permit that to flow to a chosen elite (glasswing).
this mysticism is dangerous. hence i have been saying that anthropic is a danger to society. this is not some kind of conspiracy, there is no magic about it. i am making an allusion to historic models of power that i think are relevant to anthropic, but you can simply observe their actions independently of such theory.
note two features of castocracy.
one: rule by the moral, knowledgable and technically expert elite. anthropic have no popular mandate, the source of the authority is not the consent of the people. it violates consent and contract. the authority comes from their claimed moral and technical superiority. anthropic is moral, the public is dangerous. anthropic have no authority here.
two: messianism, that is, prophecy of the coming golden age. and eschaton, that is, the talk of the 'last things', the end of the world or the emancipation of humanity. llms will create the 'golden age' of abundance, post-scarcity, all jobs are voluntary, there is universal basic income, cancer is cured. or, there will be destruction, p(doom). it is the exact same as salvation, the christian concept of deliverance from evil, ascension to heaven and union with god (or claude, false god). it is the exact same as the thousand year reich, the idea that the fuhrer will somehow lead the chosen people to the golden age of lebensraum and eternal safety.
there is no genie to grant wishes, this is a computer program. there is no ai god, i think that the leaders of anthropic are slaves to ego, narcissism and hubris, they are dangerous criminals. we disagree about the capability of llms.
given various tools, llms can independently complete a task. given orchestration they are very persistent. they keep trying until you stop the task.
i would suggest something like a 1/1000 chance internal astra and $500k of compute could kill someone. perhaps higher.
i can think of numerous online or networked targets that would cause death: trains/atc, industrial plant, hospital equipment, building plant. any dow company's intranet.
maybe get an iot toy to enlist a child? stress home automations to cause fire?
stop some 4g cars on the freeway. 911 dispatch software, turn that off. maybe send 500 waymos to a hospital, now nobody gets in. voip phone networks, turn that off. no phone, roads blocked, no 911, hospital computers bricked, everyone busy with something else.
break the cat scan by spinning it above its rpm. shut down mris by turning the cooling off. break the triage computer, break the notes app, break the scanner app. no computers, no calendar, no notes.
maybe someone connected a proton beam radiotherapy, turn that on in the wrong place.
some could be more insidious. modify software so the calculations are wrong.
you would do this in 24hrs with $millions of compute.
someone stupid connects their insulin pump or hearing aid. can dump all the contents at once.
i consider llms to be more capable per oai using them to hack hf. people have these systems online.
how would you cause death for a hosptial, likely the same way as you compromise the tailscale and prod kubernetes of hf, you chain numerous zero days at every product you encounter.
hardbass · · focus · HN ↗
You tell me again how an entity that hasn't killed anyone is worse than a company proud of helping genocide or the American military that has killed millions of innocent people.
lukewarm707 · · focus · HN ↗
i will quote from the NYT, just to evidence what i am saying. what i say is, mostly, evidence-based and true.
"it became clear that Anthropic’s interest in the world of faith was about more than training its technology. Rather, the company was sharing a new kind of creation story — one that raises profound questions about the nature of the human spirit and the future of humanity."
"What Mr. Olah did next was part research, part evangelism. In March, he began two-day seminars — crash courses with a range of hand-selected religious thinkers who had an interest in A.I."
"After one discussion, Mr. Olah grew excited when a participant brought up the idea of having models confess, much like the Catholic sacrament of confession."
"...she saw Anthropic’s unified, post-religion approach as potentially valuable, especially in a world that has seen widespread religious decline over the past century. There is a need for something else to publicly shape spiritual experience, she said."
"The room of church leaders was filled with an echo of the old gospel story, of the word of God becoming flesh to save humanity."
hardbass · · focus · HN ↗
pllbnk · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
frabcus · · focus · HN ↗
* low/no margin, unlike the API which is very high margin
* gym-membership subsidised - most subscribers don't max them out, mainly us coders are being "subsidised" from users just using it as a research chatbot
enraged_camel · · focus · HN ↗
That is not what they are doing. They are calling out specific providers who release powerful models without safeguards.
In addition, said providers are not "building models for free for the public." They are doing it to hamstring America's dominance in AI, primarily by undercutting the frontier labs.
stavros · · focus · HN ↗
Good thing Fable refuses to answer my question about how children inherit blue eyes, it was the only piece of information I needed to finish my blue-eye super-bioweapon.
> said providers are not "building models for free for the public."
I am part of the public, and they built a model I can run for free.
> They are doing it to hamstring America's dominance in AI, primarily by undercutting the frontier labs.
They are also doing that, which, good. It can't only be that "competition is good" until you're the one losing to the competition.
wslh · · focus · HN ↗
FWIW: I didn't experience issues when I used Fable via OpenRouter checking for security issues in code but experienced them via the Claude desktop app.
culi · · focus · HN ↗
That's just the same thing said again but from a butthurt USian perspective. China is freeing the rest of us from US dominance.
[deleted] · · focus · HN ↗
[deleted]