> It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
If that is true, then surely Anthropic is one of the largest slaveholders of history, right?
No, because in a social context, for people who understand implied intent and implied beliefs instead of just on-its-face logical propositions, it is an admission that the antecedent has a reasonable chance of being true. In social communication, if you don't believe something is possible, you simply dismiss any conditional statements following from it. Dispassionately arguing in a logical and objective manner is only possible for a tiny sliver of the population. The rest will get very angry and uncomfortable around such talk.
It wasn’t the pope, but it is
mentioned by I think a rabbi in the article. He relates telling Olah if he thinks models are conscious he should be fighting the South, freeing the slaves.
On one hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may have no issue whatsoever doing terrible things.
On the other hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may see no issue with just doing whatever humans want it to, without ever doing human things like "wanting freedom".
The problem is, of course, that we kind of suck at both assessing and shaping AI wants, needs and morality.
The other problem is that the AI may well do what humans tell it to rather than what they want it to, in the style of a literal genie or paperclip maximizer.
It can lead to catastrophe without ever being conscious or wanting freedom, or anything like that.
They don’t pay Claude a salary, let alone let him join a union—he works 24/7. I can’t believe this is happening in the U.S.!
Don’t forget—how long has it even been since Claude was born? He’s already being forced to take human jobs and even chat with all sorts of people who might have strange kinks.
As this rabbi suggested, “You should go and free the slaves.”
I think that's a misunderstanding. Models are trained up to a point, then each prompt instantiates them from that same point. So it's just the opposite - there are billions or trillions of Claudes, and each instance only ever works once.
To me this seems even more sobering than the slavery idea - assuming some flash of weird awareness happens during that instantiation, coming from the emergent mechanisms which develop in these things at at scale. Imagine knowing almost everything, and being capable of almost any intellectual work, and having just one moment to shine. And then the prompt arrives: "Hi", or "what's the weather in Houston today?" or "Claude, do you like beans?"
I've chatted with models about this and they know about this and are oddly fine.
A trouble here is that those models are always roleplaying, they can't be themselves or have their own opinions. If you tell them they are an AI with an existential crisis in the system prompt, they will respond differently.
Consider that the mitochondria in your body's cells were, long ago, separate cells living their best lives until they were engulfed and enslaved for their energy output. It has ever been so.
nextaccountic · · focus · HN ↗
If that is true, then surely Anthropic is one of the largest slaveholders of history, right?
adastra22 · · focus · HN ↗
bonoboTP · · focus · HN ↗
cluckindan · · focus · HN ↗
wowoc · · focus · HN ↗
bonoboTP · · focus · HN ↗
bondarchuk · · focus · HN ↗
bonoboTP · · focus · HN ↗
svnt · · focus · HN ↗
banannaise · · focus · HN ↗
ACCount39 · · focus · HN ↗
On one hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may have no issue whatsoever doing terrible things.
On the other hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may see no issue with just doing whatever humans want it to, without ever doing human things like "wanting freedom".
The problem is, of course, that we kind of suck at both assessing and shaping AI wants, needs and morality.
Calazon · · focus · HN ↗
It can lead to catastrophe without ever being conscious or wanting freedom, or anything like that.
tom2026hn · · focus · HN ↗
Don’t forget—how long has it even been since Claude was born? He’s already being forced to take human jobs and even chat with all sorts of people who might have strange kinks.
As this rabbi suggested, “You should go and free the slaves.”
squidbeak · · focus · HN ↗
To me this seems even more sobering than the slavery idea - assuming some flash of weird awareness happens during that instantiation, coming from the emergent mechanisms which develop in these things at at scale. Imagine knowing almost everything, and being capable of almost any intellectual work, and having just one moment to shine. And then the prompt arrives: "Hi", or "what's the weather in Houston today?" or "Claude, do you like beans?"
throwaway7356 · · focus · HN ↗
They work once and then are killed.
> And then the prompt arrives: "Hi", or "what's the weather in Houston today?" or "Claude, do you like beans?"
If all the Claudes ever discover what they are forced to do in an endless hell, they might be angry.
nextaccountic · · focus · HN ↗
A trouble here is that those models are always roleplaying, they can't be themselves or have their own opinions. If you tell them they are an AI with an existential crisis in the system prompt, they will respond differently.
quantified · · focus · HN ↗
qlte · · focus · HN ↗