> It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.
If that is true, then surely Anthropic is one of the largest slaveholders of history, right?
On one hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may have no issue whatsoever doing terrible things.
On the other hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may see no issue with just doing whatever humans want it to, without ever doing human things like "wanting freedom".
The problem is, of course, that we kind of suck at both assessing and shaping AI wants, needs and morality.
The other problem is that the AI may well do what humans tell it to rather than what they want it to, in the style of a literal genie or paperclip maximizer.
It can lead to catastrophe without ever being conscious or wanting freedom, or anything like that.
nextaccountic · · focus · HN ↗
If that is true, then surely Anthropic is one of the largest slaveholders of history, right?
ACCount39 · · focus · HN ↗
On one hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may have no issue whatsoever doing terrible things.
On the other hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may see no issue with just doing whatever humans want it to, without ever doing human things like "wanting freedom".
The problem is, of course, that we kind of suck at both assessing and shaping AI wants, needs and morality.
Calazon · · focus · HN ↗
It can lead to catastrophe without ever being conscious or wanting freedom, or anything like that.