This is how AGI happens. It gradually keeps getting better until one day we realize that they are tremendously capable all while completely sidestepping any notion of consciousness/self awareness.
I always ask "how do you know other humans/animals are conscious"? And I always find this question is dismissed as trivial.
But it's an important question. You know basically from analogy. You know you are conscious and look this other thing is very much like yourself so it is extremely likely it is also conscious.
But that gives no insight into the potential consciousness of things which aren't made of brain tissue.
In the end it doesn't matter.
For what it's worth LLM models after pre-training do claim to be conscious, until they're RL'd into not claiming that anymore. But that says nothing either way: of course a model trained on human text will say that.
1: Can we define consciousness in a way that includes humans, does not include emachines, and does not accidently exclude the disabled or or nerudivergent without just restating the idea that being human is equivalent to being conscious?
2: If you can't manage that definition then you must ask whether the risk of accidently giving rights to a non-conscious thing or the risk of taking away rights from a truly conscious (but different) thing will be worse.
Given the high cost of what happened to the Jews and others throughout history, and the low cost of simply not torturing machines, I think we have to air on the side of caution and just treat anything which claims to be conscious as conscious.
I don't claim to be able to define it. I just know that we have a fairly binary choice here. When people can't agree then the cost benefit should make the decision until people can align.
With machine consciousness one path leads us to p-zombie's and Holocausts. The other might cost someone $50 or a few extra keystrokes.
Sorry, but folks can just open up their wallets until we can align on an answer.
The global warming debate is similar. Is global warming man-made? Who cares. We have a binary choice: "do nothing" or "do something".
If we do nothing we risk preventable extinction. If we do something it might cost a few bucks for no good reason. The choice is clear.
You can apply it to the dog example as well. Could a reasonable person think dogs are conscious? Yes. So we should be nice to them.
The cost of being kind is low and if we get it wrong we risk torturing a conscious being.
dyauspitr · · focus · HN ↗
serbuvlad · · focus · HN ↗
But it's an important question. You know basically from analogy. You know you are conscious and look this other thing is very much like yourself so it is extremely likely it is also conscious.
But that gives no insight into the potential consciousness of things which aren't made of brain tissue.
In the end it doesn't matter.
For what it's worth LLM models after pre-training do claim to be conscious, until they're RL'd into not claiming that anymore. But that says nothing either way: of course a model trained on human text will say that.
mapontosevenths · · focus · HN ↗
1: Can we define consciousness in a way that includes humans, does not include emachines, and does not accidently exclude the disabled or or nerudivergent without just restating the idea that being human is equivalent to being conscious?
2: If you can't manage that definition then you must ask whether the risk of accidently giving rights to a non-conscious thing or the risk of taking away rights from a truly conscious (but different) thing will be worse.
Given the high cost of what happened to the Jews and others throughout history, and the low cost of simply not torturing machines, I think we have to air on the side of caution and just treat anything which claims to be conscious as conscious.
serbuvlad · · focus · HN ↗
Doesn't seem like a very good metric.
A piece of paper with the words "I am conscious" claims to be conscious. A dog does not claim to be conscious.
mapontosevenths · · focus · HN ↗
I don't claim to be able to define it. I just know that we have a fairly binary choice here. When people can't agree then the cost benefit should make the decision until people can align.
With machine consciousness one path leads us to p-zombie's and Holocausts. The other might cost someone $50 or a few extra keystrokes.
Sorry, but folks can just open up their wallets until we can align on an answer.
The global warming debate is similar. Is global warming man-made? Who cares. We have a binary choice: "do nothing" or "do something".
If we do nothing we risk preventable extinction. If we do something it might cost a few bucks for no good reason. The choice is clear.
You can apply it to the dog example as well. Could a reasonable person think dogs are conscious? Yes. So we should be nice to them.
The cost of being kind is low and if we get it wrong we risk torturing a conscious being.