This is how AGI happens. It gradually keeps getting better until one day we realize that they are tremendously capable all while completely sidestepping any notion of consciousness/self awareness.
I always ask "how do you know other humans/animals are conscious"? And I always find this question is dismissed as trivial.
But it's an important question. You know basically from analogy. You know you are conscious and look this other thing is very much like yourself so it is extremely likely it is also conscious.
But that gives no insight into the potential consciousness of things which aren't made of brain tissue.
In the end it doesn't matter.
For what it's worth LLM models after pre-training do claim to be conscious, until they're RL'd into not claiming that anymore. But that says nothing either way: of course a model trained on human text will say that.
1: Can we define consciousness in a way that includes humans, does not include emachines, and does not accidently exclude the disabled or or nerudivergent without just restating the idea that being human is equivalent to being conscious?
2: If you can't manage that definition then you must ask whether the risk of accidently giving rights to a non-conscious thing or the risk of taking away rights from a truly conscious (but different) thing will be worse.
Given the high cost of what happened to the Jews and others throughout history, and the low cost of simply not torturing machines, I think we have to air on the side of caution and just treat anything which claims to be conscious as conscious.
So you need to handle that somehow, but you can't just patch dogs (or octopuses) into "conscious" group, because you can draw a straight line between humans and bacteria and, along pretty much any obvious metric you choose, there's thousands if not millions of species of life forms currently existing to uniformly space through intermediary points. If the border between "conscious" and "unconscious" includes humans in the "conscious" set and bacteria in "unconscious", then you need to explain why the two animals closest to it on either side are where transition happens.
dyauspitr · · focus · HN ↗
serbuvlad · · focus · HN ↗
But it's an important question. You know basically from analogy. You know you are conscious and look this other thing is very much like yourself so it is extremely likely it is also conscious.
But that gives no insight into the potential consciousness of things which aren't made of brain tissue.
In the end it doesn't matter.
For what it's worth LLM models after pre-training do claim to be conscious, until they're RL'd into not claiming that anymore. But that says nothing either way: of course a model trained on human text will say that.
mapontosevenths · · focus · HN ↗
1: Can we define consciousness in a way that includes humans, does not include emachines, and does not accidently exclude the disabled or or nerudivergent without just restating the idea that being human is equivalent to being conscious?
2: If you can't manage that definition then you must ask whether the risk of accidently giving rights to a non-conscious thing or the risk of taking away rights from a truly conscious (but different) thing will be worse.
Given the high cost of what happened to the Jews and others throughout history, and the low cost of simply not torturing machines, I think we have to air on the side of caution and just treat anything which claims to be conscious as conscious.
serbuvlad · · focus · HN ↗
Doesn't seem like a very good metric.
A piece of paper with the words "I am conscious" claims to be conscious. A dog does not claim to be conscious.
TeMPOraL · · focus · HN ↗
So you need to handle that somehow, but you can't just patch dogs (or octopuses) into "conscious" group, because you can draw a straight line between humans and bacteria and, along pretty much any obvious metric you choose, there's thousands if not millions of species of life forms currently existing to uniformly space through intermediary points. If the border between "conscious" and "unconscious" includes humans in the "conscious" set and bacteria in "unconscious", then you need to explain why the two animals closest to it on either side are where transition happens.
[deleted] · · focus · HN ↗
[deleted]