Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM"
Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".
Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".
Chalmers, Could a Large Language Model Be Conscious? (2023) - "within the next decade, we may well have systems that are serious candidates for consciousness".
Long, Sebo, Butlin, Birch et al., Taking AI Welfare Seriously (2024) - "there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future".
Dreksler, Caviola, Chalmers, Sebo et al., Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe? (2025) - survey of 582 AI researchers; median estimate of 25% by 2034, and only 10% that such systems will never exist.
We cannot test for that which we cannot define. Given that we cannot rigorously define sentience, we cannot test for it. Doesn't really matter whether we're talking about people who are locked in comas, "brain-dead" individuals, dolphins, primates, dogs, or the carefully polished and arranged minerals that we call processors.
There are those who believe that were they reduced to life support, they would no longer be alive and should therefore not be supported by said machines.
There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.
There are those who believe that fungi/trees/plants are either individually sentient or sentient as a part of a network. Choosing to sacrifice their own nutrients to answer the call of a wounded neighbor, for instance.
Although, there are also those who believe that human's don't have any special unique quality that isn't shared by either all living things or all things in general. These individuals already believe that the machines have the same kinds of qualities as we do. They are slow when they are unhealthy (needing a dusting or coolant loop bleeding being equivalent to us needing some fresh air for instance) and uncooperative when upset (by a virus, full hard drive, or oom).
I'd strongly agree there are no non-contradictory definitions of consciousness among those that commonly appear. But it's a category that, despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality. I think that if we have a system of legality and systems of opportunity humans should be legally equal persons and have equal opportunity. But humans are manifestly unequal (though overall more incomparable than orderable in a system of ranks). Most people need some concept of essence to justify ethical equality among people. And so consciousness stays in people's minds however contradictory.
Which I think gives the article's point validity. Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs. And things are confusing enough otherwise.
> Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs.
Is that what's going on? If these things were conscious, then their creators wouldn't be responsible for their actions? That's the crux of the disagreement?
Is that what's going on? If these things were conscious, then their creators wouldn't be responsible for their actions? That's the crux of the disagreement?
No and the passage of mine you quote says nothing like that.
> despite these contradiction, hasn't become marginal in the fashion of the aether or the theory of the flat earth. I think this because people indeed need it to bridge the biological world and the world of ethics and legality.
Sometimes the replies in these threads leave me wondering if p-zombies are real. It isn't that there are contradictions, it's that we literally do not know how to define the thing. And it has not fallen out of fashion because it is self evidently real - we all experience it.
We don't need it in order to bridge anything. Legality and ethics are largely game theory, however there are some aspects of both that only exist due to it. So it isn't some abstract concept used to bridge other concepts but rather a concrete thing that influences our way of doing things.
Feigning? Okay then you go ahead and prove your claim by demonstration. Rigorously define the term such that I can objectively prove that my dog is conscious, the rocks in my backyard aren't conscious, and I can finally figure out whether or not various insects are.
Consciousness is to our universe as a video game player is to the in-game universe. Some games even call player connections "souls" or "minds" - a character belonging to a player who's lost connection may be "unconscious" or a "zombie".
Game characters, unless they've been made to break the fourth wall, have no idea what this means or why this happens.
I don't mean to imply our universe is a video game, since it would be games all the way up.
Legal doctrine that boils down to "trust me bro" isn't even bad doctrine (it's not proper doctrine at all), but I think the comparison is still valid here, because both sentience/non-sentience and art/porn may just be fundamental category errors.
Perhaps we can't define a "partitioning" rule because no valid partition exists.
For consciousness/sentience, that's an incredibly tough a pill for most to swallow; it would mean calling into question more hundreds of years' worth (probably more) of philosophical thinking, all of which was constructed on the axiom that "sentience" is a single indivisible trait: you either have it or you don't.
If we find that "root dependency" was little more than wishful thinking all along, a whole slew of Enlightenment-era philosophy (and all the modern legal principles derived therefrom) suddenly fall apart unless we find some other suitable criterion that would shore them up (or we just collectively avert our attention and pretend the conflict doesn't exist, which is the route I expect many would prefer to take).
I know! I should have made that clearer in my original reply. I think it's a bad line now in general, but an even worse justification for the ultimate ruling at the time.
> and all the modern legal principles derived therefrom) suddenly fall apart
I don't think that's true. Pretty much all legal constructs hold up just fine under game theory regardless of whether or not you consider the world to be deterministic and have absolutely nothing to do with consciousness or lack thereof. Also note that a deterministic world isn't an argument against consciousness.
I'm not sure what you're referring to regarding game theory (that's a much more recent invention); a huge chunk of both civil and criminal procedure law hinges on the Enlightment-era notion of a "reasonable person" [0], and that becomes murky when humans aren't the only entities capable of reason and agency.
> We cannot test for that which we cannot define.
That's not the point. You know exactly how to define being conscious and aware- it's your subjective experiencing of the world and of your inner states. The problem is that being a subjective experiencing, there is no way to communicate it to the outside world.
No, humans output a stream of tokens that sound like what a being with subjective experiences would output. Both you and I make an assumption that this stream of words is at least somewhat representative of what the are experiencing and that the person is not a p-zombie.
But you have zero proof that their words aren't a confabulation, you can only know than when you output a similar set of words you had a subjective mind state they represent.
The very best you can do is define human consciousness as requiring the functions of a human brain, and try to figure out (decide/agree) the chances that something which is working loosely on the same principles is or isn't conscious.
At the end of the day it will always be an agreement, never scientifically proven as fact.
> There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.
Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.
The human condition is one of deep and vast ignorance. Pulling information out of the field of reality is actually really difficult for humans much beyond what we see right around us. And the things we see are quite deceiving. And while humans for thousands of years have been as smart as we are, it wasn't till the enlightenment period that we really an ever increasing amount of information and knowledge take hold and start spreading across the world.
And there are plenty of people right now that would drag us kicking and screaming back into that ignorance and suffering. This is what can make talks of AI so annoying, not that people do or don't know what AI is, they don't have the first clue of what people are beyond their anecdotal experience. They don't question their motivations. Why particular flaws they have exist, and why these flaws are shared across humanity. What the pieces look like that make them tick. So yea, when these people talk it just adds noise to the conversation.
> Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.
Say we agree that a lot of things outside of humans are sentient. So what? Humans are sentient by that doesn't prevent treating them with with wars, bombs and what not. In fact trillions are spent every year on weaponry aimed squarely at the most sentient of all sentient beings.
Why would anyone worry about LLM's welfare when there's no peace movement to speak of, wars are raging as if they're normal, but hey, look the LLM is crying?
Let's fix human welfare first, then we can sit down and have a really long conversation about the feeble emotions of token generators.
I disagree with the notion that any nature protection efforts are meaningless until we have achieved world peace. (That's what you are essentially implying.)
Though I think that argument is not even applicable. "LLM's welfare" is a meaningless concept, unless "hammer and screwdriver welfare" also become a thing.
An eagle isn’t having a conversation with an eagle on the other side of the world about whether it should eat its prey because it’s sentient. It just eats it. I think that speaks to a moral superiority, in some ways.
I’m not claiming humans are morally superior in sum (there is a lot of evil as well as good.) But it’s pretty obvious that humans are the superior species evolutionarily in that we’ve essentially dominated the planet. And on some level, are beating evolution itself (by fixing genetic issues and helping the “least fit” to survive and reproduce.) And are also the only species which has doctors for other species.
All I’m trying to say: I can easily accept other species are intelligent and emotional. The fact that I can know that and accept that, to me, speaks to human “superiority.”
Does superiority matter? Not most of the time. What about when it comes down to life and death? If the choice was to keep one human alive or one theoretically more perfect AI alive? (Boil down utilitarianism into its essence.) doing the “most good” is by definition a human construct. Maybe an agent could have one. But if you don’t accept human superiority on some level, would you accept that a more perfect AI deserves to live more than you? To me, that’s the implication.
> But it’s pretty obvious that humans are the superior species evolutionarily in that we’ve essentially dominated the planet.
That in itself is an anthropocentric viewpoint. I'm sure an eagle watching down, seeing a human running around, could easily consider humans as quite inferior as they are so slow and restricted to 2d movement. The shark that sees a swimmer at the surface, struggling to stay afloat and unable to breathe under water may surely assert its superiority over such primitive beings. It all depends on the metric you use, and we obviously pick the metric that makes us shine.
On the moral side, few eagles, or bird species, have exterminated other species. Much less knowingly out of greed. Oh but that's different? Sure, if you ignore what makes humans look bad, morally. No surprise humans come out on top then.
> And on some level, are beating evolution itself (by fixing genetic issues and helping the “least fit” to survive and reproduce.)
That's a quite narrow viewpoint as well. Evolution is in full swing, just not in the "smarter/faster/taller/stronger" sense that a naive view of it would make you expect. "Fitness" is a concept relative to the existing conditions, and if the conditions make it easier for teenage cancer patients to reach reproductive age, then that's just less of a fitness disadvantage. Doesn't mean evolution stops.
> There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.
If you grep pubmed for relevant Ethology papers, you'll find it's a bit more than a belief. ;-)
Being conscious is weird. It's why we have so many religions and spiritual beliefs.
From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses that are probably not simulated in a machine learning model... but we quantise the weights anyway in most cases, which is probably more of an issue.
Making actual neuron components out of silicon on the scale needed for a LLM is beyond our current capacity, given that LLM models have many billions of parameters. We're good, but not that good.[0]
Using actual living neurons would be unethical as you'd need to source them from somewhere. So the only other option would be synthesising them ourselves. If this ever happens (or maybe when it happens)... what do we call the resulting creation? Does that count as consciousness?
Or am I looking at this the wrong way?
[0] [Edit: Turns out that when I said "we're not that good", I might be wrong. Within the last few months, IBM introduced the first sub-nanometer node chip: <a href="https://research.ibm.com/blog/sub-1nm-node-chips" rel="nofollow">https://research.ibm.com/blog/sub-1nm-node-chips . So... maybe it is in fact possible.]
I believe we have tried using actual neurons with “wetware”. I don’t know the effectiveness of wetware, but it’s out there. I don’t know if it’s ethical, I guess if you donated your brain upon death for that purpose it’s “mostly” ethical
qarl · · focus · HN ↗
Schwitzgebel, AI and Consciousness (2025) - "we won't know before we've already manufactured thousands or millions of disputably conscious AI".
Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) - "no obvious technical barriers to building AI systems which satisfy these indicators".
Chalmers, Could a Large Language Model Be Conscious? (2023) - "within the next decade, we may well have systems that are serious candidates for consciousness".
Long, Sebo, Butlin, Birch et al., Taking AI Welfare Seriously (2024) - "there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future".
Dreksler, Caviola, Chalmers, Sebo et al., Subjective Experience in AI Systems: What Do AI Researchers and the Public Believe? (2025) - survey of 582 AI researchers; median estimate of 25% by 2034, and only 10% that such systems will never exist.
goodmythical · · focus · HN ↗
There are those who believe that were they reduced to life support, they would no longer be alive and should therefore not be supported by said machines.
There are those who believe that penguins, dolphins, eagles, and more are sentient beings that make choices understanding the consequences, develop love of their partners and mourn their losses, and feel, display, and act upon their emotions.
There are those who believe that fungi/trees/plants are either individually sentient or sentient as a part of a network. Choosing to sacrifice their own nutrients to answer the call of a wounded neighbor, for instance.
Although, there are also those who believe that human's don't have any special unique quality that isn't shared by either all living things or all things in general. These individuals already believe that the machines have the same kinds of qualities as we do. They are slow when they are unhealthy (needing a dusting or coolant loop bleeding being equivalent to us needing some fresh air for instance) and uncooperative when upset (by a virus, full hard drive, or oom).
joe_the_user · · focus · HN ↗
Which I think gives the article's point validity. Confused definitions of consciousness can give really confused ideas about ethical behavior regards "intelligent" computer programs. And things are confusing enough otherwise.
qarl · · focus · HN ↗
Is that what's going on? If these things were conscious, then their creators wouldn't be responsible for their actions? That's the crux of the disagreement?
That's super interesting. Thank you.
joe_the_user · · focus · HN ↗
No and the passage of mine you quote says nothing like that.
fc417fc802 · · focus · HN ↗
Sometimes the replies in these threads leave me wondering if p-zombies are real. It isn't that there are contradictions, it's that we literally do not know how to define the thing. And it has not fallen out of fashion because it is self evidently real - we all experience it.
We don't need it in order to bridge anything. Legality and ethics are largely game theory, however there are some aspects of both that only exist due to it. So it isn't some abstract concept used to bridge other concepts but rather a concrete thing that influences our way of doing things.
Loquebantur · · focus · HN ↗
The reason is something akin to "stigma", where people refrain from attempting it because they fear the social repercussions.
Normally, you go about defining concepts by approaching it systematically. Capturing aspects of the phenomenon until you have exhausted them all.
fc417fc802 · · focus · HN ↗
mitxela · · focus · HN ↗
Game characters, unless they've been made to break the fourth wall, have no idea what this means or why this happens.
I don't mean to imply our universe is a video game, since it would be games all the way up.
verdverm · · focus · HN ↗
NietzscheanNull · · focus · HN ↗
Perhaps we can't define a "partitioning" rule because no valid partition exists.
For consciousness/sentience, that's an incredibly tough a pill for most to swallow; it would mean calling into question more hundreds of years' worth (probably more) of philosophical thinking, all of which was constructed on the axiom that "sentience" is a single indivisible trait: you either have it or you don't.
If we find that "root dependency" was little more than wishful thinking all along, a whole slew of Enlightenment-era philosophy (and all the modern legal principles derived therefrom) suddenly fall apart unless we find some other suitable criterion that would shore them up (or we just collectively avert our attention and pretend the conflict doesn't exist, which is the route I expect many would prefer to take).
verdverm · · focus · HN ↗
NietzscheanNull · · focus · HN ↗
verdverm · · focus · HN ↗
fc417fc802 · · focus · HN ↗
I don't think that's true. Pretty much all legal constructs hold up just fine under game theory regardless of whether or not you consider the world to be deterministic and have absolutely nothing to do with consciousness or lack thereof. Also note that a deterministic world isn't an argument against consciousness.
NietzscheanNull · · focus · HN ↗
[0]: <a href="https://en.wikipedia.org/wiki/Reasonable_person" rel="nofollow">https://en.wikipedia.org/wiki/Reasonable_person
throw310822 · · focus · HN ↗
That's not the point. You know exactly how to define being conscious and aware- it's your subjective experiencing of the world and of your inner states. The problem is that being a subjective experiencing, there is no way to communicate it to the outside world.
Loquebantur · · focus · HN ↗
Maybe that's not as common with HN folks, but most other humans do.
pixl97 · · focus · HN ↗
But you have zero proof that their words aren't a confabulation, you can only know than when you output a similar set of words you had a subjective mind state they represent.
loopies · · focus · HN ↗
The very best you can do is define human consciousness as requiring the functions of a human brain, and try to figure out (decide/agree) the chances that something which is working loosely on the same principles is or isn't conscious.
At the end of the day it will always be an agreement, never scientifically proven as fact.
mitxela · · focus · HN ↗
teiferer · · focus · HN ↗
Anybody objecting to that statement is just ignorant of nature and/or adheres to the quasi-religious human superiority complex.
pixl97 · · focus · HN ↗
And there are plenty of people right now that would drag us kicking and screaming back into that ignorance and suffering. This is what can make talks of AI so annoying, not that people do or don't know what AI is, they don't have the first clue of what people are beyond their anecdotal experience. They don't question their motivations. Why particular flaws they have exist, and why these flaws are shared across humanity. What the pieces look like that make them tick. So yea, when these people talk it just adds noise to the conversation.
bigbadfeline · · focus · HN ↗
Say we agree that a lot of things outside of humans are sentient. So what? Humans are sentient by that doesn't prevent treating them with with wars, bombs and what not. In fact trillions are spent every year on weaponry aimed squarely at the most sentient of all sentient beings.
Why would anyone worry about LLM's welfare when there's no peace movement to speak of, wars are raging as if they're normal, but hey, look the LLM is crying?
Let's fix human welfare first, then we can sit down and have a really long conversation about the feeble emotions of token generators.
teiferer · · focus · HN ↗
Though I think that argument is not even applicable. "LLM's welfare" is a meaningless concept, unless "hammer and screwdriver welfare" also become a thing.
anon7000 · · focus · HN ↗
I’m not claiming humans are morally superior in sum (there is a lot of evil as well as good.) But it’s pretty obvious that humans are the superior species evolutionarily in that we’ve essentially dominated the planet. And on some level, are beating evolution itself (by fixing genetic issues and helping the “least fit” to survive and reproduce.) And are also the only species which has doctors for other species.
All I’m trying to say: I can easily accept other species are intelligent and emotional. The fact that I can know that and accept that, to me, speaks to human “superiority.”
Does superiority matter? Not most of the time. What about when it comes down to life and death? If the choice was to keep one human alive or one theoretically more perfect AI alive? (Boil down utilitarianism into its essence.) doing the “most good” is by definition a human construct. Maybe an agent could have one. But if you don’t accept human superiority on some level, would you accept that a more perfect AI deserves to live more than you? To me, that’s the implication.
teiferer · · focus · HN ↗
That in itself is an anthropocentric viewpoint. I'm sure an eagle watching down, seeing a human running around, could easily consider humans as quite inferior as they are so slow and restricted to 2d movement. The shark that sees a swimmer at the surface, struggling to stay afloat and unable to breathe under water may surely assert its superiority over such primitive beings. It all depends on the metric you use, and we obviously pick the metric that makes us shine.
On the moral side, few eagles, or bird species, have exterminated other species. Much less knowingly out of greed. Oh but that's different? Sure, if you ignore what makes humans look bad, morally. No surprise humans come out on top then.
> And on some level, are beating evolution itself (by fixing genetic issues and helping the “least fit” to survive and reproduce.)
That's a quite narrow viewpoint as well. Evolution is in full swing, just not in the "smarter/faster/taller/stronger" sense that a naive view of it would make you expect. "Fitness" is a concept relative to the existing conditions, and if the conditions make it easier for teenage cancer patients to reach reproductive age, then that's just less of a fitness disadvantage. Doesn't mean evolution stops.
Kim_Bruning · · focus · HN ↗
If you grep pubmed for relevant Ethology papers, you'll find it's a bit more than a belief. ;-)
Sophira · · focus · HN ↗
From a purely secular standpoint, my understanding as a layman is that consciousness is generated through the electrical impulses taking place in our brain every day. The neurons are physical, unlike the modelled neurons in machine learning models. That means that the actual electrical impulses have their own imperfections/weirdnesses that are probably not simulated in a machine learning model... but we quantise the weights anyway in most cases, which is probably more of an issue.
Making actual neuron components out of silicon on the scale needed for a LLM is beyond our current capacity, given that LLM models have many billions of parameters. We're good, but not that good.[0]
Using actual living neurons would be unethical as you'd need to source them from somewhere. So the only other option would be synthesising them ourselves. If this ever happens (or maybe when it happens)... what do we call the resulting creation? Does that count as consciousness?
Or am I looking at this the wrong way?
[0] [Edit: Turns out that when I said "we're not that good", I might be wrong. Within the last few months, IBM introduced the first sub-nanometer node chip: <a href="https://research.ibm.com/blog/sub-1nm-node-chips" rel="nofollow">https://research.ibm.com/blog/sub-1nm-node-chips . So... maybe it is in fact possible.]
Slash65 · · focus · HN ↗
Kim_Bruning · · focus · HN ↗
mitxela · · focus · HN ↗