>AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.
Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.
If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.
The current implementation as stateless matrix multiplication... yeah there's nothing going on there.
> If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.
The argument that models are conscious only during training, and not conscious during inference feels like a painful one to make. Are we killing models when they reach the training objective?
Technically we are killing models when they don't reach the training objective by throwing those weights away via survival of the fittest (fit meaning what we humans think we want).
Kinda like creating quantum copies of your child and keeping the ones that answer correctly and shooting the other ones in the face.
They don’t have persistent memory though. Models are static, what you have is a model that will reprocess the whole series of prompts + some data fetched from a data store. A harness is just a while loop prompting an LLM and executing tool calls, but the model is purely static
The model does not. The system it runs in can. My agents have persistent memory. It is not at all clear whether or the distinction that the long-term memory is external from the model matters.
Your second sentence does not support the claim in your first sentence; it's merely a second unsupported claim. It is also entirely irrelevant to the argument.
The threshold for behaviour consistent with having emotions (note: not behaviour sufficient to prove emotions) was one I purposefully set extremely low, because your comparison to a disk drive is so ludicrous, and yet your disc drive will still not clear it.
It's different, but it is not at all clear whether or not that matters for the purposes of whether or not they can experience things, seeing as we don't know how to measure that even for other humans other than by relying on self-reporting to infer without other evidence that their experience matches ours.
It's not just memory, the physical body also encodes trauma and stress (The Body Keeps the Score is a book that touches on this).
I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
Just because you don't remember the suffering doesn't mean you didn't experience it in the moment. Suffering does not suddenly become okay if your mind and body are going to forget it. "It's okay to torture someone if their mind and body both won't remember it afterwards" sure is a take.
Low blood sugar will also do this. The record button just stops working even though your physical body can and will still do things. A common issue with diabetics with hypoglycemia.
> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
That's an interesting point... but I'd argue it falls into the same category as qualia. It's basically this: if there is no state change, the qualia did not exist.
It's not a "trap", it's the real argument which makes me hesitant to dismiss consciousness of machines. Yes, it seems silly to dismiss a sick patient as having a lesser consciousness, which is exactly the point
It's not a real argument. Dementia patients can have lucid periods that plainly show their long-term memory isn't gone. They're mental state is obviously degraded, but its disingenuous to say they have no long-term memory and that they are any way similar to LLMs.
Yeah I was gonna add I know there are case studies where sick humans don't consciously have access to long term memories but will still respond nonverbally to people they "recognize".
Yes it's seemingly the go-to rebuttal, I've lost count of how many times I've seen it with barely any variation.
Not sure if it's a product of LessWrong or Slate Star Codex or something. I kinda suspect it might be LW, given how it superficially resembles an unanswerable philosophical paradox but in reality is about as challenging as a question posed in a 200 level philosophy course like introduction to ethics.
I get where you're going, but the argument isn't quite right.
A LLM never has and, in their current architecture, never can/could experience suffering or real change of state.
The general and expected case for humans is that capacity. A human my indeed lose capacity, e.g. being braindead and in severe cases we do indeed say that they are not conscious or able to suffer (different argument: some would of course say that is suffering in and of itself)
> being braindead and in severe cases we do indeed say that they are not conscious or able to suffer
Sure, there are such cases. But in the general case, it seems to be that memory and ability to change specifically are not necessary and are not sufficient for consciousness, so we shouldn't judge machines on that basis
Is the ability to form and retain long-term memories a necessary precondition for rights in humans though?
Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).
This argument would be difficult to reconcile if we didn't have billions of examples of humans forming memories plus modern evolutionary biology to trivially demonstrate such cases are rare pathologies but still human beings.
> Is the ability to form and retain long-term memories a necessary precondition for rights in humans though?
For a species / class of being yes it is necessary. Some members of that species / class being exceptions does not make the generalization invalid.
Without the ability to form and retain long-term memories we wouldn't have developed language and abstract concepts like rights.
Whether people like it or not, rights aren't natural or god-given. Rights are only enforced or protected by the tip of a spear (or point of a gun, or hover of a drone). There are no rights in nature and they are no where to be found when social order breaks down.
Animals can't make war for their rights, so they only get the rights we deign to give them. It's not nothing, but its not the same. There is no question to me that my dogs are conscious, but its not at the same level as humans.
There is no question for me that LLMs lack consciousness as they are stateless.
hosel · · focus · HN ↗
Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.
pton_xd · · focus · HN ↗
The current implementation as stateless matrix multiplication... yeah there's nothing going on there.
Lewton · · focus · HN ↗
They do, during training
add-sub-mul-div · · focus · HN ↗
swiftcoder · · focus · HN ↗
pixl97 · · focus · HN ↗
Kinda like creating quantum copies of your child and keeping the ones that answer correctly and shooting the other ones in the face.
vidarh · · focus · HN ↗
dgellow · · focus · HN ↗
vidarh · · focus · HN ↗
chrisjj · · focus · HN ↗
vidarh · · focus · HN ↗
I'd consider that a prerequisite, but not sufficient, for that question to make sense.
chrisjj · · focus · HN ↗
vidarh · · focus · HN ↗
The threshold for behaviour consistent with having emotions (note: not behaviour sufficient to prove emotions) was one I purposefully set extremely low, because your comparison to a disk drive is so ludicrous, and yet your disc drive will still not clear it.
Do you have an actual argument?
chrisjj · · focus · HN ↗
chrisjj · · focus · HN ↗
axionbraid · · focus · HN ↗
[dead]
staticman2 · · focus · HN ↗
vidarh · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
shawnz · · focus · HN ↗
pton_xd · · focus · HN ↗
I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
JoshTriplett · · focus · HN ↗
Just because you don't remember the suffering doesn't mean you didn't experience it in the moment. Suffering does not suddenly become okay if your mind and body are going to forget it. "It's okay to torture someone if their mind and body both won't remember it afterwards" sure is a take.
fwip · · focus · HN ↗
JoshTriplett · · focus · HN ↗
pixl97 · · focus · HN ↗
qarl · · focus · HN ↗
That's an interesting point... but I'd argue it falls into the same category as qualia. It's basically this: if there is no state change, the qualia did not exist.
Qualia is a bitch to reason about.
staticman2 · · focus · HN ↗
It's also a rhetorical move I've seen several times here...
shawnz · · focus · HN ↗
staticman2 · · focus · HN ↗
mitxela · · focus · HN ↗
staticman2 · · focus · HN ↗
joshheitzman · · focus · HN ↗
staticman2 · · focus · HN ↗
qlte · · focus · HN ↗
Not sure if it's a product of LessWrong or Slate Star Codex or something. I kinda suspect it might be LW, given how it superficially resembles an unanswerable philosophical paradox but in reality is about as challenging as a question posed in a 200 level philosophy course like introduction to ethics.
mitxela · · focus · HN ↗
Mumps · · focus · HN ↗
A LLM never has and, in their current architecture, never can/could experience suffering or real change of state.
The general and expected case for humans is that capacity. A human my indeed lose capacity, e.g. being braindead and in severe cases we do indeed say that they are not conscious or able to suffer (different argument: some would of course say that is suffering in and of itself)
shawnz · · focus · HN ↗
Sure, there are such cases. But in the general case, it seems to be that memory and ability to change specifically are not necessary and are not sufficient for consciousness, so we shouldn't judge machines on that basis
myrmidon · · focus · HN ↗
Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).
qlte · · focus · HN ↗
joshheitzman · · focus · HN ↗
For a species / class of being yes it is necessary. Some members of that species / class being exceptions does not make the generalization invalid.
Without the ability to form and retain long-term memories we wouldn't have developed language and abstract concepts like rights.
Whether people like it or not, rights aren't natural or god-given. Rights are only enforced or protected by the tip of a spear (or point of a gun, or hover of a drone). There are no rights in nature and they are no where to be found when social order breaks down.
Animals can't make war for their rights, so they only get the rights we deign to give them. It's not nothing, but its not the same. There is no question to me that my dogs are conscious, but its not at the same level as humans.
There is no question for me that LLMs lack consciousness as they are stateless.
[deleted] · · focus · HN ↗
[deleted]
mitxela · · focus · HN ↗