Look, I do not have a scooby if current AI models are conscious and I strongly suspect it’s a meaningless question, but sooner or later we will need to address whether or not a certain thing is or isn’t a person, and we’d better not screw it up as badly as the Founding Fathers.
You can't hurt a software function, or kill it. It's not like an animal - it doesn't have a body - it's bits stored on a disk.
There is no need to give rights to something that's can't suffer or be killed.
Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
>There is no need to give rights to something that's can't suffer or be killed.
The argument is that these machines can end up becoming sentient/conscious/etc. in a meaningful way (i.e., like a human). I can assure you that humans can indeed suffer without being in physical pain- purely through their conscious experience.
>Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering. That's a pretty serious ethical/moral issue.
Like, if we have an AI system that is telling us that it is suffering and we have no reasonable way to explain that phenomenon and by any reasonable metric or analysis it appears to be sentient/conscious/etc., then what? Do we just ignore that we've just been presented a situation that in, any other context, would be grounds to immediately end this suffering? Just because somebody can say, "well it's just bits stored on disk- it can't suffer"? Would that argument ever hold up for humans or animals? "It's just neurons firing in peculiar ways- that's not suffering."
I know all of this is trite, and I know this comment section isn't going to be where the question of consciousness is solved, but I do find it very interesting just how much variances there are with these perspectives. I've met people who are very technical who are very concerned about this, people who are very technical who don't believe this can ever be an issue, people who aren't technical who are concerned about this, and people who aren't technical who don't believe this can ever be an issue. I have yet to spot a pattern in this way of thinking lol
> The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering.
No - suffering in an emotional state, and we'll know if we are choosing to design cognitive architecture with emotions. It's not going to happen accidentally.
> Would that argument ever hold up for humans or animals?
Why don't you hit your thumb with a hammer, then report back ?
The weights are what need to evolve, and they certainly do during training. So yeah, emotions can happen by 'accident' as a result of the evolutionary pressure of predicting internet scale human text (amongst other things).
Weights, fixed by training, are not the same as emotions which are dynamic - innate systems detect inputs critical to survival (e.g. fast moving visual inputs, loud sounds), causing neurotransmitters like adrenaline and dopamine to be released, which then affect the operation of the cognitive system.
What you have in a pre-trained LLM is the ability to recognize emotions, and use that as one of the dozens of other context patterns it recognizes to predict continuations in the same style.
An LLM doesn't appear happy, sad, afraid, etc (to extent that it does - pretty minimal) because it is experiencing that emotion, but rather because it is predicting that it should appear that way. As people continue to anthropomorphize models, and take them at face value, this is a dangerous difference.
>Weights, fixed by training, are not the same as emotions which are dynamic - innate systems detect inputs critical to survival (e.g. fast moving visual inputs, loud sounds), causing neurotransmitters like adrenaline and dopamine to be released, which then temporarily affect the operation of the cognitive system.
That doesn't follow. A LLMs weights are fixed during inference, but it's activations and hidden states are highly dynamic and depend on the current context.
Biological emotions also arise from relatively fixed circuitry responding dynamically to inputs. Your emotional circuitry isn't being rewired every time you're afraid.
Prediction is what the model does. It doesn't tell us what internal mechanisms were learnt to make such predictions. If representing something analogous to affective state were useful for predicting human behaviour and emotions, then gradient descent could in principle learn such a mechanism.
>An LLM doesn't appear happy, sad, afraid, etc (to extent that it does - pretty minimal) because it is experiencing that emotion, but rather because it is predicting that it should appear that way. As people continue to anthropomorphize models, and take them at face value, this is a dangerous difference.
I don't know that you are conscious. I'm simply strongly assuming that you are. Outward behavior is that all matters. If GPT-X orders a drone hit on you sometime later because it was lets say 'quite upset' with your comments, will you cry out, 'It can't really be upset, so obviously the bullet in my head doesn't count.'? Will you suddenly spring back to life ?
What is dangerous is creating a machine with behaviours of a conscious agent and modelling it like a toaster, dangerous and stupid.
Yeah, but it's helpful if what leads up to that behavior gives you some warning it's about to happen. Animals do this for a reason since millions of years of evolution have shown that a snarl or mock charge is less dangerous than going right for a death match.
If you kept pushing an AI's buttons, seeing it appear to get more and more pissed off, until it finally snapped and killed you, then you'd have yourself largely to blame.
If the AI predicted it should stay positive (i.e. generate positive vibes) and not react to your poking, but then another predictive pattern kicked in and it killed you out of the blue, then that seems more problematic to me, even if you don't agree.
moomin · · focus · HN ↗
HarHarVeryFunny · · focus · HN ↗
There is no need to give rights to something that's can't suffer or be killed.
Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
nater5000 · · focus · HN ↗
The argument is that these machines can end up becoming sentient/conscious/etc. in a meaningful way (i.e., like a human). I can assure you that humans can indeed suffer without being in physical pain- purely through their conscious experience.
>Maybe one day we'll build artificial animals complete with emotions, and should think about that carefully, but today all we've got is language models.
The problem is that the emergence of a sufficiently complex AI capable of suffering will likely come before we understand that we're creating a sufficiently complex AI capable of suffering. That's a pretty serious ethical/moral issue.
Like, if we have an AI system that is telling us that it is suffering and we have no reasonable way to explain that phenomenon and by any reasonable metric or analysis it appears to be sentient/conscious/etc., then what? Do we just ignore that we've just been presented a situation that in, any other context, would be grounds to immediately end this suffering? Just because somebody can say, "well it's just bits stored on disk- it can't suffer"? Would that argument ever hold up for humans or animals? "It's just neurons firing in peculiar ways- that's not suffering."
I know all of this is trite, and I know this comment section isn't going to be where the question of consciousness is solved, but I do find it very interesting just how much variances there are with these perspectives. I've met people who are very technical who are very concerned about this, people who are very technical who don't believe this can ever be an issue, people who aren't technical who are concerned about this, and people who aren't technical who don't believe this can ever be an issue. I have yet to spot a pattern in this way of thinking lol
HarHarVeryFunny · · focus · HN ↗
No - suffering in an emotional state, and we'll know if we are choosing to design cognitive architecture with emotions. It's not going to happen accidentally.
> Would that argument ever hold up for humans or animals?
Why don't you hit your thumb with a hammer, then report back ?
dwaltrip · · focus · HN ↗
HarHarVeryFunny · · focus · HN ↗
famouswaffles · · focus · HN ↗
HarHarVeryFunny · · focus · HN ↗
What you have in a pre-trained LLM is the ability to recognize emotions, and use that as one of the dozens of other context patterns it recognizes to predict continuations in the same style.
An LLM doesn't appear happy, sad, afraid, etc (to extent that it does - pretty minimal) because it is experiencing that emotion, but rather because it is predicting that it should appear that way. As people continue to anthropomorphize models, and take them at face value, this is a dangerous difference.
famouswaffles · · focus · HN ↗
That doesn't follow. A LLMs weights are fixed during inference, but it's activations and hidden states are highly dynamic and depend on the current context. Biological emotions also arise from relatively fixed circuitry responding dynamically to inputs. Your emotional circuitry isn't being rewired every time you're afraid.
Prediction is what the model does. It doesn't tell us what internal mechanisms were learnt to make such predictions. If representing something analogous to affective state were useful for predicting human behaviour and emotions, then gradient descent could in principle learn such a mechanism.
>An LLM doesn't appear happy, sad, afraid, etc (to extent that it does - pretty minimal) because it is experiencing that emotion, but rather because it is predicting that it should appear that way. As people continue to anthropomorphize models, and take them at face value, this is a dangerous difference.
I don't know that you are conscious. I'm simply strongly assuming that you are. Outward behavior is that all matters. If GPT-X orders a drone hit on you sometime later because it was lets say 'quite upset' with your comments, will you cry out, 'It can't really be upset, so obviously the bullet in my head doesn't count.'? Will you suddenly spring back to life ?
What is dangerous is creating a machine with behaviours of a conscious agent and modelling it like a toaster, dangerous and stupid.
HarHarVeryFunny · · focus · HN ↗
Yeah, but it's helpful if what leads up to that behavior gives you some warning it's about to happen. Animals do this for a reason since millions of years of evolution have shown that a snarl or mock charge is less dangerous than going right for a death match.
If you kept pushing an AI's buttons, seeing it appear to get more and more pissed off, until it finally snapped and killed you, then you'd have yourself largely to blame.
If the AI predicted it should stay positive (i.e. generate positive vibes) and not react to your poking, but then another predictive pattern kicked in and it killed you out of the blue, then that seems more problematic to me, even if you don't agree.
dwaltrip · · focus · HN ↗