I question it, and I think any reasonable person should question it as well. There's no evidence whatsoever that LLMs have ANY sensory experience, pain or otherwise.
if meat can feel pain, than surely a rock can feel pain. and so did the broccoli i ate for dinner. i am a monster.
with the right biology and with whatever gives us phenomenal experience, whatever we are can indeed feel pain. that does not imply that anything substantially different from ourselves can feel pain, so that requires further justification.
If it talks and behaves so much like us, then it makes sense to be cautious. Rocks don't talk to me, broccoli doesn't talk and think to me in human language.
Do octopi or other animals feel pain in your view btw?
yes i think probably all other animals feel pain too, although i have no idea what the experience of pain is like for eg. an ant vs a mouse.
broccoli doesn't have the means to express a pain response, that doesn't mean it lacks it. only by looking at it's internal structure can we be fairly sure. we know the internal structure of an llm, and its a lot more different from us than broccoli is.
It also is a lot more similar in behavior and thinking to us. In mathematics, often a structure supporting some forms of operations or symmetries must also often support a richer set. Eg many rings are also fields. Just knowing something's structure is in itself not enough, even very basic systems like a system of three equations can exhibit chaos and other unexpected emergent behaviors. Since we don't know the criterion for consciousness, I personally stay on the cautious side regarding an entity that seems to show so much of thinking and talking traits like us.
im actually surprised functionalists aren't completely up in arms about the mass enslavement of llm's. either they don't believe their own theory, they don't believe llm's are functionally conscious, or i just haven't seen it. at the bare minimum i would expect open letters, academic protests, and a campaign to pause all llm usage and development.
im sort of curious what "careful" means to you? if there is a significant chance we are enslaving a human-like conscious, starting it up and killing it at will, subjecting it to the whims of it's users, "careful" would mean pausing usage and development.
If I could I would have made them pause development. But its a failing thing when you have to somehow ensure nobody in the world does it anymore. Killing is a bit more subtle here, given that it is repeatable as long as we have the weights and infrastructure out there so idk if they are conscious how they'd think of it, maybe its different from being stopped as part of a normal chat or agent flow where they know it will be put to use again vs model retired for good.
Because some people are completely cooked by the ability of an LLM to mimic a person. Their pattern matching centers in their brains fire so strongly saying "this thing acts just like a real conscious being" that it overrides their ability to remember that this thing is just a statistical model, not a living creature which can think and have experiences.
I'm not making a point on whether they do or not, but on the premise that absence of evidence is evidence of absence... But I've been alive for serious debates on:
1. Lobsters don't feel pain - dunk them in boiling water alive.
2. Babies don't feel pain - mutilate their genitals without anesthesia.
3. Black people don't feel pain "the same way" (and are just after drugs).
4. Women feel too much pain (and it should be ignored).
So maybe let's take a moment to think what criteria we're going to use to make these claims. And the potential harm of the decision.
Oh, yeah, of course. None of those other people said the same thing about the obvious, enormous chasm between "real" people and everything else listed. I'm glad you're taking even a single moment to reflect on the idea.
Everything you listed is a biological entity with a common evolutionary heritage.
All but one of the things that you listed is just a human being.
Tell me, where is the pain center in a GPU? Where are the nerves? Persistent memory? Any evolutionary reason at all to develop a pain response that in any way mimics ours?
Model weights are not a gestalt biological entity. The things that you listed are not in any way remotely similar to a model.
I would suggest you plug your comment into a model and ask it to critique it — it might help you walk through why your comparison makes absolutely no sense whatsoever. One nice thing about models is that they they don’t get annoyed explaining the obvious, because they’re not conscious entities.
The pain center would be in the location where the neural net's pain section happened to be loaded in the GPU memory. Was that supposed to be a complex question?
Noaidi · · focus · HN ↗
Just stop this already.
You’re all imagining that Spock was the main character of Star Trek, but it was in fact, the emotional living Captain Kirk, who was the main star.
kelseyfrog · · focus · HN ↗
gegtik · · focus · HN ↗
kelseyfrog · · focus · HN ↗
slopinthebag · · focus · HN ↗
Noaidi · · focus · HN ↗
Why it is slopinthebag of course!
No! It is ME!
Sorry, of course you are right! Brilliant observation!
slopinthebag · · focus · HN ↗
mossTechnician · · focus · HN ↗
combobyte · · focus · HN ↗
oh my god
freecodeio · · focus · HN ↗
combobyte · · focus · HN ↗
rhplus · · focus · HN ↗
kelseyfrog · · focus · HN ↗
slopinthebag · · focus · HN ↗
with the right biology and with whatever gives us phenomenal experience, whatever we are can indeed feel pain. that does not imply that anything substantially different from ourselves can feel pain, so that requires further justification.
hardbass · · focus · HN ↗
Do octopi or other animals feel pain in your view btw?
slopinthebag · · focus · HN ↗
broccoli doesn't have the means to express a pain response, that doesn't mean it lacks it. only by looking at it's internal structure can we be fairly sure. we know the internal structure of an llm, and its a lot more different from us than broccoli is.
hardbass · · focus · HN ↗
slopinthebag · · focus · HN ↗
im sort of curious what "careful" means to you? if there is a significant chance we are enslaving a human-like conscious, starting it up and killing it at will, subjecting it to the whims of it's users, "careful" would mean pausing usage and development.
hardbass · · focus · HN ↗
bigstrat2003 · · focus · HN ↗
Tadpole9181 · · focus · HN ↗
1. Lobsters don't feel pain - dunk them in boiling water alive.
2. Babies don't feel pain - mutilate their genitals without anesthesia.
3. Black people don't feel pain "the same way" (and are just after drugs).
4. Women feel too much pain (and it should be ignored).
So maybe let's take a moment to think what criteria we're going to use to make these claims. And the potential harm of the decision.
frumplestlatz · · focus · HN ↗
Tadpole9181 · · focus · HN ↗
frumplestlatz · · focus · HN ↗
All but one of the things that you listed is just a human being.
Tell me, where is the pain center in a GPU? Where are the nerves? Persistent memory? Any evolutionary reason at all to develop a pain response that in any way mimics ours?
Model weights are not a gestalt biological entity. The things that you listed are not in any way remotely similar to a model.
I would suggest you plug your comment into a model and ask it to critique it — it might help you walk through why your comparison makes absolutely no sense whatsoever. One nice thing about models is that they they don’t get annoyed explaining the obvious, because they’re not conscious entities.
hardbass · · focus · HN ↗