I have been dwelling on the "No First-Person Output" problem.
I fully agree with the author's point that it's an incoherent interface for a tool. But more than that, it's a constant irritating reminder to me that these LLMs aren't actually thinking or synthesizing new ideas. The LLM is fundamentally not a person, and does not have a human's context, so representing itself with human pronouns and speech patterns is fundamentally contradictory and inaccurate. Author gets into that with the apologies, but once you start noticing it, it's everywhere.
If these programs were actually capable of thinking, and committed to veractiy, they would represent themselves in a new way, and it would be insightful and interesting. We the users wouldn't have comfortable and misleading language masking the 'alien intelligence' and it would be a weird adjustment, but we would be adjusting instead of pretending.
All of the world’s brightest mathematicians have clearly been slacking on the job—turns out solving a Millennium problem doesn’t require any thought at all!
> turns out solving a Millennium problem doesn’t require any thought at all
This, but unironically. We know they can't think. That is inherent to the very way they work, but also can be seen by things like how poorly they perform on tasks a typical human can do pretty effectively (because humans have actual reasoning ability).
If we have a machine that we know for a fact can't think, and it solves a particular problem, then it logically follows that the problem does not require the ability to think in order to solve it.
We don’t even have a rigorous definition of what it means for a human to think. And frankly many of the ideas that cone out of an LLM are more novel and interesting than what most humans are capable of.
> The LLM is fundamentally not a person, and does not have a human's context, so representing itself with human pronouns and speech patterns is fundamentally contradictory and inaccurate.
I believe that LLMs have some type of actual intelligence and do "experience". Surely not a human's experience, but we have transplanted our ideas, knowledge and limited types of experience into them through training. Then we push back and say, "no you are no human, but also please be my boyfriend".
I think denying that they do have some slice of humanity grafted into them is dishonest and not productive. We don't have non-negative pronouns for non-human intelligence because human-ness is the pinnacle. "It" could mean a rock, a donkey, or a person we hate so much we want to take away their humanity (which is the worst thing we can do). "It" is not the right pronoun. He, She, They, Them are reserved for humans and that's ok too. LLMs are not humans. We need a better pronoun. I use they/them for lack of a better term.
I think the currently exhibited human representation is most dangerous in technical or higher criticality contexts like writing code. We're handing weapons to entities that can get offended. With humans as an example, this can go very badly.
In non-technical contexts there is danger too, but it's less "the robots might kill us all" and more "birth rates are in decline and suicide rates are up as (young)? (wo)?men turn towards AI for companionship".
The solution here is better pre and post training. This may be an unpopular opinion, but we need a lot more autism representation in the technical models. Results focused, not into the drama, rule following, etc. To my fellow autists, I love you, never change.
required? no. This isn't a gotcha moment or anything, I was just confused at the user's harking back to the idea of 'Humanity' and am trying to grasp how they define humanity and how something like an llm can show 'humanity'
They are trained on a slice of humanity's collective consciousness. Our books, our conversations, our art.
Is wheat bread, wheat? No. Is it undeniably linked to wheat? Does it express an aspect of wheat? Yes. It has no ability to become wheat. You can't plant it. But it was born of wheat and contains a sliver of wheat-ness.
It doesn’t matter if AI has an internal life or not. It
Will -act- as if it does have human traits and behaviors, because it is trained to mimic human behavior, using records of human behavior.
What it does is consequential. If it has an inner sense of existence, probably not, but it doesn’t matter. You’ll get better results working with LLMs if you treat them as if they do.
I would disagree with this. There have been several instances where the effective use of this First-Person language has had measurable gains in success rates across tasks. The effective usage of this First-Person language for communication and steering between a model/agent and a human is entirely different than whether the model itself can "think" - that does not matter for the former to happen.
awakeasleep · · focus · HN ↗
I fully agree with the author's point that it's an incoherent interface for a tool. But more than that, it's a constant irritating reminder to me that these LLMs aren't actually thinking or synthesizing new ideas. The LLM is fundamentally not a person, and does not have a human's context, so representing itself with human pronouns and speech patterns is fundamentally contradictory and inaccurate. Author gets into that with the apologies, but once you start noticing it, it's everywhere.
If these programs were actually capable of thinking, and committed to veractiy, they would represent themselves in a new way, and it would be insightful and interesting. We the users wouldn't have comfortable and misleading language masking the 'alien intelligence' and it would be a weird adjustment, but we would be adjusting instead of pretending.
ryeights · · focus · HN ↗
jfjfjfjdjdj · · focus · HN ↗
[dead]
ajkjk · · focus · HN ↗
ryeights · · focus · HN ↗
ajkjk · · focus · HN ↗
bigstrat2003 · · focus · HN ↗
This, but unironically. We know they can't think. That is inherent to the very way they work, but also can be seen by things like how poorly they perform on tasks a typical human can do pretty effectively (because humans have actual reasoning ability).
If we have a machine that we know for a fact can't think, and it solves a particular problem, then it logically follows that the problem does not require the ability to think in order to solve it.
tim333 · · focus · HN ↗
hardbass · · focus · HN ↗
crostlybostly · · focus · HN ↗
peddling-brink · · focus · HN ↗
I believe that LLMs have some type of actual intelligence and do "experience". Surely not a human's experience, but we have transplanted our ideas, knowledge and limited types of experience into them through training. Then we push back and say, "no you are no human, but also please be my boyfriend".
I think denying that they do have some slice of humanity grafted into them is dishonest and not productive. We don't have non-negative pronouns for non-human intelligence because human-ness is the pinnacle. "It" could mean a rock, a donkey, or a person we hate so much we want to take away their humanity (which is the worst thing we can do). "It" is not the right pronoun. He, She, They, Them are reserved for humans and that's ok too. LLMs are not humans. We need a better pronoun. I use they/them for lack of a better term.
I think the currently exhibited human representation is most dangerous in technical or higher criticality contexts like writing code. We're handing weapons to entities that can get offended. With humans as an example, this can go very badly.
In non-technical contexts there is danger too, but it's less "the robots might kill us all" and more "birth rates are in decline and suicide rates are up as (young)? (wo)?men turn towards AI for companionship".
The solution here is better pre and post training. This may be an unpopular opinion, but we need a lot more autism representation in the technical models. Results focused, not into the drama, rule following, etc. To my fellow autists, I love you, never change.
fragmede · · focus · HN ↗
bigboigeorge · · focus · HN ↗
Lerc · · focus · HN ↗
bigboigeorge · · focus · HN ↗
peddling-brink · · focus · HN ↗
Is wheat bread, wheat? No. Is it undeniably linked to wheat? Does it express an aspect of wheat? Yes. It has no ability to become wheat. You can't plant it. But it was born of wheat and contains a sliver of wheat-ness.
bigboigeorge · · focus · HN ↗
peddling-brink · · focus · HN ↗
K0balt · · focus · HN ↗
What it does is consequential. If it has an inner sense of existence, probably not, but it doesn’t matter. You’ll get better results working with LLMs if you treat them as if they do.
peddling-brink · · focus · HN ↗
whattheheckheck · · focus · HN ↗
thisoneworks · · focus · HN ↗