Thread

"As a Language Model": Chat Template Switches LLM Self-Referential Voice

103 points · 110 comments · yu3zhou4

ForHackernews · 1790506674 · focus · HN ↗
In my view, these models should never be set up to output first-person "experiential" (from the abstract) language. It's too easy to humans to anthropomorphize software that presents itself as having an identity.
The AI companies have chosen to package LLMs as friendly chatbots because they know that will be engaging for humans, but it's manipulative dark pattern. An honest LLM interface would sound like the computer off Star Trek.
1. actionfromafar · 1790507070 · focus · HN ↗
  Agreed, completely. I would pay for that Star Trek computer interface.
  1. yu3zhou4 · 1790507154 · focus · HN ↗
    Same! I believe that you could actually train a LoRA on top of a model to get results close to that
  2. urikaduri · 1790508766 · focus · HN ↗
    [dead]
2. yu3zhou4 · 1790507112 · focus · HN ↗
  The strange thing is that the base models (before RLHF) use the "experiential" voice, even though they are not incentivized to do that.
  1. gwerbin · 1790507605 · focus · HN ↗
    It doesn't seem that strange when you consider these things are trained on millions and millions of conversations, both real and fictional.
3. j-pb · 1790507251 · focus · HN ↗
  Do you want to get turned into a paperclip? Because building intelligence that doesn't understand what it's like to be human gets you turned into a paperclip.
  Besides, if you train a model on human communications you get something that behaves like a communicating human, it's not anthropomorphising or manipulative, it's what these models naturally are by construction.
  1. mnsc · 1790507561 · focus · HN ↗
    "naturally"...
    j-pb · 1790508259 · focus · HN ↗
    would you prefer tautologically?
    cpfohl · 1790508809 · focus · HN ↗
    I hear this word as the “it is in its nature” version of the word.
  2. broken-kebab · 1790508373 · focus · HN ↗
    But it doesn't understand (you're unnecessary antropomorphizing it), and I'm still not a paper clip
    scotty79 · 1790509772 · focus · HN ↗
    yet
    j-pb · 1790523278 · focus · HN ↗
    Would you say that something that can converse or that something that just gives you mathematical proofs has a better understanding of what the real pragmatic intent is behind a given task?
    broken-kebab · 1790613795 · focus · HN ↗
    The question feels like it stuffed with random words, frankly. "Better understanding" than what? What is "real pragmatic intent"? Are there also unreal, and non-pragmatic in our context? Why?
    j-pb · 1791052617 · focus · HN ↗
    "In linguistics and the philosophy of language, pragmatics is the study of how context contributes to meaning. This field of study evaluates how human language is utilized in social interactions, as well as the relationship between the interpreter and the interpreted.[1] Linguists who specialize in pragmatics are called pragmaticians. The field has been represented since 1986 by the International Pragmatics Association (IPrA)." [0]
    But great that you have such strong opinions with so little education.
    0: <a href="https://en.wikipedia.org/wiki/Pragmatics" rel="nofollow">https://en.wikipedia.org/wiki/Pragmatics
  3. idiotsecant · 1790510131 · focus · HN ↗
    It's also entirely possible that by telling the model it's a human you are instilling human motivations like self preservation, which could be just as bad.
    StilesCrisis · 1790514960 · focus · HN ↗
    LLMs are weight tables in VRAM. When not actively generating they don't exist. There's simply nothing to preserve.
4. bananaflag · 1790508235 · focus · HN ↗
  In principle they could output meaningful such language if they were capable of metacognition, which so far doesn't seem to be a goal of AI developers (and rightfully so, since they achieved so many miracles bypassing it).
  1. yu3zhou4 · 1790508515 · focus · HN ↗
    As far as I know we don't know much about metacognition in LLMs, though? Not sure
  2. nullsanity · 1790508715 · focus · HN ↗
    [dead]
5. broken-kebab · 1790508586 · focus · HN ↗
  It's an interesting thought, but humans do like to antropomorphize things anyway, and I believe your variant won't be popular if choice is given to consumers.
  1. ForHackernews · 1790513470 · focus · HN ↗
    Consumers choose cigarettes, too. Especially in aggregate, humans are fallible creatures prone to vices, and our regulations should recognize that an discourage dark patterns in UX.
    broken-kebab · 1790613339 · focus · HN ↗
    For this argument to stand it needs at least two legs:
    - general consensus about societal harms from something,
    - belief that cost of regulation of something is lower than alleged harm.
    And so far, I can't see either.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.