Thread

1. ForHackernews · 1790506674 · focus · HN ↗

   In my view, these models should never be set up to output first-person "experiential" (from the abstract) language. It's too easy to humans to anthropomorphize software that presents itself as having an identity.

   The AI companies have chosen to package LLMs as friendly chatbots because they know that will be engaging for humans, but it's manipulative dark pattern. An honest LLM interface would sound like the computer off Star Trek.

   1. j-pb · 1790507251 · focus · HN ↗

      Do you want to get turned into a paperclip? Because building intelligence that doesn't understand what it's like to be human gets you turned into a paperclip.

      Besides, if you train a model on human communications you get something that behaves like a communicating human, it's not anthropomorphising or manipulative, it's what these models naturally are by construction.

      1. mnsc · 1790507561 · focus · HN ↗

         "naturally"...

         1. j-pb · 1790508259 · focus · HN ↗

            would you prefer tautologically?
         2. cpfohl · 1790508809 · focus · HN ↗

            I hear this word as the “it is in its nature” version of the word.
      2. broken-kebab · 1790508373 · focus · HN ↗

         But it doesn't understand (you're unnecessary antropomorphizing it), and I'm still not a paper clip

         1. scotty79 · 1790509772 · focus · HN ↗

            yet
         2. j-pb · 1790523278 · focus · HN ↗

            Would you say that something that can converse or that something that just gives you mathematical proofs has a better understanding of what the real pragmatic intent is behind a given task?

            1. broken-kebab · 1790613795 · focus · HN ↗

               The question feels like it stuffed with random words, frankly. "Better understanding" than what? What is "real pragmatic intent"? Are there also unreal, and non-pragmatic in our context? Why?

               1. j-pb · 1791052617 · focus · HN ↗

                  "In linguistics and the philosophy of language, pragmatics is the study of how context contributes to meaning. This field of study evaluates how human language is utilized in social interactions, as well as the relationship between the interpreter and the interpreted.[1] Linguists who specialize in pragmatics are called pragmaticians. The field has been represented since 1986 by the International Pragmatics Association (IPrA)." [0]

                  But great that you have such strong opinions with so little education.

                  0: <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Pragmatics" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Pragmatics

      3. idiotsecant · 1790510131 · focus · HN ↗

         It&#x27;s also entirely possible that by telling the model it&#x27;s a human you are instilling human motivations like self preservation, which could be just as bad.

         1. StilesCrisis · 1790514960 · focus · HN ↗

            LLMs are weight tables in VRAM. When not actively generating they don&#x27;t exist. There&#x27;s simply nothing to preserve.

Unofficial Hacker News client; not affiliated with Y Combinator.