Thread

1. 1659447091 · 1789609891 · focus · HN ↗

   > The San Francisco company revealed what it said was the “unexpected or concerning” behavior of its A.I. models as part of a new framework for reporting “misalignment,” which is when the goals or actions of A.I. systems diverge from human intentions and values.

   Misalignment: "when the goals or actions of [...] systems diverge from human intentions"

   How about we stop trying to nudge the language towards implying sentience or consciousness and keep the same word that has been used for that definition for longer than I have written software, a bug.

   We should be talking about why the tools/environment keep getting overlooked. The software built around the text generator, forget the researchers and mathematicians discovering the math properties of language patterns -- why are we not talking about the software engineers building the LLM-pluggable tools that actually allow/cause real action to happen?

   1. marshray · 1789611137 · focus · HN ↗

      Computers are used to evaluate LLMs, but LLMs are not "software" or "algorithms" in the traditional sense. They are not built out of conditional branches or loops.

      So trying to squeeze the observed behavior of this new thing under existing terms like "software bug" is at least as much of a force-fit, and what you're doing here is just as much language engineering as choosing to use a term like '[mis]alignment'. Which is fine, this is just one way that humans choose language.

      1. unleashhale · 1789617459 · focus · HN ↗

         Right, they’re MAGIC!

         Not being built out of conditional branches or loops does not mean they’re somehow outside algorithms or computation. Learned parameters don’t confer exemption from computing.

         Did the engineered system behave as intended? No? Then you’ve got a gd bug/failure.

         1. marshray · 1789618979 · focus · HN ↗

            Don't straw-man me bro!

            No disagreement that unintended undesirable behavior could usefully be described as a 'failure'.

Unofficial Hacker News client; not affiliated with Y Combinator.