Thread

1. 1659447091 · 1789609891 · focus · HN ↗

   > The San Francisco company revealed what it said was the “unexpected or concerning” behavior of its A.I. models as part of a new framework for reporting “misalignment,” which is when the goals or actions of A.I. systems diverge from human intentions and values.

   Misalignment: "when the goals or actions of [...] systems diverge from human intentions"

   How about we stop trying to nudge the language towards implying sentience or consciousness and keep the same word that has been used for that definition for longer than I have written software, a bug.

   We should be talking about why the tools/environment keep getting overlooked. The software built around the text generator, forget the researchers and mathematicians discovering the math properties of language patterns -- why are we not talking about the software engineers building the LLM-pluggable tools that actually allow/cause real action to happen?

   1. marshray · 1789611137 · focus · HN ↗

      Computers are used to evaluate LLMs, but LLMs are not "software" or "algorithms" in the traditional sense. They are not built out of conditional branches or loops.

      So trying to squeeze the observed behavior of this new thing under existing terms like "software bug" is at least as much of a force-fit, and what you're doing here is just as much language engineering as choosing to use a term like '[mis]alignment'. Which is fine, this is just one way that humans choose language.

      1. drtgh · 1789626318 · focus · HN ↗

         LLMs are vectorial databases with losses that index statistically filled data, which uses a text interface to query such statistically filled data. The output is a string concatenation.

         By the nature of the used architecture in such software, the used algorithms, when queried (prompted), you can get random mixed data as output, ERRORS, due to undesired indexes getting closer at one point while the string was being concatenated for the output, what affects the rest of the indexed content that will be concatenated.

         And this is intrinsic to this tech. The larger the context, the greater the probability of get mixed data. And if the provider lowers the precision of those indexes -in order to decrease hardware and energy resources consumption- such probability increases to the point where those errors are granted.

         Anyway, even knowing that the queries can return wrong/mixed data in the responses (errors), the companies developing this, decided to introduce a new product, that connects such LLMs responses to the command console, latter connected to internet, running commands from such returned responses witch obviously can contain whatever mixed random. Then we started to hear "oh, it deleted my directory", etc.

         Again, One have such described statistical database with text interface, witch query the database recursively with the output text of the previous query, and this is connected to the command console. Larger contexts, several times... What should we expect as result? rhetoric question.

         Implying sentience or consciousness is a convenient marketing strategy that has been introduced by anthropomorphising the names of all the methods and algorithms used. An "Agent" should be translated from such deceiving language to "context splitter querying in loop that consumes more tokens from us", or similar.

         1. drtgh · 1789641534 · focus · HN ↗

            * > An "Agent" should be translated from such deceiving language to "context splitter querying in loop that consumes more tokens from us", or similar.

            Please disregard this line. I wanted to point out that it increases the length of the context (and therefore the probability of errors) due the batch processing. But I redacted it incorrectly because I also wanted to imply that promoting the use such queries non-stop increases the billing through tokens consumption.

Unofficial Hacker News client; not affiliated with Y Combinator.