Thread

1. drtgh · 1789758284 · focus · HN ↗

   > relatively poorly understood technology

   Poorly understood? how convenient...

   LLMs are vectorial databases with losses that index statistically filled data, which uses a text interface to query such statistically filled data. The output is a string concatenation (statistically concatenated bit by bit).

   When the LLMs are queried (prompted), you can get random mixed data as output, ERRORS, due to undesired indexes getting closer at one point while the string was being concatenated for the output, what affects the rest of the indexed content that will be concatenated.

   It is intrinsic to this tech. The larger the context, the greater the probability of get mixed data. And if the provider lowers the precision of those indexes -in order to decrease hardware resources and energy consumption- such probability increases to the point where those errors are granted.

   Even knowing that the queries can return wrong/mixed data in the responses, errors, the companies developing this, decided to introduce a new product, that connects such LLMs outputs to the command console, latter connected to internet, raw 'eval' running commands from such outputs witch obviously can contain whatever mixed random. Then we started to hear "oh, it deleted my directory", etc, and it seems the next one will be "a missile killed my wife", because it is a text concatenation engine with errors.

   To name it "hallucination" is an euphemism... those are errors, and they are granted to happen at one moment. If they do not know this, then they ate too much marketing without doing their job, or it was a convenient contract for the pocket$ of someone.

   1. margalabargala · 1789758879 · focus · HN ↗

      I agree with most of your comment, but...

      > To name it "hallucination" is an euphemism... those are errors

      I find this and other "don't anthropomorphize the computer" statements incredibly unconvincing.

      People develop terms for things and language has always contained overloaded or "literally inaccurate" terms.

      An LLM can have "hallucinations" in the same way a modern computer program can have "bugs".

      1. jyounker · 1789760119 · focus · HN ↗

         From the point of view of the system, this is an error. It is incorrect information.

         The term "hallucination" feels much more like anthropomorphizing. The word hallucination implies an aberrant condition. A much better term would be "confabulation".

         You don't trust things or individuals that confabulate.

         1. ChrisLTD · 1789760340 · focus · HN ↗

            a filling in of gaps in memory through the creation of false memories by an individual who is affected with a memory disorder (as Korsakoff syndrome) and is unaware that the fabricated memories are inaccurate and false

            vs.

            a sensory perception (such as a visual image or a sound) that occurs in the absence of an actual external stimulus and usually arises from neurological disturbance (such as that associated with delirium tremens, schizophrenia, Parkinson's disease, or narcolepsy) or in response to drugs (such as LSD or phencyclidine)

            1. gizajob · 1789761221 · focus · HN ↗

               Confabulation is also a symptom very prevalent in forms of narcissism and psychopathy. Gaps in understanding or perception are back-filled by confabulating so as to not risk the omnipotence of the confabulator.

               Up to the reader to decide whether this phenomenon is found in the statements of AI leadership or not.

               1. t-3 · 1789762904 · focus · HN ↗

                  It's also something people tend to do when thinking, daydreaming, trying to solve problems, etc. We just usually don't fall for our own bullshit.

                  LLMs don't either. They just give output in response to input. If the output is wrong that's because the model is wrong, not because the LLM is doing anything it's not supposed to be. It just wasn't built well enough to produce the expected result.

Unofficial Hacker News client; not affiliated with Y Combinator.