Thread

1. drtgh · 1789758284 · focus · HN ↗

   > relatively poorly understood technology

   Poorly understood? how convenient...

   LLMs are vectorial databases with losses that index statistically filled data, which uses a text interface to query such statistically filled data. The output is a string concatenation (statistically concatenated bit by bit).

   When the LLMs are queried (prompted), you can get random mixed data as output, ERRORS, due to undesired indexes getting closer at one point while the string was being concatenated for the output, what affects the rest of the indexed content that will be concatenated.

   It is intrinsic to this tech. The larger the context, the greater the probability of get mixed data. And if the provider lowers the precision of those indexes -in order to decrease hardware resources and energy consumption- such probability increases to the point where those errors are granted.

   Even knowing that the queries can return wrong/mixed data in the responses, errors, the companies developing this, decided to introduce a new product, that connects such LLMs outputs to the command console, latter connected to internet, raw 'eval' running commands from such outputs witch obviously can contain whatever mixed random. Then we started to hear "oh, it deleted my directory", etc, and it seems the next one will be "a missile killed my wife", because it is a text concatenation engine with errors.

   To name it "hallucination" is an euphemism... those are errors, and they are granted to happen at one moment. If they do not know this, then they ate too much marketing without doing their job, or it was a convenient contract for the pocket$ of someone.

   1. theptip · 1789760419 · focus · HN ↗

      > LLMs are vectorial databases

      You use a bunch of technical-sounding words here to make it sound like you understand. But to be clear, nobody understands why the evolved weights of a NN make the decisions that they do.

      Almost nothing is understood about the actual representations used for nontrivial concepts, decision algorithms, etc.

      If you look at the field of mechanistic interpretability, compared to “GOFAI” like learned decision trees, an LLM is completely opaque.

      1. tantalor · 1789760622 · focus · HN ↗

         They don't "make decisions".

         That's like saying "my d20 decided to roll a 17"

         1. nonethewiser · 1789760768 · focus · HN ↗

            But isnt the point that it did roll a 17. And no one knows exactly how (in the case of LLMs)? Therefor any description of the conclusion should be thought of as an anology. Decided, randomly accessed, etc.

            1. Cthulhu_ · 1789762970 · focus · HN ↗

               If nobody knows exactly how, then "at random" sounds about right and the results should be treated as such.

               That is, in this case, it should not be used to influence decisions that can start a war.

               1. krapp · 1789763807 · focus · HN ↗

                  People will just roll their eyes at you and say "the human mind is nothing but a dice roll too" and call you a slope-headed neanderthal before continuing apace.

                  1. hardbass · 1790159299 · focus · HN ↗

                     Any line of thinking that begins with the assumption human consciousness has supernatural elements should be completely ignored.
               2. semiquaver · 1789764079 · focus · HN ↗

                  I agree wholeheartedly about your second sentence, but

                  “we made this artifact and don’t know why the thing it does looks spookily like cognition”

                  and

                  “this artifact makes decisions at random”

                  are obviously distinct categories and pretending otherwise is silly.

                  1. watwut · 1789764454 · focus · HN ↗

                     We know why it looks like cognition. Because OpenAI and Antropic put a lot of effort and training to humanize the output and make it sound like a person.

                     Regardless of negative consequences it brings. They have that project of creating tech god which will save the unborn people thousands years in the future ... so people living now dont matter.

                     That is why.

                     1. semiquaver · 1789777541 · focus · HN ↗

                        “Putting a lot of effort and training” into a dog or an inanimate carbon rod would never result in something that can plausibly substitute for human mental labor and looks likely to eventually surpass us at many tasks, no matter how much you put in.

                        So I don’t think “labs worked hard” is the same thing is “we know scientifically how these things work in any real level of detail”. The ability to build a thing, even if building it is hard, is not the same thing as understanding of what the thing is or how it works, not even a little bit.

               3. theptip · 1789798066 · focus · HN ↗

                  This is a false dichotomy. We can make statements about the distribution, and we have fuzzy models about certain sets of inputs and outputs. We can steer the outputs.

                  It’s not completely random. We just don’t understand why the tricks we learned work.

                  (Fully agree with the second point FWIW)

Unofficial Hacker News client; not affiliated with Y Combinator.