Thread

1. drtgh · 1789758284 · focus · HN ↗

   > relatively poorly understood technology

   Poorly understood? how convenient...

   LLMs are vectorial databases with losses that index statistically filled data, which uses a text interface to query such statistically filled data. The output is a string concatenation (statistically concatenated bit by bit).

   When the LLMs are queried (prompted), you can get random mixed data as output, ERRORS, due to undesired indexes getting closer at one point while the string was being concatenated for the output, what affects the rest of the indexed content that will be concatenated.

   It is intrinsic to this tech. The larger the context, the greater the probability of get mixed data. And if the provider lowers the precision of those indexes -in order to decrease hardware resources and energy consumption- such probability increases to the point where those errors are granted.

   Even knowing that the queries can return wrong/mixed data in the responses, errors, the companies developing this, decided to introduce a new product, that connects such LLMs outputs to the command console, latter connected to internet, raw 'eval' running commands from such outputs witch obviously can contain whatever mixed random. Then we started to hear "oh, it deleted my directory", etc, and it seems the next one will be "a missile killed my wife", because it is a text concatenation engine with errors.

   To name it "hallucination" is an euphemism... those are errors, and they are granted to happen at one moment. If they do not know this, then they ate too much marketing without doing their job, or it was a convenient contract for the pocket$ of someone.

   1. theptip · 1789760419 · focus · HN ↗

      > LLMs are vectorial databases

      You use a bunch of technical-sounding words here to make it sound like you understand. But to be clear, nobody understands why the evolved weights of a NN make the decisions that they do.

      Almost nothing is understood about the actual representations used for nontrivial concepts, decision algorithms, etc.

      If you look at the field of mechanistic interpretability, compared to “GOFAI” like learned decision trees, an LLM is completely opaque.

      1. tantalor · 1789760622 · focus · HN ↗

         They don't "make decisions".

         That's like saying "my d20 decided to roll a 17"

         1. nonethewiser · 1789760768 · focus · HN ↗

            But isnt the point that it did roll a 17. And no one knows exactly how (in the case of LLMs)? Therefor any description of the conclusion should be thought of as an anology. Decided, randomly accessed, etc.

            1. semi-extrinsic · 1789762549 · focus · HN ↗

               It's actually no different for dice than for LLMs. Explaining accurately the reason for the exact outcome of any given dice roll someone makes would be stupendously hard. It would require lots of instrumentation and math and be poorly transferrable to another surface, another player, etc.

               But even so people don't say that we don't understand how dice work.

               Saying that we don't understand how LLMs work is exactly like saying we don't understand how dice, or tires, or golf ball shots work. Or like the old myth that we don't understand how bumblebees fly.

               1. theptip · 1789797882 · focus · HN ↗

                  It’s different in this way:

                  The best way to model dice is the Physical Stance. You consider rules such as gravity, kinematics, etc. There is no “internal state”, “world model”, “knowledge”. If you prefer, in Friston’s terms, there is no Markov Blanket.

                  The best way to model a human is the Intentional Stance[1]. You mostly need things like beliefs, knowledge, biases, etc to build this model. In Friston’s terms, there is a Markov Blanket, an inside vs outside.

                  Without going into any irrelevant-but-interesting philosophical discussions about consciousness, I believe the intentional stance is most useful for modeling LLMs. Most of the success in predicting, debugging, optimizing these systems is in activities like understanding what they believe, what their intent was, what they observed, what they concluded from those observations. Also note that much simpler creatures benefit from the Intentional Stance; you will be more successful at modeling your dog if you think about what it “wants” rather than trying to run Physics on it.

                  [1]: <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Intentional_stance" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Intentional_stance - the astute reader will note that I skipped the Design Stance. If we truly understood how NNs actually implement all their cognitive processes then we could perhaps apply this to them; if we actually crafted and designed every parameter of its mind. But we are talking about why dice are different.

                  1. tantalor · 1789832947 · focus · HN ↗

                     Great comment. I didn&#x27;t know about this! Thanks

                     The latest episode of On The Media also uses this framing.

                     &gt; On the Media: How Extinction Entered the AI Debate

                     <a href="https:&#x2F;&#x2F;www.wnycstudios.org&#x2F;podcasts&#x2F;otm" rel="nofollow">https:&#x2F;&#x2F;www.wnycstudios.org&#x2F;podcasts&#x2F;otm

Unofficial Hacker News client; not affiliated with Y Combinator.