Thread

1. drtgh · 1789758284 · focus · HN ↗

   > relatively poorly understood technology

   Poorly understood? how convenient...

   LLMs are vectorial databases with losses that index statistically filled data, which uses a text interface to query such statistically filled data. The output is a string concatenation (statistically concatenated bit by bit).

   When the LLMs are queried (prompted), you can get random mixed data as output, ERRORS, due to undesired indexes getting closer at one point while the string was being concatenated for the output, what affects the rest of the indexed content that will be concatenated.

   It is intrinsic to this tech. The larger the context, the greater the probability of get mixed data. And if the provider lowers the precision of those indexes -in order to decrease hardware resources and energy consumption- such probability increases to the point where those errors are granted.

   Even knowing that the queries can return wrong/mixed data in the responses, errors, the companies developing this, decided to introduce a new product, that connects such LLMs outputs to the command console, latter connected to internet, raw 'eval' running commands from such outputs witch obviously can contain whatever mixed random. Then we started to hear "oh, it deleted my directory", etc, and it seems the next one will be "a missile killed my wife", because it is a text concatenation engine with errors.

   To name it "hallucination" is an euphemism... those are errors, and they are granted to happen at one moment. If they do not know this, then they ate too much marketing without doing their job, or it was a convenient contract for the pocket$ of someone.

   1. theptip · 1789760419 · focus · HN ↗

      > LLMs are vectorial databases

      You use a bunch of technical-sounding words here to make it sound like you understand. But to be clear, nobody understands why the evolved weights of a NN make the decisions that they do.

      Almost nothing is understood about the actual representations used for nontrivial concepts, decision algorithms, etc.

      If you look at the field of mechanistic interpretability, compared to “GOFAI” like learned decision trees, an LLM is completely opaque.

      1. semiquaver · 1789762968 · focus · HN ↗

         I’m shocked how many otherwise well-informed people don’t understand or agree with this very fundamental fact of just how little we actually understand about why LLMs work as well as they do. They figure “it’s science, of course there’s math and theory behind it.”

         AI research is almost as purely empirical as the gradient descent loops its practitioners use to optimize their models. “Why” anything at all works is barely an afterthought.

         1. bigyabai · 1789763371 · focus · HN ↗

            > why LLMs work as well as they do.

            That's a very different claim from being "poorly understood" though. The emergent properties of any system with billions of parameters is hard to understand completely, that's the fault of data science more than computer science or even mathematics.

            1. semiquaver · 1789763816 · focus · HN ↗

               I think ”poorly understood” is accurate. Understanding has levels. How brains think is also poorly understood.

               1. tripzilch · 1789817826 · focus · HN ↗

                  If you are serious, comparing how well we understand the brain vs how well we understand LLMs, .. it's not a stretch simplifying that to "we don't understand brains, we do understand LLMs".

                  Because the extent to which we don't understand the brain, is quite overpowering.

                  Some people forget that when they say "but it's not different from what a human does" ...

                  1. semiquaver · 1789824330 · focus · HN ↗

                     I am absolutely serious. I agree with you that the extent to which we don’t understand brains is overpowering. And I would stand by the proposition that “we don’t understand LLMs in almost exactly the way we don’t understand brains”.

                     We know lots about human development and genetics and biology and evolution and neuroscience and the physics of how brains are connected and send signals and how generally they are put together and have names for their parts and all that, but we’re clueless when it comes to “the hard question” of how qualia and consciousness emerges from that.

                     The scenario with the spooky simulation of thinking that emerges from LLMs is in the same category, with different details. Lots of knowledge about the substrate of the phenomenon, little to none about the much bigger question of how we get the appearance of cognition from these trained artifacts.

                     Clearly we understand extremely well how LLMs are created mechanically. We invented them and are currently putting massive amounts of work into studying and improving them. But that work is perforce largely empirical; figuring out the why once again eludes us. It just goes to show how mysterious the underlying phenomenon of cognition is.

                     1. tripzilch · 1789897590 · focus · HN ↗

                        I'm not following you here, you seem to be conflating LLMs ability of language use with the brain's ability of thinking and cognition?

                        Are you saying that thinking and cognition requires language use? Cause I think not.

                        Or are you saying that language use is sufficient for cognition and thinking? Cause I'm also not convinced of that.

                        What I am convinced of, is that a machine capable of language use is capable of tricking people into believing there's a "there", there. In pretty much the same way as the famous supra-normal stimuli experiment made baby seagulls believe that a stick with a red dot was their parent. It's exploiting our instincts.

                        1. semiquaver · 1790087652 · focus · HN ↗

                           I re-read my comment and I’m not sure where you got that impression. LLMs perform a simulation of cognition. if you have used them for anything nontrivial you would be a fool to deny the fact that behind the language there is something approximating cognition. If you haven’t, solving a bunch of millennium prizes is pretty good evidence.

Unofficial Hacker News client; not affiliated with Y Combinator.