Thread

1. drtgh · 1789758284 · focus · HN ↗

   > relatively poorly understood technology

   Poorly understood? how convenient...

   LLMs are vectorial databases with losses that index statistically filled data, which uses a text interface to query such statistically filled data. The output is a string concatenation (statistically concatenated bit by bit).

   When the LLMs are queried (prompted), you can get random mixed data as output, ERRORS, due to undesired indexes getting closer at one point while the string was being concatenated for the output, what affects the rest of the indexed content that will be concatenated.

   It is intrinsic to this tech. The larger the context, the greater the probability of get mixed data. And if the provider lowers the precision of those indexes -in order to decrease hardware resources and energy consumption- such probability increases to the point where those errors are granted.

   Even knowing that the queries can return wrong/mixed data in the responses, errors, the companies developing this, decided to introduce a new product, that connects such LLMs outputs to the command console, latter connected to internet, raw 'eval' running commands from such outputs witch obviously can contain whatever mixed random. Then we started to hear "oh, it deleted my directory", etc, and it seems the next one will be "a missile killed my wife", because it is a text concatenation engine with errors.

   To name it "hallucination" is an euphemism... those are errors, and they are granted to happen at one moment. If they do not know this, then they ate too much marketing without doing their job, or it was a convenient contract for the pocket$ of someone.

   1. theptip · 1789760419 · focus · HN ↗

      > LLMs are vectorial databases

      You use a bunch of technical-sounding words here to make it sound like you understand. But to be clear, nobody understands why the evolved weights of a NN make the decisions that they do.

      Almost nothing is understood about the actual representations used for nontrivial concepts, decision algorithms, etc.

      If you look at the field of mechanistic interpretability, compared to “GOFAI” like learned decision trees, an LLM is completely opaque.

      1. semiquaver · 1789762968 · focus · HN ↗

         I’m shocked how many otherwise well-informed people don’t understand or agree with this very fundamental fact of just how little we actually understand about why LLMs work as well as they do. They figure “it’s science, of course there’s math and theory behind it.”

         AI research is almost as purely empirical as the gradient descent loops its practitioners use to optimize their models. “Why” anything at all works is barely an afterthought.

         1. dwattttt · 1789780130 · focus · HN ↗

            > as purely empirical as the gradient descent loops

            Are you suggesting that gradient descent is an empirically found and not understood technique? It was originally proposed by Cauchy in 1847, its properties are very well understood.

            You might be referring to properties of the domains its being applied to.

            1. semiquaver · 1789783666 · focus · HN ↗

               I’m not saying gradient descent was empirically discovered, I’m saying that its use in machine learning (or elsewhere, I suppose) is itself a form of empiricism in that what it is is essentially a repeated observe/measure error/adjust cycle.

               1. skydhash · 1789784294 · focus · HN ↗

                  > I’m not saying gradient descent was empirically discovered, I’m saying that its use in machine learning is itself a form of empiricism. A repeated observe/adjust-based-on-data cycle

                  The data is the input, the output is to generally find the lowest amount of a loss function. It’s a greedy approach because brute forcing is inefficient.

                  It’s no more empirical than a greedy algorithm for scheduling.

                  1. theptip · 1789818545 · focus · HN ↗

                     > It’s no more empirical than a greedy algorithm for scheduling.

                     Right, GP is drawing a distinction between search, ie mechanical exploration of a space, with understanding, ie having a map of the territory such that you don’t need trial and error.

Unofficial Hacker News client; not affiliated with Y Combinator.