Thread

1. stratos123 · 1791107241 · focus · HN ↗

   LeCun also said back in 2022 that "if you train a machine, as powerful as it could be, your 'GPT-5000', on text", it will never be able to learn basic common-sense physics like that objects placed on tables will move along with them.

   1. mdp2021 · 1791117069 · focus · HN ↗

      > never be able to learn basic common-sense physics

      And has it at this stage, within in-depth take of said "learning", foundationally?

      I have not been able to properly check the studies for a long time now, but I remain unaware of achieved solutions on the problem of reliably referencing a world model out of a language model - that "counting the 'r's in 'raspberry'" be not guessing, not memory, but actually counting.

      1. cavoirom · 1791120561 · focus · HN ↗

         counting 'r' in 'raspberry' to the LLM is similar to 4-dimension space to human. Their world's unit is token, not character, although they could use indirect method such as "run code" to find out. It will stay that way until they change the fundamental of the token that the LLM can perceive characters.

         1. estearum · 1791121325 · focus · HN ↗

            It's not even fair to call "run code" to be indirect compared to what a human would do. The word raspberry has no Rs in it in human language either. We have a written representation of it, which we can then write down either in our head or on paper, and then we can "run the algorithm" of counting each of the letters.

            Nothing intrinsically more or less direct about the LLM's method than ours.

            1. cavoirom · 1791122182 · focus · HN ↗

               I could argue LLM only have "token" as their perceivable dimension, compare to human multiple senses as the physic perceivable dimension and a brain with many other dimension of "learning" and "thinking". In spoken language, we may not have 'r' but in written we have, both spoken language and written language are learned skills.

               1. scratcheee · 1791123564 · focus · HN ↗

                  You could argue in return that humans only have electro-chemistry as our one perceivable dimension. We only indirectly perceive light through the signals our eyes send to our brains.

                  In my mind agi is pretty much by definition a virtual machine, so the mechanisms behind thought are only relevant for the sake of efficiency (ie you can argue that LLMs make a poor basis for intelligence because tokens and natural language are a poor way to encode the world, but if you can run it on a big enough computer to counteract the inherent wasteful virtualisation then who really cares how it works under the hood?)

                  1. cavoirom · 1791130382 · focus · HN ↗

                     So LLM and human all have 1 dimenion perceivable signal, just LLM is 240p, and human is 8K in resolution, that's why we have 'r' in our signal, LLM still have 'r' in their signal, just because of the "low resolution", raspberry wasn't encoded with so many 'r' as in human signal.

                     I will stop here before our analogies go too far.

Unofficial Hacker News client; not affiliated with Y Combinator.