Thread

1. stratos123 · 1791107241 · focus · HN ↗

   LeCun also said back in 2022 that "if you train a machine, as powerful as it could be, your 'GPT-5000', on text", it will never be able to learn basic common-sense physics like that objects placed on tables will move along with them.

   1. mdp2021 · 1791117069 · focus · HN ↗

      > never be able to learn basic common-sense physics

      And has it at this stage, within in-depth take of said "learning", foundationally?

      I have not been able to properly check the studies for a long time now, but I remain unaware of achieved solutions on the problem of reliably referencing a world model out of a language model - that "counting the 'r's in 'raspberry'" be not guessing, not memory, but actually counting.

      1. Version467 · 1791121712 · focus · HN ↗

         LeCun's argument wasn't about the definition of learning though. He stated that they would never get these common sense things correct because they weren't sufficiently part of the training data. A statement that we can hopefully all agree has been thoroughly refuted.

         1. bonzini · 1791123352 · focus · HN ↗

            As of a few months ago they still have trouble, with low thinking, at the "should I drive to a car wash that is 100 m away" kind of question.

            1. BobbyJo · 1791125334 · focus · HN ↗

               It's a nonsensical question to ask, and how an LLM answers gives 0 signal.

               If you were home and a family member asked you that question, you'd probably criticise the question rather than answering. LLM are RLHF'd into being milk-toast helpers that just try to answer questions like that with no criticism.

               This is all beside the fact that the world of AI has changed pretty dramatically in the last few months.

               1. names_are_hard · 1791125718 · focus · HN ↗

                  [delayed]

                  1. BobbyJo · 1791176948 · focus · HN ↗

                     TIL. I feel like I've learned this a few times now, so we'll see if it sticks this time.
               2. daveguy · 1791125784 · focus · HN ↗

                  It is so nonsensical because it has such an obvious answer. The answer is so obvious, in fact, that one answer can be considered nonsense and the other common sense.

                  1. BobbyJo · 1791176910 · focus · HN ↗

                     I disagree pretty strongly. If someone asked "Should I drive to the carwash?", the most obvious response, and the one nearly everyone would give, is a question: "why are you going to the car wash?" because asking the question implies you don't need the car with you.
               3. frrrree · 1791128792 · focus · HN ↗

                  This is just a stupid post.

                  It’s nonsense to test if a product that is marketed and sold as being able to provide generalised intelligence on demand, does what it says on the tin?

                  Check yourself

                  1. joquarky · 1791132883 · focus · HN ↗

                     Since you're new here, I'd suggest you read the guidelines for etiquette.

                     <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;newsguidelines.html">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;newsguidelines.html

                     1. WaltPurvis · 1791136089 · focus · HN ↗

                        It&#x27;s very unlikely that person is either new or unfamiliar with the guidelines. They almost certainly created a throwaway account specifically because they know the guidelines and want to flout them without consequences. (It seems like there has been an uptick in the number of these kinds of throwaway flame comments. I wonder if HN tracks that?)
               4. SpicyLemonZest · 1791128903 · focus · HN ↗

                  [delayed]
            2. lern_too_spel · 1791132842 · focus · HN ↗

               Low thinking is an artificial constraint. It can fail spectacularly on things that aren&#x27;t in the training data.
            3. keeda · 1791136205 · focus · HN ↗

               Simply appending “check your assumptions” to the question fixed it even back then: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=47040530">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=47040530

               Similarly for Apple’s “red herring” paper, simply adding a generic caveat to “disregard irrelevant factors” (without specifying which ones) restored performance even in the weaker local llama models back then.

               The flaw was not in the reasoning; the flaw seems to be simply that the assumptions we make are often different from the assumptions it makes. I wonder if that might be a fundamental underlying cause of misalignment.

Unofficial Hacker News client; not affiliated with Y Combinator.