Thread

LeCun has "zero concerns" about AI wiping out humanity, recent "rogue" incidents

399 points · 752 comments · Anon84

stratos123 · 1791107241 · focus · HN ↗
LeCun also said back in 2022 that "if you train a machine, as powerful as it could be, your 'GPT-5000', on text", it will never be able to learn basic common-sense physics like that objects placed on tables will move along with them.
1. mdp2021 · 1791117069 · focus · HN ↗
  > never be able to learn basic common-sense physics
  And has it at this stage, within in-depth take of said "learning", foundationally?
  I have not been able to properly check the studies for a long time now, but I remain unaware of achieved solutions on the problem of reliably referencing a world model out of a language model - that "counting the 'r's in 'raspberry'" be not guessing, not memory, but actually counting.
  1. ethbr1 · 1791117362 · focus · HN ↗
    My perspective is that the addition of thinking loops to models allows sufficiently advanced ones to approximate world models.
    Incredibly inefficiently because of the recursive loops ("Wait, the object is on the table. I should think about this more deeply..."), and likely instantly surpassed by large world models if/when those are shipped, but effectively enough vs non-thinking models.
    hyperman1 · 1791119697 · focus · HN ↗
    This sounds like a human trying to reason about quantum mechanics. We als simplify to newtonian for day to day tasks.
    kurthr · 1791128890 · focus · HN ↗
    I like this analogy. Both GenRel and QM are well beyond our experience, and although there is some intuition that comes from working with the equations over time, it is bizarre and "just calculate" often gets the correct answer faster.
    Picking the right tool or model is like picking the right problem to work on. It's actually quite hard (often you can't just try them all), but without it you will be incredibly inefficient and occasionally, fundamentally wrong.
    All models are wrong, but some are useful. -Box
    red75prime · 1791121336 · focus · HN ↗
    LeCun calling them "world models" gives a high-level description of the desired functionality. They are Joint Embedding Predictive Architectures (with SIGReg). They might produce more useful world models, but it's yet to be seen.
  2. cavoirom · 1791120561 · focus · HN ↗
    counting 'r' in 'raspberry' to the LLM is similar to 4-dimension space to human. Their world's unit is token, not character, although they could use indirect method such as "run code" to find out. It will stay that way until they change the fundamental of the token that the LLM can perceive characters.
    estearum · 1791121325 · focus · HN ↗
    It's not even fair to call "run code" to be indirect compared to what a human would do. The word raspberry has no Rs in it in human language either. We have a written representation of it, which we can then write down either in our head or on paper, and then we can "run the algorithm" of counting each of the letters.
    Nothing intrinsically more or less direct about the LLM's method than ours.
    cavoirom · 1791122182 · focus · HN ↗
    I could argue LLM only have "token" as their perceivable dimension, compare to human multiple senses as the physic perceivable dimension and a brain with many other dimension of "learning" and "thinking". In spoken language, we may not have 'r' but in written we have, both spoken language and written language are learned skills.
    azornathogron · 1791123016 · focus · HN ↗
    Is "token" a directly perceivable unit for the LLM? If you ask it "how many tokens are in this sentence?" can it count them (again, not guessing or making a tool call)?
    I've never tried it and it might take some thought and effort to conduct an experiment to find out properly, but I would be interested in the answer.
    lawandjustice · 1791123924 · focus · HN ↗
    I dont think so. This is akin to asking a person, what is the frequency of the light hitting your eye when watching a leaf for example. You either know the (approximate) answer by knowing the frequency of green, or use a tool to measure it. If the LLM gives the correct answer it is either.guessing based on intution(and this intuition is based on pairs of word to tokenization length in text form in training data), writing code(or executing a tokenizer) or running a tokenizer mentally (reasoning via CoT).
    mdp2021 · 1791149435 · focus · HN ↗
    Not the point: the simulated intelligence in this context needs to create proper representation. It is not a matter of what it sees but of what it can see.
    scratcheee · 1791123564 · focus · HN ↗
    You could argue in return that humans only have electro-chemistry as our one perceivable dimension. We only indirectly perceive light through the signals our eyes send to our brains.
    In my mind agi is pretty much by definition a virtual machine, so the mechanisms behind thought are only relevant for the sake of efficiency (ie you can argue that LLMs make a poor basis for intelligence because tokens and natural language are a poor way to encode the world, but if you can run it on a big enough computer to counteract the inherent wasteful virtualisation then who really cares how it works under the hood?)
    cavoirom · 1791130382 · focus · HN ↗
    So LLM and human all have 1 dimenion perceivable signal, just LLM is 240p, and human is 8K in resolution, that's why we have 'r' in our signal, LLM still have 'r' in their signal, just because of the "low resolution", raspberry wasn't encoded with so many 'r' as in human signal.
    I will stop here before our analogies go too far.
    [deleted] · 1791125592 · focus · HN ↗
    [deleted]
    omneity · 1791125703 · focus · HN ↗
    I’m working on this problem using a vocab-free, byte-based approach. It’s definitely solvable.
    <a href="https://huggingface.co/posts/omarkamali/593639295164067" rel="nofollow">https://huggingface.co/posts/omarkamali/593639295164067
    <a href="https://huggingface.co/blog/omarkamali/tokenization" rel="nofollow">https://huggingface.co/blog/omarkamali/tokenization
    lern_too_spel · 1791133474 · focus · HN ↗
    I used to think byte level tokenization was the answer, but humans also think at a word level and only reevaluate the words at a character level when asked. The solution to better tokenization across languages is likely to be learned tokenization. Here is one attempt I have seen: <a href="https://github.com/SamD770/bitter-lesson-tokenization" rel="nofollow">https://github.com/SamD770/bitter-lesson-tokenization
    mdp2021 · 1791148580 · focus · HN ↗
    Careful: the problem is very certainly ___not___ counting letters. That is only a telling way to check "is the NN checking or not?". We demand that NNs for consultancy tasks check, strictly.
    kevhito · 1791130750 · focus · HN ↗
    How many 'r's are there in the next 30 seconds of this [1] song?
    [1]: <a href="https://youtu.be/l7vRSu_wsNc?si=SndkB6GBaRyhvNNA&t=61" rel="nofollow">https://youtu.be/l7vRSu_wsNc?si=SndkB6GBaRyhvNNA&t=61
    mdp2021 · 1791149145 · focus · HN ↗
    I hope you understand: it is a core point that systems that answer questions must have the ability to internally represent the objects they assess in a way that allows reliability. Whatever the object.
    ozgung · 1791187083 · focus · HN ↗
    Similar to asking how many 'r's in raspberry to a Chinese person when speaking through interpreter. They would answer 0 because there are no 'r's in Chinese. And they would say raspberry has just two letters, tree and berry.
  3. Version467 · 1791121712 · focus · HN ↗
    LeCun's argument wasn't about the definition of learning though. He stated that they would never get these common sense things correct because they weren't sufficiently part of the training data. A statement that we can hopefully all agree has been thoroughly refuted.
    bonzini · 1791123352 · focus · HN ↗
    As of a few months ago they still have trouble, with low thinking, at the "should I drive to a car wash that is 100 m away" kind of question.
    BobbyJo · 1791125334 · focus · HN ↗
    It's a nonsensical question to ask, and how an LLM answers gives 0 signal.
    If you were home and a family member asked you that question, you'd probably criticise the question rather than answering. LLM are RLHF'd into being milk-toast helpers that just try to answer questions like that with no criticism.
    This is all beside the fact that the world of AI has changed pretty dramatically in the last few months.
    names_are_hard · 1791125718 · focus · HN ↗
    [delayed]
    BobbyJo · 1791176948 · focus · HN ↗
    TIL. I feel like I've learned this a few times now, so we'll see if it sticks this time.
    daveguy · 1791125784 · focus · HN ↗
    It is so nonsensical because it has such an obvious answer. The answer is so obvious, in fact, that one answer can be considered nonsense and the other common sense.
    BobbyJo · 1791176910 · focus · HN ↗
    I disagree pretty strongly. If someone asked "Should I drive to the carwash?", the most obvious response, and the one nearly everyone would give, is a question: "why are you going to the car wash?" because asking the question implies you don't need the car with you.
    frrrree · 1791128792 · focus · HN ↗
    This is just a stupid post.
    It’s nonsense to test if a product that is marketed and sold as being able to provide generalised intelligence on demand, does what it says on the tin?
    Check yourself
    joquarky · 1791132883 · focus · HN ↗
    Since you're new here, I'd suggest you read the guidelines for etiquette.
    <a href="https://news.ycombinator.com/newsguidelines.html">https://news.ycombinator.com/newsguidelines.html
    WaltPurvis · 1791136089 · focus · HN ↗
    It's very unlikely that person is either new or unfamiliar with the guidelines. They almost certainly created a throwaway account specifically because they know the guidelines and want to flout them without consequences. (It seems like there has been an uptick in the number of these kinds of throwaway flame comments. I wonder if HN tracks that?)
    SpicyLemonZest · 1791128903 · focus · HN ↗
    [delayed]
    lern_too_spel · 1791132842 · focus · HN ↗
    Low thinking is an artificial constraint. It can fail spectacularly on things that aren't in the training data.
    keeda · 1791136205 · focus · HN ↗
    Simply appending “check your assumptions” to the question fixed it even back then: <a href="https://news.ycombinator.com/item?id=47040530">https://news.ycombinator.com/item?id=47040530
    Similarly for Apple’s “red herring” paper, simply adding a generic caveat to “disregard irrelevant factors” (without specifying which ones) restored performance even in the weaker local llama models back then.
    The flaw was not in the reasoning; the flaw seems to be simply that the assumptions we make are often different from the assumptions it makes. I wonder if that might be a fundamental underlying cause of misalignment.
    randysalami · 1791126127 · focus · HN ↗
    I thought it was more because of fundamental limitations in the architecture. As in, no matter the training data, it could not be consistently and generally represented
    intended · 1791126331 · focus · HN ↗
    No?
    This is always the issues in the discussions.
    There’s the outcomes camp (objectivists?), which points at the things LLMs can do.
    Then there’s the process methods camp, which talks about what is actually going on.
    If you only care about the outcome, then the process does t matter.
    If you are talking about what is happening, what the underlying mechanics and science of it is, then the process matters.
    These models aren’t thinking. They simulate cognition well enough to do useful work in several fields and domains.
    Both are true.
    IshKebab · 1791131534 · focus · HN ↗
    > These models aren’t thinking.
    They are for any definition of the word that makes any kind of sense. I'm sure you have a contorted definition that magically only includes humans though...
    intended · 1791139161 · focus · HN ↗
    [delayed]
    aesthesia · 1791140320 · focus · HN ↗
    It depends on whether you assume that thinking requires doing everything that humans do. I think it would be silly to say that an AI doesn't think because it doesn't wrinkle its forehead in concentration. So you need to decide which parts of the way that humans think are actually necessary components of the process.
    IshKebab · 1791140631 · focus · HN ↗
    Tbh it doesn't even matter if humans turn out to have a soul, or quantum microtubules or whatever other magic LLMs can't have.
    The normal definition of the word "thinking" definitely includes what LLMs do. Hell people used to say computers were thinking even before AI. It's super weird to get all uppity about the semantics of the word now.
    intended · 1791173469 · focus · HN ↗
    [delayed]
    sampullman · 1791174915 · focus · HN ↗
    Do we have to invent a new word, then? Thinking seems close enough, and I don't see how it's useful to quibble about semantics in this particular case.
    Language changes over time anyway, so even if you really believe what LLMs are doing isn't the "thinking" of 2024, it probably will be the "thinking" of 2027, because most people are using it that way.
    mdp2021 · 1791146239 · focus · HN ↗
    > for any definition of the word
    For "thinking" here we mean "assessing a representation of an object". That, or equivalent, is required to be reliable. So it is fundamental and critical.
    kooi · 1791148864 · focus · HN ↗
    I think where both camps get hung up is sometimes the process method group "ignores" the obvious outcomes and effectiveness of LLMs.
    But the outcomes group "ignores" the fundamental limitations of models which are purely text based.
    E.g, a baseball players trains to catch high-speed balls and they dont do it by: "ball velocity 50mph, vector:[1,2,3], run move hand command now"
    That's absurd.
    No, there is an embodied network which is "trained" on visual, tactile input, and control as direct output.
    LLMs are fundamentally not the right tool for that.
    mdp2021 · 1791150353 · focus · HN ↗
    > E.g, a baseball players trains to catch high-speed balls and they dont do it by: "ball velocity 50mph, vector:[1,2,3], run move hand command now"
    That is a NN that learns a skill.
    But that is not an Analyst. If it were ballistics, then the answer to "how to parametrize the launch to reliably hit the target" excludes getting the result through natural skill.
    The problem lies in the need to get "AI" facing "LLMs": the latter create a need for reliability, for "AI".
    Speech is an endowment of both those who give educated guesses via developed skills and of those who return answers like Analysts, who check and compute. LLMs create a confusion between the two, and they will remain a problem until an ability to act as Analysts - strictly - will be implemented.
    kooi · 1791168059 · focus · HN ↗
    Not quite sure what all those words mean.
    Dynamical systems will never be solved in a semantic domain.
    IMO, they can be helpful the in robotics stack, from planning level up, but that's it.
    mdp2021 · 1791182706 · focus · HN ↗
    > Not quite sure what all those words mean
    What exactly is not clear? I will rephrase.
    The internal process of the blackbox oracle determine the reliability of the output.
    Two abilities are very different: learning trajectories through empirical training ("increasingly catching thousands of thrown balls"), and determining trajectories through computation (a rational thinker at work). The former is a finetuned parametrized engine (a «NN that learns a skill»), the latter is an Analyst. The former is fuzzy, the second deterministic.
    In front of fuzzy LLMs, which use potentially misleading outputs - text ("has it guessed or has it thought?") - the urgency of warranties of reliable output gets evident.
    So, that they «simulate cognition [only] well enough» (Intended wrote), and that there are «fundamental limitations of models ... purely text based» (Kooi wrote) raises the urgency to overcome the "fuzzy" and achieve the "deterministic" - it is not that we can stall on a «fundamentally not the right tool for that».
    Inventing an oracle calls for urgent striving to overcome the original weakness.
    jcoq · 1791126777 · focus · HN ↗
    Actually, I think my fundamental challenge with AI is that it has no common sense. The way it builds things, writes, and operates is out of touch with reality.
    Incidents like hugging face are partly rooted in the lack of common sense. It still functions like a supercharged toddler.
    I'd love to overcome this because it'd mean I spend less time guiding the the LLM to produce usable outputs.
    shagie · 1791130886 · focus · HN ↗
    [delayed]
    varjag · 1791127303 · focus · HN ↗
    Last week I asked a frontier model draw me a backplane PCB and it placed daughterboard slots side by side in a chain.
    customguy · 1791132837 · focus · HN ↗
    > A statement that we can hopefully all agree has been thoroughly refuted.
    Uh, no? So much of what we learn and take for granted as common sense is not learned via language, and not even expressible in it.
    cztomsik · 1791149877 · focus · HN ↗
    nothing indicated otherwise at the time. IMO he just underestimated RL-scaling. chinese models improved a lot too, they are not parrots anymore, there's some real intelligence, at 27B params.
    consider me optimist now, but just few months ago, even frontier models were dumb, doing stupid mistakes all the time, all of them were so dumb I'd never expect anything to change in just few months.
  4. lawandjustice · 1791124510 · focus · HN ↗
    Can you tell me what is the exact frequency of light hitting your eye as you read this comment? Not by guessing, not from knowledge, but from actually counting? No? Then you are not generally intelligent :)
    lefra · 1791135652 · focus · HN ↗
    All the frequencies, in varying amounts. Next question, please.
    mdp2021 · 1791149673 · focus · HN ↗
    Justify your statement (the other similar post nearby is not sufficient), or realize that we are not talking about that.
    We can have adequate representations of light that are the instances over which we reason. Your simile is about perception, not about instancing ideas.
  5. ben_w · 1791133458 · focus · HN ↗
    To determine this, it would first need to be able to spell "raspberry" as letters rather than as tokens.
    Given you also don't want it to memorise [for all tokens, count([for all letters]), this would probably be more like "here's two images, count all things in the big image that look like the thing in the small image", which can then be r's in a photo of a raspberry jam jar in a supermarket, or dragons in a photo of a furry convention, or whatever.
    That said, they are competent enough at coding that I keep seeing them write code to do even simple tasks.
    On a related note: why did I see Claude editing a file by using cat to write a python script to do a grep search and replace?
    aesthesia · 1791140005 · focus · HN ↗
    > Given you also don't want it to memorise [for all tokens, count([for all letters])
    Why not? You've memorized how words are spelled, and how sounds correspond with letters, and how concepts correspond with words. To the extent that there are shortcuts that enable compression you use these, and the model will do something similar.
    ben_w · 1791142744 · focus · HN ↗
    Combinatorial explosion, and facts merely memorised is a huge waste of parameters that are better dedicated to effective reasoning.
    Being able to spell all the words then count letters is simpler, and more generalisable to other tasks, than memorising answers to all possible word questions.
    aesthesia · 1791152763 · focus · HN ↗
    Ah, I misunderstood what you meant. I was just trying to highlight that in order to answer these types of questions the model needs to memorize the spelling of each token. But you're right that that's all they need to memorize, and algorithms like counting are pretty simple for transformers to implement.
    mdp2021 · 1791147070 · focus · HN ↗
    > Why not?
    Because to "123x456" we want a reply that goes "this times that plus that...", not "Was that not nnnnnn?". If it does not perform its duty it is a liability.
    mdp2021 · 1791146845 · focus · HN ↗
    > it would first need to be able to spell "raspberry" as letters rather than as tokens
    Of any object in question they should be able to create a representation that allows correct assessment.
    > Given you also don't want it to memorise
    That is obviously necessary: what we want from the consultant is to check, not to remember. Answers must be correct and that implies having performed all due diligence - and being capable of doing it, before that. So, objects must be instanced internally in a way that allows effective handling. Counting letters is a good example of the ability (that must remain general).

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.