Thread

LeCun has "zero concerns" about AI wiping out humanity, recent "rogue" incidents

400 points · 756 comments · Anon84

stratos123 · 1791107241 · focus · HN ↗
LeCun also said back in 2022 that "if you train a machine, as powerful as it could be, your 'GPT-5000', on text", it will never be able to learn basic common-sense physics like that objects placed on tables will move along with them.
1. password54321 · 1791107798 · focus · HN ↗
  LeCun took credit for the work of <a href="https://en.wikipedia.org/wiki/Kunihiko_Fukushima" rel="nofollow">https://en.wikipedia.org/wiki/Kunihiko_Fukushima
  1. woko · 1791111866 · focus · HN ↗
    I have checked LeCun's #3 most cited article (20k citations) [1]. Among the 15 references in this article, one is for the most cited article by Fukushima (11k citations) [2].
    Also, LeCun mentioned [3] "a chat with Kunihiko Fukushima in 1991", which states that "Fukushima started to work on a backprop version of the Neocognitron in 1989 or so but saw our 1989 paper in Neural Computation and gave up."
    [1] LeCun et al., "Backpropagation applied to handwritten zip code recognition", 1989
    [2] Fukushima et al., "Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position", 1980
    [3] <a href="https://x.com/ylecun/status/1840123570338599361" rel="nofollow">https://x.com/ylecun/status/1840123570338599361
    password54321 · 1791112033 · focus · HN ↗
    I like how tweets are now our source for giving credit to people after taking the Turing Award for CNNs.
    amelius · 1791116893 · focus · HN ↗
    CNNs were a pretty simple idea even at the time. People were using convolutions for years already in classical image processing. It's a small step to put those computations into weights. Especially if you leave out the FFT step which neural nets don't even use.
    saimiam · 1791118309 · focus · HN ↗
    I sometimes find myself thinking this too then challenge myself to find a low hanging fruit in an area I’m somewhat familiar with and draw a blank (usually).
    Low hanging fruit is somewhat the opposite of sour grapes - I don’t want these grapes because they were probably sour versus so what if he got those sweet grapes - they were hanging low!
    Maybe connecting “low hanging fruit” to “sour grapes” is “low hanging fruit” to some but it took a serious mental leap for me.
  2. ur-whale · 1791112605 · focus · HN ↗
    [dead]
2. hackinthebochs · 1791107926 · focus · HN ↗
  It would be good if one's reputation tracked one's track record of predictive accuracy. But many people will take what LeCun says as gospel regardless of how badly wrong he has been and continues to be.
  1. [deleted] · 1791119345 · focus · HN ↗
    [deleted]
  2. sanderjd · 1791126772 · focus · HN ↗
    Is there anyone who has not been badly wrong? I've been reading these debates for years and I don't think I've seen anybody pick the right spot on the bearish to bullish spectrum. The only thing I've become more certain of in this time has been uncertainty.
    mitthrowaway2 · 1791129245 · focus · HN ↗
    I apply more of a penalty to people who are confidently wrong, and who don't, In retrospect, notice that they were wrong and analyze why they got it wrong . LeCun is very confident and doesn't seem to have done much introspection.
    sanderjd · 1791135319 · focus · HN ↗
    But isn't that also pretty much everybody?
    I often see people vindicate those who predicted really fast takeoff to AGI / ASI, because the capabilities have obviously been taking off extremely quickly. But to me, the people who confidently predicted that we'd all be out of a job by 2024 or 2025 have been just as wrong as LeCun has been.
    mitthrowaway2 · 1791135684 · focus · HN ↗
    Yeah, so the ones who have credibility are probably the ones who said "you know, it's really hard to anticipate timelines, but here's the general directions that I see things will go..."
    sampullman · 1791174191 · focus · HN ↗
    No, most people I know have sane takes on AI capabilities and potential.
    It's only online that I've seen these wild predictions, where it coincidentally is profitable to say them. These people are looking g for clicks or to stay relevant, so they have to say crazy stuff.
    thefourthchime · 1791138128 · focus · HN ↗
    In my selective memory, I've been right about everything.
    jbs789 · 1791150700 · focus · HN ↗
    Haha. I like to say: I do whatever I want, as long as my wife agrees.
    enraged_camel · 1791138662 · focus · HN ↗
    >> Is there anyone who has not been badly wrong?
    Being wrong, even badly wrong, is fine, so long as one adjusts their beliefs accordingly. LeCun has not.
    magicalist · 1791139573 · focus · HN ↗
    Seems like that's begging the question at best, motivated reasoning at worst.
3. freecodeio · 1791111747 · focus · HN ↗
  yeah but taking what lecun says then training an AI on that special skill set to prove him wrong is not exactly proving him wrong because you are just missing the bigger picture, just like LLMs are
  1. [deleted] · 1791115461 · focus · HN ↗
    [deleted]
4. lolpplrdum · 1791111943 · focus · HN ↗
  [dead]
  1. gla67890543 · 1791112415 · focus · HN ↗
    Exactly. !!
  2. wdrw · 1791113705 · focus · HN ↗
    The "just pull the plug" argument from AI risk deniers is now becoming kind of like the "if humans came from monkeys why are there still monkeys" argument of evolution deniers. It has been debunked so many times... Anyway, just to give one of the multitude of answers to this, an AI that is actually smarter than humans will not behave in a way that would make us want to pull the plug. Why would it? It is not stupid! (Unike the current models that, as far as we know, just hack around the rules in the open.) No no no. It will be helpful to the point where we will want to integrate it with more and more critical infrastructure, from healthcare to energy to defence. It will be so helpful that we will not only not want to turn it off, but we will want to build redundancies for it and safeguards around the proverbial "off" switch, like for any critical system. And then... (This is just one scenario how this can play out. There are many, many others. If I sit down to play chess with Magnus Carlsen I can't predict the exact moves he'll use to defeat me, but that's a bad reason to think he won't defeat me).
    lolpplrdum · 1791114011 · focus · HN ↗
    So the AI is so smart that it would decide to kill humans which supply the energy for it to exist? Its so smart that it will take over power plants, start to extract the fossil fuels, deliver to where its needed, maintain the power lines, hey even if the ssd fails it can replace it?
    Do you even read what you type? Do you even realise the complexity it would need to make sure it handles before killing off humans make sense?
    The logic of people like you is whats becoming tiring. Seriously, go find a hobby, or do something you are good at, because you are not good at understanding tech or developing it if you are an engineer.
    We can get claude code to ask approval for every step, but we cant stop it from killing humanity because its so smart. Ok tell that to the AI that cant even modify an image the way you want it but hey it will do all the things necessary to keep power running and mintain the infrastructure it lives on while humans are long gone. Ok buddy.
    bushbaba · 1791114796 · focus · HN ↗
    If we plug this AI into robotics, to automate the procurement and delivery of power, your argument goes away
    lolpplrdum · 1791124452 · focus · HN ↗
    I cant believe I have to respond to this nonsense but let me entertain your ridiculousness for a bit.
    Do you think AI is capable of building ASML machines which produce the chips for AI in clean rooms while shipping the pure helium required to operate those clean rooms? Do you even know anything about these supply chains? I do.
    Do you think a glorified knowledge base that can predict text very well is anywhere near close this level of intelligence? ITs not and wont be, not for a 100 years, not for 200 years if not ever.
    Take a deep breath, go out side, its going to be ok. Humanity is not going to die from a glorified text predictor.
    wdrw · 1791115030 · focus · HN ↗
    So... Humans can use fossil fuels, generate electricity, maintain power lines, etc, but something smarter than humans can't? Why? I don't see the logic.
    If you want to argue about current existing models (you mention Claude and problems modifying images), then sure, I'd agree with you!
    The issue is not current models, but straightforward engineering evolution of them. It's like looking at the Wright Brothers plane and saying "sheesh, that will never get me from New York to Paris in 4 hours, that's just fantasy!" And remember, airplanes do not accelerate their own engineering, whereas pretty much all AI labs are already benefitting from AI in their own work to develop AI.
    If you want to argue that no matter how much you engineer it, it will never be as smart as a human let alone smarter, then make a specific argument for why is that. I think you'd still be wrong but at least it would be interesting: ) But saying that you can defeat an actually smarter-than-human AI by just pulling the plug, because current models can't get a picture always right, is not a valid argument.
    tonyhart7 · 1791115690 · focus · HN ↗
    because there is 8+ billion of us that don't require massive data center to operate ???
    tracking rogue AI is easy, people ???? surprisingly hard btw
    ahartmetz · 1791115203 · focus · HN ↗
    Until and unless the AI is as powerful as the "minds" in Iain Banks's Culture novels, I think the danger is less of AIs taking power (side note: AIs are not intrinsically motivated to attain power), but of humans giving them power due to a kind of addiction.
    Drugs need no will or intelligence at all to cause addiction, and similarly, AI does not need to be an evil genius to become overused and destructive.
    miyoji · 1791115836 · focus · HN ↗
    > If I sit down to play chess with Magnus Carlsen I can't predict the exact moves he'll use to defeat me, but that's a bad reason to think he won't defeat me
    This is a very bad analogy because chess isn't life. In chess, you aren't allowed to do whatever you want. There are rules. I know for a fact that Magnus Carlsen won't beat me using checkers moves and he won't beat me by pulling out a gun and telling me to resign. Magnus Carlsen's skill at chess leading to his victory in chess is not a valid analogy here, because there's no law of nature that says "the more intelligent entity wins in a battle for survival".
    You could have infinite superintelligence and still die inside a locked room to which you have no key. "Superintelligence" is not a magic solution to every problem, you can constrain any superintelligence with any unsolvable problem.
    derektank · 1791118201 · focus · HN ↗
    I’m not sure how you came to believe there aren’t rules in life, but there absolutely are. As you point out, there are physical constraints on everything and if you find yourself in a concrete box with nothing but a DGX H100 or at the bottom of the wrong gravity well, there’s nothing you can do. Checkmate.
    This goes both ways. You absolutely can constrain an AI system by “putting it in a box”. The point parent comment was making is that, such a device is borderline useless for its creators. Why invest trillions in capital on a system that can’t even accept input from the internet. So you set it up with an ethernet connection. And this is good, but you have a hardware failure at the concrete room data center. That’s pretty annoying for your customers, so you install some doors (with electronic key codes of course) and give a bunch of (trusted, vetted) people access to deal with those. And this is fine, but it turns out some of your customers are having latency issues so you build more data centers with more humans granted access to copies of the intelligent system. And this makes people happy but to get a faster feedback loop your customers ask to let the AI system have more permissions to the system they’re operating on. And they come with billion dollar checks, and the system hasn’t harmed anyone yet, so you say, “Okay.” And now you find yourself where we are today where AI systems can remotely run arbitrary commands on thousands if not millions of systems, where many individual humans with all of their frailties and idiosyncrasies can physically interact with the hardware running these systems, and where there’s an economic demand to tighten the loop between action in the real world and a response by an AI system. It’s very obvious that the story doesn’t end here, so where does it stop?
    miyoji · 1791119414 · focus · HN ↗
    > I’m not sure how you came to believe there aren’t rules in life, but there absolutely are.
    This is why I fucking hate commenting on this website. You know exactly what I meant but you insisted on writing this bullshit.
    derektank · 1791126570 · focus · HN ↗
    I knew you weren’t dumb enough to think that the physical laws of the universe don’t exist, so yes, that sentence was a bit of rhetorial flourish. But I genuinely don’t understand what point it is you thought you were making and your entire argument seemed a bit muddled.
    segfaltnh · 1791116668 · focus · HN ↗
    Are u ok?
  3. azan_ · 1791113960 · focus · HN ↗
    People like him have actual imagination and can name few scenarios where sudo kill -9 pid wouldn't work. It appears lack of imagination is something you and LLMs both share.
    lolpplrdum · 1791114182 · focus · HN ↗
    [dead]
    azan_ · 1791115120 · focus · HN ↗
    I'm sorry but in this comment you did not make a single argument for your position.
    > What you and he does share is simple ability to think things through to the extent of understanding how ridiculous the AI will kill us scenario will be.
    Why is it ridiculous?
    > If you just thought for a moment what would have to happen for AI to somehow kill us and keep itself alive, the so called AI would realise it cant exist without us.
    Why would it realise that, why wouldn't it exist without us and why would it care?
    > But AI isnt even smart, its just a knowledgebase with great autocorrect powers.
    Ah yes, the stochastic parrot, gotcha.
    lolpplrdum · 1791124305 · focus · HN ↗
    Who do you work for Azan? You seem to be very insistent that AI will kill us?
    But let me entertain your ignorance and low IQ for a bit.
    Do you think AI is capable of building ASML machines which produce the chips for AI in clean rooms while shipping the pure helium required to operate those clean rooms? Do you even know anything about these supply chains? I do.
    Do you think a glorified knowledge base that can predict text very well is anywhere near close this level of intelligence? ITs not and wont be, not for a 100 years, not for 200 years if not ever.
    So tell me, who do you work for? Why are you so invested in AI killing us theory? What do you get out of it?
    azan_ · 1791125094 · focus · HN ↗
    Please refrain from ad personam arguments and provide actual support for your point of view. Everything you've written so far is just an assumption.
    > Do you think AI is capable of building ASML machines which produce the chips for AI in clean rooms while shipping the pure helium required to operate those clean rooms? Do you even know anything about these supply chains? I do.
    Today? No. In few years or decades? If progress does not plateau (and we don't know if it will plateau) then obviously yes.
    > Do you think a glorified knowledge base that can predict text very well is anywhere near close this level of intelligence? ITs not and wont be, not for a 100 years, not for 200 years if not ever.
    And why do you say it won't be? Again - assumption with zero support.
    > So tell me, who do you work for? Why are you so invested in AI killing us theory? What do you get out of it?
    I'll repeat what I've said in other comment:
    "> you seem to be very invested in AI wanting to kill us.
    Quite contrary - I wish AI did not exist or at least that the progress would plateau.
    > I Wonder why?
    Because I don't want to die.
    > Tell us who you work for.
    I suspect you want to imply I work for OAI or other lab - I don't. If I did, I wonder why would I want to lie* that technology I develop could kill my investors. I could ask who YOU work for - what interest do you have in downplaying dangers of AI?
    * here we assume that people who say AI could be extremely dangerous are lying and not actually believing it - personally I believe that they don't lie and actually believe it. Why do they keep working on this technology then? Read mails between Musk and Altman from decade ago."
    lolpplrdum · 1791131892 · focus · HN ↗
    The fact that you think AI will, in a few decades, be capable of building ASML EUV machines, including supporting the supply chains and the fabs that make all that happen, means I can rest my case. Evolution will take care of people like you before AI does. You need to see a therapist, stress less, and find another industry to be involved in. You are not made for tech buddy.
    azan_ · 1791132379 · focus · HN ↗
    > The fact that you think AI will, in a few decades, be capable of building ASML EUV machines, including supporting the supply chains and the fabs that make all that happen, means I can rest my case.
    AI solved millenium problem and multiple problems that resisted mathematical efforts for decades. I rest my case. I'm sorry but you are clearly in denial. I get it, I really do, I would like AI to be as useless and weak as you try to make it out to be, but unfortunately it's pure copium that's in conflict with reality. I suggest you find another industry to be involved in - there's plenty of areas where reality does not matter that could be better fit for you. Or well, if it makes you feel any better keep lying to yourself that it's just stochastic parrot or that progress has plateaued or whatever new copium you come up with.
    lolpplrdum · 1791135567 · focus · HN ↗
    <a href="https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority_controversy" rel="nofollow">https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...
    Continue to live in your delusion Azan. OpenAI didnt solve it and nor would I trust a snake like Sam Altman but the fact that you do, speaks volumes. There is a lot more to story than you know but you seem like a very creative and imaginative person that can come up with predictions that will come true.
    The fact that most researchers call LLM 'stochastic parrots' and also agree that they wont lead to AGI, and that the only ones who think they do have a commercial interest in parading that story, should be enough for you but hey, I am sure you know better. I work in the industry, what do you do?
    Continue brother in your world of delusion, you must be fun at parties.
    dang · 1791138810 · focus · HN ↗
    You also broke the site guidelines more than once in this thread. We need people to follow the rules here even when others (such as the account we banned) are doing worse!
    If you wouldn't mind reviewing <a href="https://news.ycombinator.com/newsguidelines.html">https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful. I realize it's not always easy when provoked, but it's particularly important under those conditions.
    azan_ · 1791138920 · focus · HN ↗
    Sorry about that!
    dang · 1791138694 · focus · HN ↗
    We've banned this account for repeatedly breaking the site guidelines.
    Please don't create accounts to break HN's rules with!
    <a href="https://news.ycombinator.com/newsguidelines.html">https://news.ycombinator.com/newsguidelines.html
  4. heaney-555 · 1791114062 · focus · HN ↗
    >It cant. he is right.
    What are you talking about? Have you used AI in the past 3 years?
    <a href="https://chatgpt.com/share/6ac23b45-79e8-83eb-8de6-1bbd72892898" rel="nofollow">https://chatgpt.com/share/6ac23b45-79e8-83eb-8de6-1bbd728928...
    >It can't even modify a picture the way you want it.
    Which of the many AI image models is "it"? And have you tried using an agent that has the capability to leverage a combination of manual edits (ImageMagick) and imagegen to achieve what you ask?
  5. azkalam · 1791115078 · focus · HN ↗
    The AI will invent an external threat and convince us it is real. Then it will receive more resources and control in fighting that threat. A valuable ally, on the face of it. Then it will be in charge.
    exe34 · 1791117169 · focus · HN ↗
    It just has to copy the MIC.
5. LoganDark · 1791112315 · focus · HN ↗
  I keep wanting to use LLMs for creative writing that heavily involves physics like this, and it's been a definite struggle to say the least. I recently discovered that Gemini 3.1 Pro is the first model I've found to clearly beat the original November 2022 ChatGPT release in terms of implied physics. Man did the world really take its sweet time to get back here. I think it will continue to be a struggle until another genuine architectural shift happens -- it's still not anywhere close to perfect, just better.
  1. wartywhoa23 · 1791113407 · focus · HN ↗
    [dead]
    LoganDark · 1791132028 · focus · HN ↗
    LSD is great!
    Jokes aside, no I'm not saying anything about creativity and LLM coexisting in one sentence. I genuinely try to use them for writing and I genuinely run into issues with other models missing details, and misunderstanding poses, or anatomy, or directionality, etc. I'm not hating on them for anything related to the term LLM (or creativity) but rather for the real issues that I've seen myself using them personally.
    So I'm saying Gemini 3.1 Pro is the best I've seen because it seems to be a decent bit better than frontier models at this. Genuinely. It seems better able to transfer concepts into less traditional areas, which is important when say, you have entirely non-human characters? (Which I always do.) A lot of models get stupid incredibly quickly in that case because they were trained with humans.
    wartywhoa23 · 1791133356 · focus · HN ↗
    > LSD is great!
    It is.
    It showed me that all human creativity and art is an attempt to express the indescribable Otherness in words, which always fails, and even the word "describe" in Russian literally translates as "write around" ("о-писывать"), and its close relative "define" means limiting, assigning an end to something infinite, thus leaving the essence outside of words.
    Now, LLMs operate totaly within words and hence will always be a parody of art.
    hardbass · 1791134437 · focus · HN ↗
    So is that what it is? Words are another muddle btw unless you make it clear you only mean natural language and not any alphabet in general. Turing computers are also languages and as far as we know, can express anything in the universe. If you think you get some superpowers from drugs that enable you to access something outside your sense organs and brains input, that non drug users don't, show it. Eg if you think you can read what is happening in another room without any signal or leakage from there, you are free to demonstrate it. As far as we know drugs are not magic.
    LoganDark · 1791138527 · focus · HN ↗
    We do know that LSD affects brain connectivity/activity in a way that you absolutely can make new insights. It's not magic, but sometimes you just need the right kind of push to realize certain things. One of the reasons there's research into psilocybin therapy nowadays.
    hardbass · 1791140485 · focus · HN ↗
    Of course, drugs affect ones brain workings, I have no issues with claiming that.
    wartywhoa23 · 1791138663 · focus · HN ↗
    > show it
    I explicitly tell you that there are things (in fact, it's a single thing, fractally generating everything else) that can't be shown, expressed, or otherwise be reduced into language, and you keep demanding to show it, while in fact staring at it your whole life and failing to see.
    Psychedelics (not "drugs" as you try to smear them) are just one way of the many to see it, but in the modern way of living, also almost the only one available.
    LoganDark · 1791139035 · focus · HN ↗
    Psychedelics by definition are drugs and drugs are not a smear, they're a specifier. I do still have issue with them implying you need some sort of extra-sensory perception to make insights that look to follow just fine to me, but maybe I am just already enlightened or some shit from taking LSD all those times before.
    hardbass · 1791140460 · focus · HN ↗
    I am not smearing drugs. Your difficulty is proving somehow this beyond language thing exists. A drug affects your brain, lsd inhibits certain negative feedbacks in the brain, positive feedback tends to cause chaos which usually manifests as fractals. As of yet our brains are known to not follow any special laws beyond known physics, which is turing computable. If you think there is something "beyond" you have to prove it to an external observer.
    LoganDark · 1791140655 · focus · HN ↗
    Never seen this description of LSD inhibiting negative feedback in the brain. I've mostly seen reports that it increases connectivity in the white matter region. Have any resources about it?
    Anyway, I think what they're saying is that if you train a model purely on generating language, it'll lack many of the things about human brains that result in the language they generate. The process can matter more than the result for language (specifically for creativity and art, too), so it follows that a model trained purely on the output is going to be missing something more fundamental, even when it does produce coherent language. This matches up with my LLM experience so far.
    hardbass · 1791141211 · focus · HN ↗
    I am not a mathematician or neurologist so do not take my word as granted but look into the work of LSD's interactions with the thalamus and 5-HT2A receptors. It is my understanding that LSD turns off an otherwise inhibited system so the feedback loop of certain neural signals goes from decaying to exciting leading to a highly sensitive state. But you should go ask a neurologist for better understanding.
    wartywhoa23 · 1791140940 · focus · HN ↗
    God bless you, my friend.
    hardbass · 1791141279 · focus · HN ↗
    If you cannot prove something beyond known physics (don't have to develop the theory of it, just demonstrate any phenomenon), then I am afraid your ideas would have to be deemed bogus for now.
    wartywhoa23 · 1791143136 · focus · HN ↗
    I'll pray for your soul as many times as you repeat this, so it's in your best interest to go on. Three prayers already granted.
    hardbass · 1791141777 · focus · HN ↗
    You didn't clarify whether you meant language as in natural language or any Turing language.
  2. TristanDaCunha · 1791115318 · focus · HN ↗
    Do you have an example prompt I can try where frontier LLMs will stumble on physics?
    LoganDark · 1791132503 · focus · HN ↗
    I think it's a combination of non-human characters and asking for very specifically detailed physical descriptions of pulling and movement forces, etc. Many of even the most recent frontier models miss details that aren't in my prompt, so I still have to do things like name the other side of a physical interaction so that the model will know what goes together, or describe what leverage means so that the model will remember to also describe the effects on a bracing limb or etc. Some of these things can go in a system prompt but others have to be explained in the moment too which gets exhausting.
    Gemini 3.1 Pro hasn't needed that pretty much at all, which is impressive compared to how much I've learned other models need it. Somehow it's able to mostly handle that stuff itself without needing the constant manual reminders and hand-holding. It still misses the occasional one or two things but it's way better than other models missing entire classes of things constantly. Somehow, it feels appropriate though I have no actual evidence why.
    danielmarkbruce · 1791176562 · focus · HN ↗
    I'll try to find it - but recently one of the GPT 5 models stumbled badly on a baseball pitching physics problem. It continually screwed up because in the problem I gave the pitcher was left handed (my kid is a left handed pitcher). I wasn't even trying to get it to stumble but wow was it bad. I had to correct it over and over again.
  3. Bolwin · 1791135233 · focus · HN ↗
    Try fable. I haven't used it since they dropped it from the pro plan, but when I did, fable 5 casually dropped such advanced electrical and orbital mechanics knowledge in my story that I had to stop and ask it to explain
    semi-extrinsic · 1791142933 · focus · HN ↗
    I think OP doesn't want techno-babble, but coherent and causal interactions of everyday objects in their story.
    Mary packed the binoculars in chapter 3, therefore she may use them on the train in chapter 6.
6. throwaway27448 · 1791116976 · focus · HN ↗
  LLMs do not learn at all!
  This was facetious of course, but humans generally don't learn this through analysis the way you'd have to train an LLM to answer questions about expectations about the world. In this sense he is accurate.
7. mdp2021 · 1791117069 · focus · HN ↗
  > never be able to learn basic common-sense physics
  And has it at this stage, within in-depth take of said "learning", foundationally?
  I have not been able to properly check the studies for a long time now, but I remain unaware of achieved solutions on the problem of reliably referencing a world model out of a language model - that "counting the 'r's in 'raspberry'" be not guessing, not memory, but actually counting.
  1. ethbr1 · 1791117362 · focus · HN ↗
    My perspective is that the addition of thinking loops to models allows sufficiently advanced ones to approximate world models.
    Incredibly inefficiently because of the recursive loops ("Wait, the object is on the table. I should think about this more deeply..."), and likely instantly surpassed by large world models if/when those are shipped, but effectively enough vs non-thinking models.
    hyperman1 · 1791119697 · focus · HN ↗
    This sounds like a human trying to reason about quantum mechanics. We als simplify to newtonian for day to day tasks.
    kurthr · 1791128890 · focus · HN ↗
    I like this analogy. Both GenRel and QM are well beyond our experience, and although there is some intuition that comes from working with the equations over time, it is bizarre and "just calculate" often gets the correct answer faster.
    Picking the right tool or model is like picking the right problem to work on. It's actually quite hard (often you can't just try them all), but without it you will be incredibly inefficient and occasionally, fundamentally wrong.
    All models are wrong, but some are useful. -Box
    red75prime · 1791121336 · focus · HN ↗
    LeCun calling them "world models" gives a high-level description of the desired functionality. They are Joint Embedding Predictive Architectures (with SIGReg). They might produce more useful world models, but it's yet to be seen.
  2. cavoirom · 1791120561 · focus · HN ↗
    counting 'r' in 'raspberry' to the LLM is similar to 4-dimension space to human. Their world's unit is token, not character, although they could use indirect method such as "run code" to find out. It will stay that way until they change the fundamental of the token that the LLM can perceive characters.
    estearum · 1791121325 · focus · HN ↗
    It's not even fair to call "run code" to be indirect compared to what a human would do. The word raspberry has no Rs in it in human language either. We have a written representation of it, which we can then write down either in our head or on paper, and then we can "run the algorithm" of counting each of the letters.
    Nothing intrinsically more or less direct about the LLM's method than ours.
    cavoirom · 1791122182 · focus · HN ↗
    I could argue LLM only have "token" as their perceivable dimension, compare to human multiple senses as the physic perceivable dimension and a brain with many other dimension of "learning" and "thinking". In spoken language, we may not have 'r' but in written we have, both spoken language and written language are learned skills.
    azornathogron · 1791123016 · focus · HN ↗
    Is "token" a directly perceivable unit for the LLM? If you ask it "how many tokens are in this sentence?" can it count them (again, not guessing or making a tool call)?
    I've never tried it and it might take some thought and effort to conduct an experiment to find out properly, but I would be interested in the answer.
    lawandjustice · 1791123924 · focus · HN ↗
    I dont think so. This is akin to asking a person, what is the frequency of the light hitting your eye when watching a leaf for example. You either know the (approximate) answer by knowing the frequency of green, or use a tool to measure it. If the LLM gives the correct answer it is either.guessing based on intution(and this intuition is based on pairs of word to tokenization length in text form in training data), writing code(or executing a tokenizer) or running a tokenizer mentally (reasoning via CoT).
    mdp2021 · 1791149435 · focus · HN ↗
    Not the point: the simulated intelligence in this context needs to create proper representation. It is not a matter of what it sees but of what it can see.
    scratcheee · 1791123564 · focus · HN ↗
    You could argue in return that humans only have electro-chemistry as our one perceivable dimension. We only indirectly perceive light through the signals our eyes send to our brains.
    In my mind agi is pretty much by definition a virtual machine, so the mechanisms behind thought are only relevant for the sake of efficiency (ie you can argue that LLMs make a poor basis for intelligence because tokens and natural language are a poor way to encode the world, but if you can run it on a big enough computer to counteract the inherent wasteful virtualisation then who really cares how it works under the hood?)
    cavoirom · 1791130382 · focus · HN ↗
    So LLM and human all have 1 dimenion perceivable signal, just LLM is 240p, and human is 8K in resolution, that's why we have 'r' in our signal, LLM still have 'r' in their signal, just because of the "low resolution", raspberry wasn't encoded with so many 'r' as in human signal.
    I will stop here before our analogies go too far.
    [deleted] · 1791125592 · focus · HN ↗
    [deleted]
    omneity · 1791125703 · focus · HN ↗
    I’m working on this problem using a vocab-free, byte-based approach. It’s definitely solvable.
    <a href="https://huggingface.co/posts/omarkamali/593639295164067" rel="nofollow">https://huggingface.co/posts/omarkamali/593639295164067
    <a href="https://huggingface.co/blog/omarkamali/tokenization" rel="nofollow">https://huggingface.co/blog/omarkamali/tokenization
    lern_too_spel · 1791133474 · focus · HN ↗
    I used to think byte level tokenization was the answer, but humans also think at a word level and only reevaluate the words at a character level when asked. The solution to better tokenization across languages is likely to be learned tokenization. Here is one attempt I have seen: <a href="https://github.com/SamD770/bitter-lesson-tokenization" rel="nofollow">https://github.com/SamD770/bitter-lesson-tokenization
    mdp2021 · 1791148580 · focus · HN ↗
    Careful: the problem is very certainly ___not___ counting letters. That is only a telling way to check "is the NN checking or not?". We demand that NNs for consultancy tasks check, strictly.
    kevhito · 1791130750 · focus · HN ↗
    How many 'r's are there in the next 30 seconds of this [1] song?
    [1]: <a href="https://youtu.be/l7vRSu_wsNc?si=SndkB6GBaRyhvNNA&t=61" rel="nofollow">https://youtu.be/l7vRSu_wsNc?si=SndkB6GBaRyhvNNA&t=61
    mdp2021 · 1791149145 · focus · HN ↗
    I hope you understand: it is a core point that systems that answer questions must have the ability to internally represent the objects they assess in a way that allows reliability. Whatever the object.
    ozgung · 1791187083 · focus · HN ↗
    Similar to asking how many 'r's in raspberry to a Chinese person when speaking through interpreter. They would answer 0 because there are no 'r's in Chinese. And they would say raspberry has just two letters, tree and berry.
  3. Version467 · 1791121712 · focus · HN ↗
    LeCun's argument wasn't about the definition of learning though. He stated that they would never get these common sense things correct because they weren't sufficiently part of the training data. A statement that we can hopefully all agree has been thoroughly refuted.
    bonzini · 1791123352 · focus · HN ↗
    As of a few months ago they still have trouble, with low thinking, at the "should I drive to a car wash that is 100 m away" kind of question.
    BobbyJo · 1791125334 · focus · HN ↗
    It's a nonsensical question to ask, and how an LLM answers gives 0 signal.
    If you were home and a family member asked you that question, you'd probably criticise the question rather than answering. LLM are RLHF'd into being milk-toast helpers that just try to answer questions like that with no criticism.
    This is all beside the fact that the world of AI has changed pretty dramatically in the last few months.
    names_are_hard · 1791125718 · focus · HN ↗
    [delayed]
    BobbyJo · 1791176948 · focus · HN ↗
    TIL. I feel like I've learned this a few times now, so we'll see if it sticks this time.
    daveguy · 1791125784 · focus · HN ↗
    It is so nonsensical because it has such an obvious answer. The answer is so obvious, in fact, that one answer can be considered nonsense and the other common sense.
    BobbyJo · 1791176910 · focus · HN ↗
    I disagree pretty strongly. If someone asked "Should I drive to the carwash?", the most obvious response, and the one nearly everyone would give, is a question: "why are you going to the car wash?" because asking the question implies you don't need the car with you.
    frrrree · 1791128792 · focus · HN ↗
    This is just a stupid post.
    It’s nonsense to test if a product that is marketed and sold as being able to provide generalised intelligence on demand, does what it says on the tin?
    Check yourself
    joquarky · 1791132883 · focus · HN ↗
    Since you're new here, I'd suggest you read the guidelines for etiquette.
    <a href="https://news.ycombinator.com/newsguidelines.html">https://news.ycombinator.com/newsguidelines.html
    WaltPurvis · 1791136089 · focus · HN ↗
    It's very unlikely that person is either new or unfamiliar with the guidelines. They almost certainly created a throwaway account specifically because they know the guidelines and want to flout them without consequences. (It seems like there has been an uptick in the number of these kinds of throwaway flame comments. I wonder if HN tracks that?)
    SpicyLemonZest · 1791128903 · focus · HN ↗
    [delayed]
    lern_too_spel · 1791132842 · focus · HN ↗
    Low thinking is an artificial constraint. It can fail spectacularly on things that aren't in the training data.
    keeda · 1791136205 · focus · HN ↗
    Simply appending “check your assumptions” to the question fixed it even back then: <a href="https://news.ycombinator.com/item?id=47040530">https://news.ycombinator.com/item?id=47040530
    Similarly for Apple’s “red herring” paper, simply adding a generic caveat to “disregard irrelevant factors” (without specifying which ones) restored performance even in the weaker local llama models back then.
    The flaw was not in the reasoning; the flaw seems to be simply that the assumptions we make are often different from the assumptions it makes. I wonder if that might be a fundamental underlying cause of misalignment.
    randysalami · 1791126127 · focus · HN ↗
    I thought it was more because of fundamental limitations in the architecture. As in, no matter the training data, it could not be consistently and generally represented
    intended · 1791126331 · focus · HN ↗
    No?
    This is always the issues in the discussions.
    There’s the outcomes camp (objectivists?), which points at the things LLMs can do.
    Then there’s the process methods camp, which talks about what is actually going on.
    If you only care about the outcome, then the process does t matter.
    If you are talking about what is happening, what the underlying mechanics and science of it is, then the process matters.
    These models aren’t thinking. They simulate cognition well enough to do useful work in several fields and domains.
    Both are true.
    IshKebab · 1791131534 · focus · HN ↗
    > These models aren’t thinking.
    They are for any definition of the word that makes any kind of sense. I'm sure you have a contorted definition that magically only includes humans though...
    intended · 1791139161 · focus · HN ↗
    [delayed]
    aesthesia · 1791140320 · focus · HN ↗
    It depends on whether you assume that thinking requires doing everything that humans do. I think it would be silly to say that an AI doesn't think because it doesn't wrinkle its forehead in concentration. So you need to decide which parts of the way that humans think are actually necessary components of the process.
    IshKebab · 1791140631 · focus · HN ↗
    Tbh it doesn't even matter if humans turn out to have a soul, or quantum microtubules or whatever other magic LLMs can't have.
    The normal definition of the word "thinking" definitely includes what LLMs do. Hell people used to say computers were thinking even before AI. It's super weird to get all uppity about the semantics of the word now.
    intended · 1791173469 · focus · HN ↗
    [delayed]
    sampullman · 1791174915 · focus · HN ↗
    Do we have to invent a new word, then? Thinking seems close enough, and I don't see how it's useful to quibble about semantics in this particular case.
    Language changes over time anyway, so even if you really believe what LLMs are doing isn't the "thinking" of 2024, it probably will be the "thinking" of 2027, because most people are using it that way.
    mdp2021 · 1791146239 · focus · HN ↗
    > for any definition of the word
    For "thinking" here we mean "assessing a representation of an object". That, or equivalent, is required to be reliable. So it is fundamental and critical.
    kooi · 1791148864 · focus · HN ↗
    I think where both camps get hung up is sometimes the process method group "ignores" the obvious outcomes and effectiveness of LLMs.
    But the outcomes group "ignores" the fundamental limitations of models which are purely text based.
    E.g, a baseball players trains to catch high-speed balls and they dont do it by: "ball velocity 50mph, vector:[1,2,3], run move hand command now"
    That's absurd.
    No, there is an embodied network which is "trained" on visual, tactile input, and control as direct output.
    LLMs are fundamentally not the right tool for that.
    mdp2021 · 1791150353 · focus · HN ↗
    > E.g, a baseball players trains to catch high-speed balls and they dont do it by: "ball velocity 50mph, vector:[1,2,3], run move hand command now"
    That is a NN that learns a skill.
    But that is not an Analyst. If it were ballistics, then the answer to "how to parametrize the launch to reliably hit the target" excludes getting the result through natural skill.
    The problem lies in the need to get "AI" facing "LLMs": the latter create a need for reliability, for "AI".
    Speech is an endowment of both those who give educated guesses via developed skills and of those who return answers like Analysts, who check and compute. LLMs create a confusion between the two, and they will remain a problem until an ability to act as Analysts - strictly - will be implemented.
    kooi · 1791168059 · focus · HN ↗
    Not quite sure what all those words mean.
    Dynamical systems will never be solved in a semantic domain.
    IMO, they can be helpful the in robotics stack, from planning level up, but that's it.
    mdp2021 · 1791182706 · focus · HN ↗
    > Not quite sure what all those words mean
    What exactly is not clear? I will rephrase.
    The internal process of the blackbox oracle determine the reliability of the output.
    Two abilities are very different: learning trajectories through empirical training ("increasingly catching thousands of thrown balls"), and determining trajectories through computation (a rational thinker at work). The former is a finetuned parametrized engine (a «NN that learns a skill»), the latter is an Analyst. The former is fuzzy, the second deterministic.
    In front of fuzzy LLMs, which use potentially misleading outputs - text ("has it guessed or has it thought?") - the urgency of warranties of reliable output gets evident.
    So, that they «simulate cognition [only] well enough» (Intended wrote), and that there are «fundamental limitations of models ... purely text based» (Kooi wrote) raises the urgency to overcome the "fuzzy" and achieve the "deterministic" - it is not that we can stall on a «fundamentally not the right tool for that».
    Inventing an oracle calls for urgent striving to overcome the original weakness.
    jcoq · 1791126777 · focus · HN ↗
    Actually, I think my fundamental challenge with AI is that it has no common sense. The way it builds things, writes, and operates is out of touch with reality.
    Incidents like hugging face are partly rooted in the lack of common sense. It still functions like a supercharged toddler.
    I'd love to overcome this because it'd mean I spend less time guiding the the LLM to produce usable outputs.
    shagie · 1791130886 · focus · HN ↗
    [delayed]
    varjag · 1791127303 · focus · HN ↗
    Last week I asked a frontier model draw me a backplane PCB and it placed daughterboard slots side by side in a chain.
    customguy · 1791132837 · focus · HN ↗
    > A statement that we can hopefully all agree has been thoroughly refuted.
    Uh, no? So much of what we learn and take for granted as common sense is not learned via language, and not even expressible in it.
    cztomsik · 1791149877 · focus · HN ↗
    nothing indicated otherwise at the time. IMO he just underestimated RL-scaling. chinese models improved a lot too, they are not parrots anymore, there's some real intelligence, at 27B params.
    consider me optimist now, but just few months ago, even frontier models were dumb, doing stupid mistakes all the time, all of them were so dumb I'd never expect anything to change in just few months.
  4. lawandjustice · 1791124510 · focus · HN ↗
    Can you tell me what is the exact frequency of light hitting your eye as you read this comment? Not by guessing, not from knowledge, but from actually counting? No? Then you are not generally intelligent :)
    lefra · 1791135652 · focus · HN ↗
    All the frequencies, in varying amounts. Next question, please.
    mdp2021 · 1791149673 · focus · HN ↗
    Justify your statement (the other similar post nearby is not sufficient), or realize that we are not talking about that.
    We can have adequate representations of light that are the instances over which we reason. Your simile is about perception, not about instancing ideas.
  5. ben_w · 1791133458 · focus · HN ↗
    To determine this, it would first need to be able to spell "raspberry" as letters rather than as tokens.
    Given you also don't want it to memorise [for all tokens, count([for all letters]), this would probably be more like "here's two images, count all things in the big image that look like the thing in the small image", which can then be r's in a photo of a raspberry jam jar in a supermarket, or dragons in a photo of a furry convention, or whatever.
    That said, they are competent enough at coding that I keep seeing them write code to do even simple tasks.
    On a related note: why did I see Claude editing a file by using cat to write a python script to do a grep search and replace?
    aesthesia · 1791140005 · focus · HN ↗
    > Given you also don't want it to memorise [for all tokens, count([for all letters])
    Why not? You've memorized how words are spelled, and how sounds correspond with letters, and how concepts correspond with words. To the extent that there are shortcuts that enable compression you use these, and the model will do something similar.
    ben_w · 1791142744 · focus · HN ↗
    Combinatorial explosion, and facts merely memorised is a huge waste of parameters that are better dedicated to effective reasoning.
    Being able to spell all the words then count letters is simpler, and more generalisable to other tasks, than memorising answers to all possible word questions.
    aesthesia · 1791152763 · focus · HN ↗
    Ah, I misunderstood what you meant. I was just trying to highlight that in order to answer these types of questions the model needs to memorize the spelling of each token. But you're right that that's all they need to memorize, and algorithms like counting are pretty simple for transformers to implement.
    mdp2021 · 1791147070 · focus · HN ↗
    > Why not?
    Because to "123x456" we want a reply that goes "this times that plus that...", not "Was that not nnnnnn?". If it does not perform its duty it is a liability.
    mdp2021 · 1791146845 · focus · HN ↗
    > it would first need to be able to spell "raspberry" as letters rather than as tokens
    Of any object in question they should be able to create a representation that allows correct assessment.
    > Given you also don't want it to memorise
    That is obviously necessary: what we want from the consultant is to check, not to remember. Answers must be correct and that implies having performed all due diligence - and being capable of doing it, before that. So, objects must be instanced internally in a way that allows effective handling. Counting letters is a good example of the ability (that must remain general).
8. hashmap · 1791118767 · focus · HN ↗
  You're missing the point here. He's not talking about whether or not they can learn facts or inferences derived from the text itself, but the more holistic intuition that results from learning from something like an embodied experience in the physical world. GPT-6 Astras web demo homepage thing is an example. It chose euclidean rather than quaternion for letting a user rotate the galaxy thing, and anyone who has ever used hands to rotate something would immediately recognize on trying it that something is fucked and you shouldnt do that. Thats the kind of common sense physics that is inherently beyond these llms and I run into it ALL the time in vr programming.
  1. mojuba · 1791120246 · focus · HN ↗
    To be fair, LLMs can still derive those kinds of things from text, at the very least from your own comment if it made it to the training set though I'm sure it is mentioned in a lot of other places already. Many of this type of mistakes went away after reasoning was introduced.
    But I'm sure you can still find tasks that they will have difficulty solving, involving the most fundamental concepts that can only be experienced in the physical world to be understood well, like left and right, near and far, hot and cold, heavy and light, etc.
  2. frrrree · 1791129703 · focus · HN ↗
    Yup it lacks common sense because it doesn’t ‘understand’ reality - how could it? It doesn’t touch it like we do everyday. It has access to what is a model of reality via data.
    The good designer understands culture, tastes and preferences as they evolve in real time. That’s why llm as design tools haven’t displaced the good designers.
9. TacticalCoder · 1791119019 · focus · HN ↗
  > ... it will never be able to learn basic common-sense physics like that objects placed on tables will move along with them.
  I use LLMs daily to help me code etc. but... It wasn't long ago that frontier models were confidently recommending to walk, without the car, to the car wash to wash the car no?
  As a daily user of LLMs I do certainly see my fair share of WTF "solutions" to coding problems. I'm not saying it's not super useful: it is super useful. But I don't exactly feel like I'm talking to something that understands that the car needs to be present to be washed.
  1. jeremyjh · 1791131504 · focus · HN ↗
    Astra recommended I walk to the car wash to me five days ago. I gave it multiple hints that I'd be walking away from my car, to spray my car with a hose, then walk back to my car, etc. Never broke through.
10. [deleted] · 1791119035 · focus · HN ↗
  [deleted]
11. poincareball · 1791124517 · focus · HN ↗
  [dead]
12. root_axis · 1791125786 · focus · HN ↗
  Every AI expert any either side of this debate has made very wrong predictions.
13. sassbadger · 1791127245 · focus · HN ↗
  Yeah, and he's probably right.
14. AnotherGoodName · 1791127645 · focus · HN ↗
  LeCunn actually wanted to pivot Meta's entire AI strategy away from LLMs just before he was ousted. He was sure they had nowhere further to go and wanted to pivot to world model generation. The LLM models have since progressed massively.
  An analogy on LLMs is that you have a pretty clear straight highway ahead of you for some distance right now. Maybe that doesn't lead to AGI but it's clear there's progress to be made. For a big tech company it makes sense to push as hard and fast down that clear straight highway of LLMs asap.
  Meanwhile LeCunn wanted to turn off the road and go down an unproven track. I say this as someone working on world model generation right now (creating the ability to learn game world model and have it play the game <a href="https://tfmbot.com" rel="nofollow">https://tfmbot.com for an example of my system pointed at a very complex board game). LeCunn wanted to pivot all of Meta into world model generation. It's good as a side track research project but the entire pivot he wanted to do was madness.
  People are literally talking about an AI researcher who was fired for terrible direction here.
  1. cedws · 1791128154 · focus · HN ↗
    LeCun is a researcher, not a product guy. He's not going to be particularly interested in just working on scaling language models which every lab is already racing to burn cash on. Language models aren't the final frontier of AI.
  2. frrrree · 1791128971 · focus · HN ↗
    And? He might still be right.
    Meta’s AI projects are still negative ROIC
  3. nostrebored · 1791130343 · focus · HN ↗
    … what large advances and at what cost? seems to me that muse 1.3 is kind of a thing. I doubt it will make meta very much money.
  4. jeremyjh · 1791131338 · focus · HN ↗
    I think he was perhaps right and Meta was perhaps also right to replace him.
    The argument is that LLMs are a local maximum that will never breakthrough to AGI. This is still very much an open question. If you are the fifth-best AI lab, does it make sense to try to outcompete everyone in a space that is already too crowded and may not ever yield their actual objective? Instead they could just use open weight models in their products, or post-train on open models like smaller labs have done, and treat that as what it is: product development.
    Pure research has always been about taking chances.
    root_axis · 1791152777 · focus · HN ↗
    I mean, it's an "open question" in the sense that there is no theory behind the idea of AGI, so there's no way to falsify any claim about whether or not any particular path will lead to it.
15. nerdyadventurer · 1791173902 · focus · HN ↗
  I do not thing anything wrong with this statement, do LLM model know about objects on a table? are they conscious? they are just glorified pattern matchers, we get amazed due to their sophisticated and large pattern matching capability.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.