Thread

Thinking fast and slow in AI: The role of metacognition (2021)

177 points · 78 comments · teleforce

Loading the complete thread in the background. This saved snapshot is available now. Refresh

jannyfer · 1790569507 · focus · HN ↗
> submitted Oct 5 2021
(In case people miss that before discussion)
1. tomhow · 1790575936 · focus · HN ↗
  Updated, thanks!
bbor · 1790570603 · focus · HN ↗
This is still a great paper, but it's missing the second axis of the quadric -- if the only two options are thinking fast or thinking about thinking, that leaves no room for thinking slow yet deliberately, AKA selfconsciousness. See <a href="https://www.gutenberg.org/cache/epub/4280/pg4280-images.html" rel="nofollow">https://www.gutenberg.org/cache/epub/4280/pg4280-images.html for details
I do wonder if any of these folks ever got a chance to try this at one of the big labs, tho...
1. BOOSTERHIDROGEN · 1790589174 · focus · HN ↗
  Isn't system 2 by itself semantically means thinking by thinking
crorella · 1790573511 · focus · HN ↗
It looks like a lot like how data bases query optimizers work, with the exception that in the paper there is also a learning/memory component that conditions the evaluation of the answer provided by the first model.
1. mdk2578 · 1790581257 · focus · HN ↗
  [dead]
2. alansaber · 1790589937 · focus · HN ↗
  Given how LLMs access compressed knowledge from their model weights, the similarities make sense
red75prime · 1790574076 · focus · HN ↗
It reminds me of "At the time we drew boxes labeled 'perception', 'cognition' with arrows between them." An imprecise quote that I can't place.
I guess my box labelled 'subconsciousness' is trying to say that low-level mechanisms that give rise to the observed cognitive phenomena might have nothing to do with neat boxes.
1. bananaflag · 1790575438 · focus · HN ↗
  You mean "Artificial Intelligence meets Natural Stupidity" by Drew McDermott
  <a href="https://dl.acm.org/doi/pdf/10.1145/1045339.1045340" rel="nofollow">https://dl.acm.org/doi/pdf/10.1145/1045339.1045340
  1. arethuza · 1790583172 · focus · HN ↗
    McDermott's A Critique of Pure Reason pretty much captured all of the misgivings I had about "Good Old-Fashioned Artificial Intelligence", which was slightly unfortunate as I was trying to complete a PhD in that very area at the time (around 1990 or so...)
    bananaflag · 1790587168 · focus · HN ↗
    Did you complete it?
    arethuza · 1790600817 · focus · HN ↗
    No - there were lots of other interesting things going on in the early 90s that I managed to pivot to. Never regretted not completing as I had given up any desire to work in academia by that point.
2. CipherText · 1790581700 · focus · HN ↗
  [dead]
3. Phemist · 1790592346 · focus · HN ↗
  Donald Broadbent drew a lot of boxes in the 50s/60s (<a href="https://en.wikipedia.org/wiki/Broadbent's_filter_model_of_attention" rel="nofollow">https://en.wikipedia.org/wiki/Broadbent's_filter_model_of_at...). There have been plenty of critics of this tendency, I recall some calling it 'boxology' rather than psychology.
sinuhe69 · 1790574665 · focus · HN ↗
[delayed]
1. cjg · 1790597489 · focus · HN ↗
  Good luck linking to that rule.
aidiscoverywire · 1790575206 · focus · HN ↗
[dead]
simianwords · 1790576423 · focus · HN ↗
This has already been solved by GPT 5 Adaptive reasoning. A single model that knows when to reason or not based on a thinking parameter we provide (like xhigh). What’s the relevancy to post it today?
1. bpshaver · 1790576936 · focus · HN ↗
  Surely that is obvious
2. lelanthran · 1790580771 · focus · HN ↗
  > This has already been solved by GPT 5 Adaptive reasoning. A single model that knows when to reason or not based on a thinking parameter we provide (like xhigh). What’s the relevancy to post it today?
  Tell me you didn't read Daniel Khaneman's book without telling me you didn't read Daniel Khaneman's book.
  1. simianwords · 1790581046 · focus · HN ↗
    Asking earnestly, I don’t know what you mean by this reply. I know what system 1 and 2 is. But this has already been solved using same model.
    lelanthran · 1790582237 · focus · HN ↗
    > I know what system 1 and 2 is. But this has already been solved using same model.
    No, it hasn't. Maybe you have a different definition of System 1 and System 2. I last read the book well over a decade ago (2011, maybe? 2012?), but System 1 and System 2 are different systems. IOW, System 2 is not a more computational version of System 1.
    The argument you made implies that System 2 is just a more capable System 1, which is not what the book (nor this paper, AIUI) proposes.
    In computery terms, System 1 runs in O(1) time, System 2 runs in O(log n) (or maybe just O(n)) time.
    This means that any System 1 will run the input once through the heuristics, using the same computational power and taking the same time whether the input is 100 tokens or 1 million tokens, for quick but perhaps wrong decision (not "answer"). We don't have LLMs that do that. We have System 2 - run in O(log n) time and produce an answer.
    System 1 is completely bereft of thought.
    simianwords · 1790582580 · focus · HN ↗
    > The argument you made implies that System 2 is just a more capable System 1, which is not what the book (nor this paper, AIUI) proposes.
    No, system 2 is the emergent capability to reason and increase the space of places to find the answer. Forget the paper's proposal, and look at the problem it is trying to solve. Ability to give quick answers, ability to give thought out answers, and the ability to know when to choose what. Adaptive reasoning does all three.
    > This means that any System 1 will run the input once through the heuristics, using the same computational power and taking the same time whether the input is 100 tokens or 1 million tokens, for quick but perhaps wrong decision (not "answer").
    No, I don't think we humans use o(1) to for understanding 1000 tokens or 2 tokens. I simply don't think that's the case. There's a new model called "Jev" and it is literally named System 1 (from the book) and even it is billed per input token.
    hoppp · 1790606156 · focus · HN ↗
    The difference in humans is consciousness. System 1 is subconscious and fast, system 2 is conscious and slow.
    System 1 can do multiple passes and all that, the speed is related to how aware are you of the computation occuring and system 2 must be consciously managed.
    pixl97 · 1790608721 · focus · HN ↗
    > but System 1 and System 2 are different systems.
    They also share a lot of overlap in brain structures and they interplay while executing. A thought can start out Sys1 and quickly migrate to Sys2 as the pattern fails to match. Or a System 2 chain of thinking can be made from a bunch of smaller system 1 actions. Heck, in the middle of a system 2 thought you can plunge into system 1 system/actions. It's more of a who matches the pattern up with reality the fastest and acts on it.
    > We don't have LLMs that do that. We have System 2
    Eh. LLMs are system 1 thinkers by default. "Quick" response with no reflection is where LLMs started. It's later we added Chain of Thought and reflection layers and all kinds of other things like harnesses and agent training to make them act like system 2 thinkers. Of course we have other technologies being tested on LLMs these days like Dynamic Sparse Attention that likely match more of your thoughts on what system 1 thinking is too. Where the network doesn't have to parse the full context of the prompt and instead pattern matches with a much smaller percentage of the input giving responses back in ms versus seconds.
    creativeSlumber · 1790610841 · focus · HN ↗
    is this just a fancy way of saying that sometimes humans makes quick heuristic based decisions, and sometimes they think through things thoroughly?
    I don't think there's any problem category that is strictly a quick heuristic decision or something that you will think through thoroughly always. I think it's more about how much time you have. If you don't have time you'll make a quick heuristic based decision. If you have more time you will think through it more. Imagine you're driving and suddenly like a branch falls on the road right in front of you. And you need to avoid it. Your brain will quickly use a heuristic based approach to avoid the branch. But on the other hand, if this branch was already there on the road and you saw it from far away, you will probably take a lot more time to think through and figure out which path you need to take.
3. globnomulous · 1790582088 · focus · HN ↗
  It's being downvoted, I think, for a few reasons:
  * The person who posted it likely wasn't posting it as an out-of-date paper but as an interesting idea. Your comment ignores the idea and focuses on the out-of-dateness.
  * You say "this has been solved" without defining what "this" is.
  * The solution to the problem -- different effort levels -- seems to indicate that you misunderstand the idea that the paper is proposing. If I understand their proposal, it's that the system itself decides how to reason based on the nature of the problem it faces, given the model's world model.
  * The title is an allusion to a book by Daniel Kahneman. The brisk dismissal without acknowledging the idea or the history doesn't leave a good impression, even if I'm mistaken and you're right.
  In short, Hacker News readers tend to reward depth and detail (the FAQ specifically encourages thoughtful contributions and explicitly discourages dismissal). Your comment doesn't provide them, and it appears to make a mistake that further undermines its value as a contribution to discussion.
  1. simianwords · 1790582133 · focus · HN ↗
    > it's that the system itself decides how to reason based on the nature of the problem it faces, given the model's world model.
    do you even know how adaptive reasoning works?
    globnomulous · 1790582561 · focus · HN ↗
    Godspeed to you in your efforts to contribute productively to Hacker News threads.
    simianwords · 1790582724 · focus · HN ↗
    > For the first time, GPT‑5.1 Instant can use adaptive reasoning to decide when to think before responding to more challenging questions, resulting in more thorough and accurate answers, while still responding quickly. This is reflected in significant improvements on math and coding evaluations like AIME 2025 and Codeforces.
    <a href="https://openai.com/index/gpt-5-1/" rel="nofollow">https://openai.com/index/gpt-5-1/
    It says literally the thing you wanted from system 2. Its almost exactly that.
    This is what you said btw:
    "it's that the system itself decides how to reason based on the nature of the problem it faces"
    globnomulous · 1790617918 · focus · HN ↗
    My second response was sassier than it should have been. Sorry for that. I'm not making any claims about whether or not GPT implemented this. My point was narrower in a way that is probably annoying: you asked why you were being downvoted; I tried to explain that, as far as I could tell, the adaptive system you explicitly described (regardless of what adaptive reasoning actually is), where humans choose the reasoning level, isn't the one that the paper proposes -- and that this may be one of the reasons for the downvotes.
    I don't know whether that's why anybody downvoted you, but I think that's probably part of the reason. On the other hand, the Hacker News guidelines ask us to respond to the strongest interpretation of a comment or post. In that spirit, just as you could have said something to the effect of "Yep, AI researchers learned from behavioral economics and research into human decision making and here's where OpenAI mentions it," I could have said just "the relevance may not be that it's a new idea but rather that it's a neat idea, to the person posting it at least. Maybe someone who knows AI systems more deeply wouldn't find it interesting. I do though!" That's a bit of hypocrisy on my part, I think.
    Anyhow, for the record, yep, I knew of adaptive reasoning, but, no, I didn't know that it matched what this paper proposes so closely. Thanks for the correction!
zfoong · 1790579581 · focus · HN ↗
At least this is written before ChatGPT.
creativeSlumber · 1790580636 · focus · HN ↗
How relevant is this fast/slow thinking thing with regards to current frontier models?
I know a large organization who's built their AI framework completely around this concept, and I feel that it's not really meaningful concept with the capabilities of current models.
1. Retric · 1790582220 · focus · HN ↗
  You can ask a model for output directly and stop, or you can recursively ask it to keep refining the output.
  That seems to fit the fast vs slow model of human thought reasonably well.
  1. usernametaken29 · 1790587118 · focus · HN ↗
    > You can ask a model for output directly and stop
    That’s still several orders of magnitudes too slow to fit fast vs slow. Think of 30ms vs 3-4 seconds to get an idea of what we’re talking about here
    Retric · 1790587655 · focus · HN ↗
    That’s a function of the amount of processing power involved not the underlying architecture of decision making.
    usernametaken29 · 1790591813 · focus · HN ↗
    In terms of making an LLM faster but not in terms of meta-cognition. System 1 thinking as defined by Kahneman doesn’t have 100000x more compute than System 2, it is actually the opposite. That completely contradicts your claim
    Retric · 1790600944 · focus · HN ↗
    The ratio between a single pass and multiple passes is unchanged when you throw more processing power at both.
    hoppp · 1790605840 · focus · HN ↗
    Multiple passes doesn't make it system 2. The defining characteristic of system 2 is consciousness which is expensive and slows down the system.
    So to talk about system 2 in AI we need to talk about consciousness. As long as AI is not officially conscious there is no System 2 thinking implemented
    Retric · 1790616213 · focus · HN ↗
    Consciousness is ill defined hogwash when used in such descriptions.
    The process of internal refinement without external action however fits.
    hoppp · 1790619639 · focus · HN ↗
    Using system 1 and 2 for AI is ill defined hogwash indeed.
    The systems apply for humans and describe conscious and subconscious processing.
    Retric · 1790622703 · focus · HN ↗
    System 1 and 2 is a description of physical process in people.
    Using conscious vs subconscious processing is not however a meaningful definition. Subconscious processing isn’t necessarily fast.
    hoppp · 1790687584 · focus · HN ↗
    Not all subconscious processing is fast, for sure.
    But for example the amygdala is super fast at tagging dangerious situations with emotions.
    And humans socialize subconsciously, so all the small talk conversations are all subconsciously generated.
    The definition from the book was:
    System 1 is subconscious, fast, automatic and relies on heuristics
    System 2 is conscious, slow, deliberate and effortful
    Retric · 1790709822 · focus · HN ↗
    The amygdala isn’t particularly fast here. It’s got significant connections to higher order centers in the brain because classifying dangers can be quite complicated.
    hoppp · 1790799764 · focus · HN ↗
    Reaction to danger must be fast, otherwise the animal dies.
    I am not buying that danger classification is slow because it's very clear that it's not. Animals must react fast or die.
    Thats just natural selection.
    Retric · 1790810136 · focus · HN ↗
    Danger isn’t some universal thing, people are adapted to notice fact moving objects heading towards them quickly very fast. A very dangerous snake isn’t in that category of danger and can take a significant chunk of time to notice.
    Emotions are extremely complex and hormones are inherently slow to diffuse through the brain. Pair bonding between a mother and child needs to happen, it doesn’t need to happen in 50ms.
    hoppp · 1790811195 · focus · HN ↗
    Yeah, pair bonding doesn't happen instantly but thats also not about processing and reacting to signals, it also involves the endocrine system.
    If a tiger attacks you, you better react fast, but one strictly human example is noticing and reacting to angry faces.
    An angry person comes at you with a bloody hammer you react before you understand what's even happening
    Retric · 1790814414 · focus · HN ↗
    > thats also not about processing and reacting to signals > not an information processing system in the same sense
    I can only assume you have some sort of philosophical objection here, but this is information processing utilizing biological systems that are inherently messy here.
    Neurons are reacting to a range of chemical signals not just a single neurotransmitter because they don’t map to what CS style neural networks. Understanding the brain requires understanding the nuances not simplified models.
    hoppp · 1790879971 · focus · HN ↗
    I think it modulates information processing, the endocrine system does deliver information but it's internally recieved.
    The key word is modulation.
    Thats why I would not add the endocrine system as a system 3 or whatever.
    DNA copying is also an information processing system but it's not considered in cognitive psychology but transcription factors will greatly effect behavior also.
    hoppp · 1790880131 · focus · HN ↗
    But in the case of rats I would agree, their brains have more advanced olfactory systems and they communicate directly via pheromones to each other using the endocrine system.
    But it humans that processing is more nuanced and the olfactory system is less advanced.
    pixl97 · 1790606318 · focus · HN ↗
    >System 1 thinking as defined by Kahneman doesn’t have 100000x more compute than System 2, it is actually the opposite.
    When are you measuring?
    Systems 1 thinking is closer to precomputed tables in some ways. That is by evolution or massive amounts of training your neural network has a narrow fast path it can execute with as little compute at execution as needed.
    Retric · 1790616381 · focus · HN ↗
    LLM’s operate strictly feed forward neural networks.
    Slower paths means loops here.
    usernametaken29 · 1790645438 · focus · HN ↗
    Your assumption is wrong. Unlike artificial neurons, real neurons process massively and concurrently, let’s say 10-20 thousand inputs at once in less than 1 millisecond. In Kahnemans theory system 2 always involves the anterior cingulate cortex and arises when there are multiple conflicting streams of information. That’s the defining characteristic. Both system 1 and system two process in the same way, including looping back to previous areas. So really, no, stopping an LLM early vs letting it run has nothing to do with Kahnemans theory. In general LLMs are so far removed from what we consider natural cognition that it’s very hard to apply neuroscience concepts to LLMs simply because they share no real world similarity. At best, you’re simulating something in a very inefficient way.
    Retric · 1790650165 · focus · HN ↗
    I’ve studied the underlying biology of neurons and desiring it as 10-20 thousand inputs in 1 millisecond isn’t accurate the way you’re suggesting.
    We’re talking about biological processes which have physical limitations on how fast a neuron can fire as well as a signaling system that based on how rapidly neurons fire. Loops anywhere in the system enforce a minimum 5ms latency. And a typical neuron is firing closer to 0.1 to 2 times a second.
  2. schainks · 1790617901 · focus · HN ↗
    Not quite. The better analogy for you is the "thinking" setting on your model.
2. olgava · 1790583952 · focus · HN ↗
  [dead]
3. anotha_one · 1790585064 · focus · HN ↗
  [dead]
4. Shorel · 1790587320 · focus · HN ↗
  That's because an LLM thinks in terms of language, while we think in a different way, then convert the ideas to language. It can be said that language is a tool for the serialization (writing) and deserialization (reading) of human ideas. It is also an incredible useful and powerful tool by itself. This last sentence has been proved true by LLMs themselves. However, since it is working on the serialized version of ideas, I agree with you in that's not the optimal way to think and something not serialized (maybe world models) can be invented that's better for thinking. All this in no way diminishes the usefulness of language and of automated language generation.
  1. virgilp · 1790594350 · focus · HN ↗
    > That's because an LLM thinks in terms of language, while we think in a different way, then convert the ideas to language
    Do we? I just learned from a speaker[1] that we literally need words to recognize emotions. People who have a poor vocabulary have lower emotional intelligence because without being able to attach a word to an emotion, the brain is unable to recognize & process it.
    [1] Dude seemed to be knowledgeable about the subject. He's a specialized trainer, should be educated in this exact field. So hopefully I'm not lying to anyone here :)
    globalise83 · 1790602693 · focus · HN ↗
    Sounds uncannily similar to the pseudofacts you hear a lot in Neurolinguistic Programming training courses for sales reps. Would ask for a scientific publication reference on that one.
    Shorel · 1790604584 · focus · HN ↗
    It seems to me that idea is rooted in social consensus.
    Someone expresses an emotion but doesn't know how to react to it, their inner group all have an opinion about it, and the consensus is selected as the "appropiate" reaction to it. The individuals who react this way will claim this consensus is the same as emotional intelligence.
    Just as there are also people who react in one way, and completely disregard any external opinion about it. They simply have firm opinions and don't need the consensus.
    I will not comment on who can belong to each group, that's an exercise for the reader.
    Shorel · 1790604245 · focus · HN ↗
    I can't agree about the "unable to recognize and process it", simply because that idea is totally contrary to my own experience. I have in fact many memories which have emotions in them, without words or other external elements. However, seeing that language serialization seems to enable a vastly extended memory (entire sagas remembered as songs), it is understandable that something is gained by serialization of emotional experiences, just as something more immediate is lost.
5. ghm2199 · 1790589379 · focus · HN ↗
  Structurally speaking we learn nothing like AI, we don't use vast amounts of information to pick up completely new skills. We also make decisions by using prior knowledge and emotions.The latter part is important, Thinking fast and slow cannot operate in a world of AIs as they stand today unless we are willing to grant them rights — because you have to teach them to make decisions based on all kinds of emotions — which is tricky at best.
  1. TeMPOraL · 1790590822 · focus · HN ↗
    > we don't use vast amounts of information to pick up completely new skills.
    Except, we do.
6. emp17344 · 1790590650 · focus · HN ↗
  Seems like a terrible idea in the first place to build an entire organization around a single pop-sci book, but that’s just me.
  1. TeMPOraL · 1790591132 · focus · HN ↗
    It's indeed a terrible idea - as in, it's great. You get benefits of cross-marketing: you ride on a popularity of a well-known book, and as you also drive more sales of it, even if you don't have a deal and don't benefit from that directly, you strengthen the loop and solidify your brand.
    This choice doesn't really constrain what the organization can do, either. Pop-sci books have plenty of wiggle room in interpretation, and afford a lot of "you're holding it wrong" dismissals of criticism, that with a bit of clever copywriting, the organization can do absolutely anything and still claim it's embodying the framework/theory of the book.
  2. glial · 1790607690 · focus · HN ↗
    System 1/2 is the pop version, but the fast/slow distinction is prevalent in both RL and computational cognitive science, sometimes going under different names: procedural/deliberative, model-free/model-based, automatic/controlled, associative/rule-based, autonomous/algorithmic, etc.
7. jhrmnn · 1790594157 · focus · HN ↗
  No clue what’s the consensus on this but my internal mental model is absolutely that LLM AI is pure fast mode, no slow mode. The “reasoning” loops are an attempt to mimic the slow mode but ultimately it doesn’t really work. I’m curious about the recent maths advances though, they seem to possibly challenge this.
  1. hoppp · 1790605991 · focus · HN ↗
    Its just hype talk. Slow mode is conscious in humans, fast is subconscious, so we need to discuss AI consciousness to talk about system 2 thinking.
    So the entire debate is fubar.
8. hoppp · 1790605732 · focus · HN ↗
  Not relevant at all. Its a way to hype.
9. glial · 1790607471 · focus · HN ↗
  'fast' means executing a policy, that is, a state-action mapping. A trained RL model does this.
  'slow' means making one or several action-dependent forecasts, evaluating the expected value of the outcomes, and making a decision based on that.
  Neither map exactly to the situation with LLMs, but very roughly, the first is analogous to trained classifiers and the second to reasoning models.
  The analogy breaks down, since each instance of token being produced is an example of a policy execution (system 1), and reasoning is just stringing lots of these together. But there are those who argued, before LLMs, that system 2 is just "policy composition" anyway...
  1. creativeSlumber · 1790610027 · focus · HN ↗
    Wouldn't you need a classifier to even decide if it is system 1 or 2? How capable does this classier need to be?
    glial · 1790610614 · focus · HN ↗
    I think of System 1 as a hash map. If you have a map, and see a new state/key that you haven't seen before, you have to go with System 2.
10. theptip · 1790615925 · focus · HN ↗
  Very relevant. Modern models use CoT to do “slow thinking” and this enables them to achieve much greater performance. You can also turn off thinking and answer directly which is quite similar to “fast thinking”, good at approximate maths, not capable of algorithms, etc.
  Of course the shapes of what an AI can do in fast vs slow are quite different.
vist_orn · 1790582612 · focus · HN ↗
Trying to get LLMs to 'think about their thinking' is my daily struggle. This paper nails why it's so critical.
readthenotes1 · 1790586270 · focus · HN ↗
If I recall correctly, all that fast and slow business has been debunked as yet more non-replicable pop psychology.
I shouldn't be surprised that it shows up in a screed on AI
gchamonlive · 1790592114 · focus · HN ↗
[delayed]
1. Obscurity4340 · 1790598709 · focus · HN ↗
  How can things be compressed without losing information or structure?
  Like for text, what would that involve? How do you compress a string or multi-line string without losing information and hopefully structure (paragraphs, would it be like replacing periods and the following space with just sticking the starting capitalized letter of the following word to the previous sentence's last letter and when it decompresses theres some kind of note that converts that back into the. First letter of the next sentence
  1. mpalmer · 1790599484 · focus · HN ↗
    How can things be compressed without losing information or structure?
    Because the initial content is not optimal, so it's possible to store fewer bytes that can deterministically be converted into the original.
    Like for text, what would that involve?
    Most compression algos don't care what information you're compressing. All they see (all they need to see) is bytes. It ends up being way more sophisticated than removing repeated periods and whitespace.
    gchamonlive · 1790619986 · focus · HN ↗
    [delayed]
  2. nkmnz · 1790599667 · focus · HN ↗
    You analyze the frequency of combinations of bytes, then replace those with high frequency with pointers to a single instance.
    selicos · 1790614704 · focus · HN ↗
    Deduplication ^
  3. doctoboggan · 1790616186 · focus · HN ↗
    Other than what the other mentioned about finding more efficient representations, you can also compress by pre-agreeing on some common terminology.
    gchamonlive · 1790619871 · focus · HN ↗
    [delayed]
  4. fizzbuzzbarbazz · 1790637969 · focus · HN ↗
    comments would be omitted.
``` th #1 - numbers are just comments ng #2, this has a space at the end information #3 compress #4 letter #5 this has a space at the start and #6 this has a space at the start and end sentence #7 this has a space at the start
How can 1ings be 4ed without losi23 or structure?
Like for text, what would 1at involve? How do you 4 a stri2or multi-line stri2wi1out losi236hopefully structure (paragraphs, would it be like replaci2periods61e followi2space wi1 just sticki21e starti2capitalized5 of 1e followi2word to 1e previous7's last56when it de4es 1eres some kind of note 1at converts 1at back into 1e. First5of the next7 ```
I'm on a phone, so I may have mistakes here, but I'm pretty sure that's shorter than your original text, in bytes, by about (9+18+20+21+12+12+16=108), minus the dictionary size of 51 -- so, 57 bytes shorter, but still containing your full text. With predistributed compressor binaries and a lot of analysis, you can even predistribute a global dictionary for common sequences, and simply specify "xyz0", x, y, and z being 24-bit numbers, or whatever bit size can index into your full reference dictionary, and 0 meaning end-of-file-dictonary. then, assuming the byte sequences in your text above are common enough to be in the 24-bit indexed dictionary, that initial dictionary could be just 22 bytes (21 and a terminator) -- so, 86 bytes shorter than the original, but still containing your original message unaltered. ..assuming i didn't make mistakes in my hand-compression.
2. sourdecor · 1790621018 · focus · HN ↗
  I don't know how this is related, but it reminds me of how I have always believed compression to be the ultimate sign of intelligence. If you can reduce something, you are finding more abstract symbols to represent the information of the original source.
  1. acuozzo · 1790625738 · focus · HN ↗
    > I don't know how this is related
    <a href="https://en.wikipedia.org/wiki/Hutter_Prize" rel="nofollow">https://en.wikipedia.org/wiki/Hutter_Prize
BigDogAU2026 · 1790592137 · focus · HN ↗
[dead]
arbirk · 1790594919 · focus · HN ↗
Interestingly that is not what we got, but maybe we should loop at architectures like this again? The JEPA loop is interesting, but might fail for the in-flexibility of the component ordering
ot4t · 1790604633 · focus · HN ↗
[dead]
hoppp · 1790605645 · focus · HN ↗
Most humans hace weak meta-cognition, a large percentage doesn't have verbal thoughts.
Meta-cognition makes sense in a dynamic and updatable and modular system, for example I can monitor thoughts coming from my amygdala with my prefrontal cortex and then adjust how I process these thoughts.
In LLMs it makes zero sense, even if you feed the output of one model into another, there is no way they can update the heuristics behind how those were computed.
tug2024 · 1790608599 · focus · HN ↗
[dead]
locitra · 1790610254 · focus · HN ↗
[dead]

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.