Thread

Calling the AI bluff: Adding "Do not guess" cut made-up claims from 71% to 20%

96 points · 42 comments · FKJ

BatchJob · 1790562354 · focus · HN ↗
The LLM will take a statistical path to reply and will not refuse to do so under any circumstances except where its been coded to do so.
Your examples are contrived and will not be borne out in any significant way. Inaccuracies are usually not simply made up claims they are false information based on statistical paths to misleading results or which elude the current context. LLMS dont understand the word dont. LLMS dont understand the meaning of any words.
Neither you, nor aristotle nor god will ever make an LLM return the truth or correct results via prompting.
1. astrange · 1790616388 · focus · HN ↗
  > LLMS dont understand the meaning of any words.
  In what way do you understand the meaning of the word "unicorn" that an LLM does not? It has experienced exactly as many real unicorns as you have.
  1. shiandow · 1790617605 · focus · HN ↗
    Experience is not understanding, but an LLM does not reason so it cannot understand.
    It can produce text that looks like reasoning. It can even produce text with mostly sound logic, but there is no internal experience or reasoning that occured there just the generation of language.
    LLMs therefore tend to be very bad at tasks that involve meta cognition. I've yet to successfully convince one to tell me when it knows something.
    serf · 1790618111 · focus · HN ↗
    I think the X cant do Y arguments require strict definitions of X and Y.
    w.r.t. this post : let's define 'reasoning' here, because there are definitions of 'reason' and 'reasoning' that would fit to a simple condition comparison let alone a massively complex llm.
    as for the meta cognition bit : show me a human that can accurately affirm when they know something. These kind of things aren't binary, nor can they be.
    otabdeveloper4 · 1790618366 · focus · HN ↗
    LLMs generate text, and they do not use any system of logic or syllogisms to do so. It's a pretty cut-and-dry obvsious statement of fact, no need to muddy the conversation here.
    allturtles · 1790619336 · focus · HN ↗
    So by your lights, the vast majority of humans don't reason either (and even those who do don't do it most of the time)?
    _g0xr · 1790647941 · focus · HN ↗
    Have you read the chinese room argument?
    <a href="https://rintintin.colorado.edu/~vancecd/phil201/Searle.pdf" rel="nofollow">https://rintintin.colorado.edu/~vancecd/phil201/Searle.pdf
    People have been having these discussions since 1980 at least. I'm not going to bother rehashing the issue endlessly in shortform internet comments, but this paper exists if you're really interested in the subject.
    allturtles · 1790698029 · focus · HN ↗
    Yep, familiar with it. I think the Chinese room is clearly a bad argument. The brain is just as much a "chinese room" as a computer is.
    astrange · 1790623660 · focus · HN ↗
    There is internal experience.
    <a href="https://www.anthropic.com/research/global-workspace" rel="nofollow">https://www.anthropic.com/research/global-workspace
    However, we don't /want/ them to have too much internal experience, because we want to know what they're thinking* for safety reasons.
    * or, we want to be able to assume that the answer text is causally related to the thinking text
    Zambyte · 1790681295 · focus · HN ↗
    > I've yet to successfully convince one to tell me when it knows something.
    This is a near daily experience for me when using a coding harness. I will ask it for some favts about the environment, and it will continuously explore the environment until it exhausts reasonable exploration, or it finds the facts.
  2. ASalazarMX · 1790624989 · focus · HN ↗
    The word itself? There's little practical difference.
    The concept, though, it's a very wide moat. You could call a unicorn "nyati" for all we care, we still know it's the concept of a magical flying horse with a single horn. We know, besides the literary corpus, the concepts of magic, horse, flying, and horns. It's fictional, yet we have a very good idea of what if would sound, feel, or even smell like. Ask an LLM to describe what a unicorn feels like, and it will ramble about forests and sparkles.
    In fact, I asked Gemini (thinking, to see the process) to describe a mindful experience about meeting an unicorn in real life, and it did ramble about the event. When I reminded it about mindfulness being about experiencing with all your senses, and asked it to focus on the creature, to its credit, it even described the taste:
    > Taste: Even the air surrounding the creature tastes different on your tongue—thin, crisp, and tinged with a faint, sweet metallic tang, like snow melting on limestone or fresh rain falling through high canopy.
    Still nonsense, as a unicorn (or the air around it) will likely taste like horse, whatever that flavor is. It finishes with more nonsense, and I doubt the flash version will give better results.
    > Every micro-detail of its anatomy becomes an anchor to the present moment. You are not thinking about what it means or where it came from; you are simply perceiving the texture, heat, sound, and weight of a living, breathing reality standing inches away.
    Still, current LLMs can do amazing things given their inherent limitations, and that makes it easy for us to overestimate their capabilities.
    astrange · 1790629294 · focus · HN ↗
    Gemini is kind of cooked. It's a personality hire.
    I did try asking Gemini and Claude "what would a unicorn taste like" and got… acceptable and accurate answers, but the accurate answer maybe wasn't "acceptable", because I don't think a little girl asking Claude that question should have gotten a long description about what horse meat tastes like, like I did.
    So there of course are issues where an LLM having explicit knowledge about something doesn't mean it has tacit knowledge about it in all contexts, but also hiring the LLM to do a job of being a helpful harmless chat assistant constrains its abilities.
    (Gemini also points out a Starbucks unicorn drink doesn't taste like horses.)
    ASalazarMX · 1790630287 · focus · HN ↗
    Gemini, Claude, ChatGPT, and all the others are variants of the same architecture and methodology. Unless something profound changes in the AI space, not a single one of them is more 'cooked' than the other.
    astrange · 1790655555 · focus · HN ↗
    Hm? They have pretty different post-training, personality, constitutions and quantization levels.
  3. Shacharp · 1790630249 · focus · HN ↗
    [dead]
  4. yesitcan · 1790676800 · focus · HN ↗
    > LLM is next token generator
    > Isn’t the human brain also just a next token generator?
    The two most cliche messages on this forum. They occur in every LLM discussion. It’s fascinating.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.