Thread

"As a Language Model": Chat Template Switches LLM Self-Referential Voice

103 points · 110 comments · yu3zhou4

Izmaki · 1790508818 · focus · HN ↗
"As a Language Model..." is one of the beginnings of a sentence I hate the most from LLMs and is the reason why I support free (as in "Liberty"), local models. I'm well aware that it is not a doctor and cannot replace a real doctor with multiple years of experience, I don't need to waste braincell activity on reading that it "as a Language Model" cannot give a precise diagnosis and that I should ask a real doctor - all I want to know is if I what I experience justifies either A) ER, B) 3-4 weeks scheduled doctors appointment or C) two paracetamol and a nap.
I don't want "jailbroken" LLMs to commit crime. I want them to avoid having this vendor-specific "bloatware" all over the product I'm using.
1. Anduia · 1790510825 · focus · HN ↗
  Be careful there. LLMs may be good at identifying a condition based on a description of the symptoms, but they are much worse at recommending the correct course of action (getting it wrong half of the time).
  [0] <a href="https://www.nature.com/articles/s41591-025-04074-y" rel="nofollow">https://www.nature.com/articles/s41591-025-04074-y
  1. bastawhiz · 1790516120 · focus · HN ↗
    That's still incredibly valuable, though. I suffered from issues that I'd seen doctors for, undergone an upper endoscopy, adjusted my diet, and taken medication for. An LLM suggested my thyroid was at the root of it. My mom confirmed thyroid issues run in our family and just...never thought to tell me.
    Just this year, our cat has been having digestive problems. We got special food for her, which she hates, with the suggestion that she'll need to eat it for the rest of her life. Six vet visits later, Fable 5 suggested two tests that my doctor recommended we didn't get. Both found issues that explain her symptoms, and the vet says she will probably only need a supplement and infrequent two week courses of medicine if she has a flare-up.
    All that to say, the helplessness of not knowing what's wrong and the people who could know not really caring enough is something that LLMs do a really great job of mitigating. If you don't have any way to know what's wrong with you or a loved one, or how to find out, you're stuck spending a ton of money (in the US at least) and crossing your fingers that someone gets it right.
    therealpygon · 1790517466 · focus · HN ↗
    Playing the critic that alternate “50%” outcomes were:
    - You didn’t have a thyroid issue and you spent thousands more on tests the doctor was right that you didn’t need.
    - Your cat didn’t have that issue and you spent hundreds on unnecessary tests.
    Are you essentially saying that it is better for an LLM to sell you an idea that sometimes might be right, selling a dream to anyone with and without the money for it?
    So… a telephone psychic?
    The thing about WebMD telling everyone they have cancer is that sometimes it will be right.
    (Just for clarity, I’m really glad it was helpful for you.)
    mrandish · 1790534904 · focus · HN ↗
    > an LLM to sell you an idea that sometimes might be right
    An LLM is just an information resource that can be useful when used properly. It can also be useless or net harmful when used improperly. The same is true for web search, libraries and even human experts. A licensed medical doctor is a domain expert and competent domain experts are often correct, but not always.
    I don't think it's possible to suggest a universally 'correct' default position on when (and how much) to accept your GP's medical opinion over any alternatives. It depends on too many variables: the context, the person, the alternative info sources, the expected value of correctness and the potential consequences of error. But decision theory suggests this is the kind of combinatorially complex problem space for which any single default position, whether "Always trust your GP" or "Never trust your GP", for every person and situation cannot be optimal.
    therealpygon · 1790607897 · focus · HN ↗
    I agree with you. And if they flipped a coin, and the coin happened to land on heads which they decided was a prediction of no cancer, would you recommend they tell others flip their coin if they are tired of their doctor because it was helpful when their test came back negative? “It didn’t hurt anything”, right? What happens when that specialized test is $100k because it is rare, and your doctor is pushing back, “but the quarter told me i have this cancer and you said your other tests couldn’t rule it out”. Is there no harm then? Would you still think they should go around advertising their cancer detecting quarter without anyone pointing out it’s freaking quarter?
    Btw, no one suggested people shouldn’t consider various forms of information. All I pointed out is that if an LLM happened to guess right, it doesn’t mean that is some sort of datapoint that others should believe validates an AI’s predictive abilities any more than a telephone psychic or webmd should be a trusted source because they are occasionally right.
    It is exactly the tactic that makes snake-oil…I mean the vitamin industry…so profitable from people going around and telling others crap like “rubbing the oil from this tree on my foot made me sleep better”, and “I took this garnish as a supplement and it cured my blah blah”. Can people need vitamins? Sure. Does the average person need anywhere near the amount they take? Rarely. People sell each other a dream of feeing better; this would be cutting out the middle man and suggesting an AI take over.
    It’s funny how pointing out the risk means people act like I’m suggesting it should be banned. We can see how many MLM Karens came out of the woodwork to defend AI in this, as though I somehow said people shouldn’t be allowed to consider feedback from an LLM by simply pointing out exactly how wrong — best case — it could have easily been from a system known to make errors. In fact, the only reliable part about AI right now is that it will make mistakes. Literally almost all investment in AI is about having it make less mistakes,. Every single metric we use to measure AI is about how many mistakes it makes. Every time we test how smart it is, we measure how much it got wrong and how many times it failed.
    I understand it. Sick people will turn to just about anything as a cure or to make them feel better, or simply to find some hope. Sometimes they get better, whether that thing helped or not, and then loudly tell everyone that whatever thing did it regardless of realities. I’m not discounting anyone’s stories, I’m pointing out how they were lucky it didn’t go exactly the opposite, which is what the next person could experience by trusting and not considering the feedback from an LLM cautiously.
    Also, can literally anyone point out where the comment I replied to said a word about “doing research”? Anyone? People immediately Karen in on how “it’s just part of research” — where was that said? Or did people just read what they wanted to? Too funny.
    PS, I have MS… before any more idiots want to tell me that I don’t know anything about the problems with the US healthcare system.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.