Ten days ago I had an experience with Gemini 3.8 flash that made me wonder if I was being routed to a different model under test. I was trying to use rocm with llama.cpp on my 128gb Strix Halo but could only get it to run Vulkan. I pasted the error message into agy and it proceeded to attach GDB to my GPU driver, reverse-engineer the kernel queue ioctl interface, and author an LD_PRELOAD C shim to get ROCm llama.cpp working on my Strix Halo. My jaw was hanging open the whole time.
Edit to add the fix: <a href="https://gist.github.com/birep/6f2c8d490c7a29820997d57bd654c351" rel="nofollow">https://gist.github.com/birep/6f2c8d490c7a29820997d57bd654c3...
My experience with Gemini 3.8 Flash has been awful; it gives me the most hallucinations out of the major models. I'm not using it for coding, but general research on different topics.
Hallucination is accurate for what I'm seeing -- e.g. it's making up information about the 2nd gen Toyota Tundra that has no basis in reality. When challenged, it corrects itself.
My colleague wanted to diagnose a specific error code on his car himself, and Gemini told him that it's simple to do with an OBD2 dongle - he asked it about the details thoroughly, to confirm, and bought the dongle.
It didn't work. Gemini: "Oh yeah, that obviously cannot work, it's not possible to do it through OBD2" (paraphrasing)
It was quite funny to me, but a bit less so to my colleague.
Its always been a bad term.
If we wanted an accurate term then it's 'confabulation', but 'muddled' or just 'wrong' are also good.
What do we use for the “model made stuff up and claimed it as facts”? I can see hallucinations somehow anthropomorphizing LLM even more. I don’t like that we’re doing that to begin with but it’s a losing battle. I prefer “it’s broken” and “IT produced shit results” personally.
Models don't make claims. That would require a degree of interiority and intent that they don't have.
The bigger problem is that people expect LLMs to know what facts are. That assumption is even baked into the term "hallucination." Someone who hallucinates is expected to otherwise have a grounding in objective reality, to "not" hallucinate, and to be able to recognize reality from fantasy. We wouldn't allow a person who "hallucinates" as much as an LLM anywhere near the roles we give to LLMs. But everything an LLM does is as much a "hallucination" as anything else, it's just stochastically generating grammar. Some grammar just happens to be useful because of the quality of its training data, which was probably created by humans who do possess interiority and awareness of fact.
And it isn't "broken" either. Broken assumes that the correct mode of operation is to act as a source of truth or fact generation. When LLMs "apologize" for bad results, for instance they aren't actually apologizing. Try getting it to apologize for returning the correct data. It probably will. There is no cognition happening. It doesn't know either way. It isn't a calculator crunching numbers or a computer doing data analysis. It's just pattern matching.
"Hallucination" is no less correct than "confabulation" which also presupposes intent and contextual awareness. Unfortunately the way LLMs operate is so unintuitive (as opposed to the intuitive nature of the interface) that the only language we have to describe it is the language of human behavior, with all of the biases and false assumptions that brings.
>The knowledge cutoff date for Gemini 3.8 Flash is March 2026 – users can expect updated information for some domains while in others they may experience the model’s knowledge is limited to January 2025 (in line with the Gemini 3 Model Family). For more information about known limitations, see the Gemini 3.7 Flash
The "some domains" are very narrow. They likely just RL'ed popular queries.
While that's annoying, the other frontier models easily overcome this with appropriate tool usage. I do a lot of research with frontier models and they're very good about identifying where their parametric knowledge is insufficient and searching for the correct knowledge on the internet. 3.8 Flash is HORRIFIC. The majority of the time it doesn't use any tools and infers things from its parametric knowledge. Things which should clearly have implied tool calls. Historical statistics, legal precedent, economic data, etc. I think it's incredibly clear that it has been tuned for speed and not accuracy.
Of course, it's called "flash," and that implies its purpose. I have little use for speed and a LOT of use for accuracy, so I'm hopeful 4.0 is much better. I saw a benchmark earlier today showing that it is much less prone to hallucinations. Let's see.
I agree. It's much worse than the cheap Chinese models. They appear to heavily bias parametric knowledge and discourage tool use. That's fine for things like "how do I perform CPR?" but worse than useless for any kind of research. [There is one benchmark showing far lower rates of hallucination, so let's see how accurate this is.](<a href="https://www.reddit.com/r/singularity/comments/1wuj72j/gemini_4_argon_solved_hallucinations/" rel="nofollow">https://www.reddit.com/r/singularity/comments/1wuj72j/gemini...)
taylorfinley · · focus · HN ↗
Edit to add the fix: <a href="https://gist.github.com/birep/6f2c8d490c7a29820997d57bd654c351" rel="nofollow">https://gist.github.com/birep/6f2c8d490c7a29820997d57bd654c3...
gottorf · · focus · HN ↗
staticman2 · · focus · HN ↗
MILP · · focus · HN ↗
robobo96 · · focus · HN ↗
mattjoyce · · focus · HN ↗
nkozyra · · focus · HN ↗
xdavidliu · · focus · HN ↗
- AI is just a tool, like excel; it does what the human operating it tells it to
- next token prediction cannot be true understanding
- models can have no desires and goals, don't anthropomorphize it
However, "hallucination" is very much not one of them
gottorf · · focus · HN ↗
alluro2 · · focus · HN ↗
It didn't work. Gemini: "Oh yeah, that obviously cannot work, it's not possible to do it through OBD2" (paraphrasing)
It was quite funny to me, but a bit less so to my colleague.
Gareth321 · · focus · HN ↗
mattjoyce · · focus · HN ↗
rdtsc · · focus · HN ↗
mattjoyce · · focus · HN ↗
krapp · · focus · HN ↗
The bigger problem is that people expect LLMs to know what facts are. That assumption is even baked into the term "hallucination." Someone who hallucinates is expected to otherwise have a grounding in objective reality, to "not" hallucinate, and to be able to recognize reality from fantasy. We wouldn't allow a person who "hallucinates" as much as an LLM anywhere near the roles we give to LLMs. But everything an LLM does is as much a "hallucination" as anything else, it's just stochastically generating grammar. Some grammar just happens to be useful because of the quality of its training data, which was probably created by humans who do possess interiority and awareness of fact.
And it isn't "broken" either. Broken assumes that the correct mode of operation is to act as a source of truth or fact generation. When LLMs "apologize" for bad results, for instance they aren't actually apologizing. Try getting it to apologize for returning the correct data. It probably will. There is no cognition happening. It doesn't know either way. It isn't a calculator crunching numbers or a computer doing data analysis. It's just pattern matching.
"Hallucination" is no less correct than "confabulation" which also presupposes intent and contextual awareness. Unfortunately the way LLMs operate is so unintuitive (as opposed to the intuitive nature of the interface) that the only language we have to describe it is the language of human behavior, with all of the biases and false assumptions that brings.
UpsideDownRide · · focus · HN ↗
WarmWash · · focus · HN ↗
I'm assuming that Argon has at least a June 2026 date, but man, the 3 series models were a mess with newer information.
blinding-streak · · focus · HN ↗
> The knowledge cutoff date for Gemini 3.8 Flash is March 2026
<a href="https://deepmind.google/models/model-cards/gemini-3-8-flash/" rel="nofollow">https://deepmind.google/models/model-cards/gemini-3-8-flash/
WarmWash · · focus · HN ↗
The "some domains" are very narrow. They likely just RL'ed popular queries.
Gareth321 · · focus · HN ↗
Of course, it's called "flash," and that implies its purpose. I have little use for speed and a LOT of use for accuracy, so I'm hopeful 4.0 is much better. I saw a benchmark earlier today showing that it is much less prone to hallucinations. Let's see.
esafak · · focus · HN ↗
Gareth321 · · focus · HN ↗