Thread

1. gjskngnf · 1789569601 · focus · HN ↗

   I remember when the US captured Venezuelan president Maduro, and when I posed a prompt related to this, the model said that’s pure fiction. I told it to double check. Still didn’t want to entertain the idea. It only acquiesced when I specifically directed it to check Reuters. I haven’t noticed this problem in months. Model cutoff seems to be less of a problem these days.

   1. super256 · 1789569887 · focus · HN ↗

      It's a "problem" of compute, I think. If you query without an account on ChatGPT you will see the model look up less stuff and research less, than when you have a paid account and choose "medium" or "high" in the effort slider.

      Which makes sense, because of you have looked into search and crawlers you notice that search is actual quite expensive (which is why e.g. Kagi charges a few bucks for search every month).

      1. Catloafdev · 1789573231 · focus · HN ↗

         It's not strictly compute, because this has noticeably improved in open-weight models too, such as Gemma and Qwen. I suspect they noticed this issue and adjusted their training to be better about it over time.

         1. ahmedfromtunis · 1789578108 · focus · HN ↗

            I built a toy news-summarizing agent with Gemma 4, and it was so frustrating, actually, because of the cut-off date.

            The model wasted over half the token budget, each time, on internal debates over the current date.

            When generating a World Cup summary, for example, it refused to believe qualification rounds were over and refused to even call the web searching tool to collect the data.

            I injected the current datetime at the very beginning of the system prompt, but Gemma refused to believe it!

            The m-effer insisted the timestamp was fake and hypothesized it was being evaluated in a synthetic lab test with simulated future dates!

            No amount of system prompting could convince it to trust the clock.

            That was the most frustrating and bizarre "bug" I ever faced!

            1. ThunderSizzle · 1789590228 · focus · HN ↗

               Did you try other models (e.g. Qwen3.6 or others)? I'm curious how others fare.

               I've noticed Qwen3.6 struggles a bit with today/date based logic.

            2. carlos_rpn · 1789590606 · focus · HN ↗

               Qwen 3.6 did the same thing for me.

               Only after some cajolling it finally went to check the history I asked it to (I had been testing KoboldCPP's web search).

Unofficial Hacker News client; not affiliated with Y Combinator.