‹ BackHN Continuity

Thread

Show HN: How Stale Is Your AI? Release age and training cutoff for 20 models

85 points · 50 comments · joozio

  1. gjskngnf · · focus · HN ↗
    I remember when the US captured Venezuelan president Maduro, and when I posed a prompt related to this, the model said that’s pure fiction. I told it to double check. Still didn’t want to entertain the idea. It only acquiesced when I specifically directed it to check Reuters. I haven’t noticed this problem in months. Model cutoff seems to be less of a problem these days.
    1. Isamu · · focus · HN ↗
      >the model said that’s pure fiction.

      Were you expecting your model to be updated on current events? Why?

      Also the specific event you are referring to is a statistically very improbable event, prior to its actually happening.

      >It only acquiesced when I specifically directed it to check Reuters.

      Do all models do this? They check in with Reuters? Why would a model think that you asking about an extremely improbable event warranted reaching out to Reuters?

      1. mywittyname · · focus · HN ↗
        He asked it to double check. It's reasonable to expect the LLM to handle that trivial task.
      2. gjskngnf · · focus · HN ↗
        I was not expecting model weights to be updated on current events.

        It’s clearly warranted because a model that trusts its weights on current events will give an outdated answer. Extremely improbable events happen all the time.

        1. xienze · · focus · HN ↗
          I think the models are trying to optimistically avoid doing web searches, because they're surprisingly a lot harder to do well than you'd think.
          1. DenisM · · focus · HN ↗
            How so?
      3. SirMaster · · focus · HN ↗
        If OpenAI is going to call Astra AGI, then I would expect it to be able to update it's weights to new knowledge, because a generally intelligent being can indeed do this.

        I can teach myself to play an instrument, and I'm not just building this huge lookup table that I have to access every time I play the instrument. I am updating the weights in my neurons.

        Until AI can do this it's not AGI in my book.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.