‹ BackHN Continuity

Thread

Show HN: Mini-AGI – Dynamic continual learning model trained on 8GB VRAM

277 points · 81 comments · volotat

  1. ilaksh · · focus · HN ↗
    If you actually scroll through the transcript he links to, you will see that something that looks like it could be training is happening, but no coherent responses are coming out at any point. At least not that I saw skimming through.

    That might explain why there are no benchmarks of any kind.

    1. synctext · · focus · HN ↗
      Using the term AGI and not including any performance analysis. My AI calls it: "massive marketing overreach". Somebody called this slop in the comments.

      As a professor who published on continual learning I'm leaning towards agreement[1]. It lacks any substance. No relation to related work, no description of algorithm, no ablation study, just hand-waving that we're feeding some data and "Chess is not forgotten".

      This "how-continual-learning-works" markdown text is not an algorithm [2].

      [1] <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2301.12530" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2301.12530

      [2] <a href="https:&#x2F;&#x2F;github.com&#x2F;volotat&#x2F;mini-AGI&#x2F;#how-continual-learning-works" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;volotat&#x2F;mini-AGI&#x2F;#how-continual-learning-...

      1. ilaksh · · focus · HN ↗
        Actually I&#x27;m mad that I wasted my time looking at it based on the claims. He implies it is trained and uses the term &quot;AGI&quot; and &quot;continuous learning&quot;. He never finished a single training run or enough that he considers not &quot;undertrained&quot;. It&#x27;s not trained. And actually there is no evidence that it can actually learn anything useful.
        1. synctext · · focus · HN ↗
          Indeed this is wasting HN time.

          &quot;The model reads 524,000 characters of chess&quot;. This is 100KByte of training data in a toy model with rigid parameters and no global learning. Gap with real LLM and trillions of tokens.

          This model really addresses the problem of preserving previously learned knowledge, but by restricting the LR of the trunk it stops acquiring new knowledge. Details: &quot;Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective&quot;

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.