‹ BackHN Continuity

Thread

Show HN: Fine-tune an 8B model on a 4 GB laptop GPU

135 points · 28 comments · MakazhanAlpamys

  1. simonw · · focus · HN ↗
    There are some samples of training data in this folder - <a href="https:&#x2F;&#x2F;github.com&#x2F;MakazhanAlpamys&#x2F;Soup&#x2F;tree&#x2F;main&#x2F;examples&#x2F;data" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;MakazhanAlpamys&#x2F;Soup&#x2F;tree&#x2F;main&#x2F;examples&#x2F;d...

    They&#x27;re all very short though. Anyone got a good rule for how much data of this nature is needed to successfully fine-tune a model of this size?

    1. MakazhanAlpamys · · focus · HN ↗
      Those are format examples and test fixtures. Five to ten rows each. Not training data. You did spot a real problem though. Eight configs in `examples&#x2F;configs` pointed at those fixtures as training data. Seven were still on the old schema and would not even parse. I&#x27;ve fixed that, added a README to the folder, and a test that parses every config so it doesn&#x27;t quietly break again. Dataset size mostly depends on the task. Format or style, a few hundred examples is often enough. A task the model already half knows, usually a few thousand. New facts are often a bad fit for fine-tuning. I&#x27;d reach for RAG instead. I haven&#x27;t measured how this changes with model size, so I&#x27;m not going to make up a number.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.