Thread

Show HN: Fine-tune an 8B model on a 4 GB laptop GPU

135 points · 28 comments · MakazhanAlpamys

bookmon · 1785860760 · focus · HN ↗
This looks cool - what 4 GB GPU laptop do you recommend?
1. MakazhanAlpamys · 1785865566 · focus · HN ↗
  Do not buy a 4 GB card for this. Mine is an RTX 3050 Laptop, I picked it because it is boring hardware that many people already have. If you are buying, buy VRAM. At 0.5B where I could measure both, resident training was 1.43x faster than streaming. And if you do stream, system RAM matters more, the base sits there and has to page-lock. 16 GB is about the floor for 8B.
  1. bookmon · 1785902172 · focus · HN ↗
    what price is the cheapest
    MakazhanAlpamys · 1785918374 · focus · HN ↗
    [dead]
    MakazhanAlpamys · 1785918802 · focus · HN ↗
    i don't follow gpu prices.
    the cheapest one is the one you already have.
    more vram for the money is what i'd look for. usually used :)
    bookmon · 1785943235 · focus · HN ↗
    cool thanks yeah I'll aim for a used one

Unofficial Hacker News client; not affiliated with Y Combinator.