Thread

1. euroderf · 1789742062 · focus · HN ↗

   Stupid question: Does "total model footprint of 5.9GB" mean it will run in 8GB of RAM ? Or is that the size on disk ?

   1. Aurornis · 1789743799 · focus · HN ↗

      That’s the size of the model. Running it requires additional space for the KV cache depending on how much context you use. You can probably get it running in 8GB of RAM for short outputs but to get okay context length you’d want more.

Unofficial Hacker News client; not affiliated with Y Combinator.