In case you like the store kv to disk so you can resume I keep this branch of llama.cpp that includes that same functionality<a href="https://github.com/alainnothere/llama.cpp/commits/disk-cache-eviction" rel="nofollow">https://github.com/alainnothere/llama.cpp/commits/disk-cache...And you know it's load bearing each of the load baerings parts that bear some load and load a bear... you fight a bear because it took a load... or something like that...
Last time I was using llama cpp you could just do: llama_state_save_file or llama_state_seq_save_file and the load equivalents.That was a year or so ago though...
xlayn · · focus · HN ↗
<a href="https://github.com/alainnothere/llama.cpp/commits/disk-cache-eviction" rel="nofollow">https://github.com/alainnothere/llama.cpp/commits/disk-cache...
And you know it's load bearing each of the load baerings parts that bear some load and load a bear... you fight a bear because it took a load... or something like that...
jasonjmcghee · · focus · HN ↗
That was a year or so ago though...