Love pi. I tried to run some local models and pi was the only one that actually worked decently because it didn’t have a gargantuan system prompt that would take minutes to prefill on my scrawny ass laptop.
Been running it almost barebones vanilla for a couple of months. Just a bunch of basic extensions and some skills.
Now, if only they could fix the very annoying bug of the history jumping back at the beginning if I am not a the end while the model is reasoning that would great.
You inspired me to try Pi out - so far it's worked flawlessly. Plugged it into OpenRouter and ~$.50 of Deepseek later I've installed llama.cpp and Llama 3.1. The local model doesn't work with Pi yet (and I know it will be bad and slow even if it does) but I'm curious to see what you can do on an 8GB consumer GPU these days...
If that's 8GB is available dedicatedly for the model then a lot but if it's the sad story like my M1 Pro where even wtih 16GB unififed I've barely anything left for myself.
You should go to huggingface and maybe create an a/c with a throwaway email and enter your hardware details and that will filter the models for you.
FacelessJim · · focus · HN ↗
Been running it almost barebones vanilla for a couple of months. Just a bunch of basic extensions and some skills.
Now, if only they could fix the very annoying bug of the history jumping back at the beginning if I am not a the end while the model is reasoning that would great.
simpaticoder · · focus · HN ↗
crossroadsguy · · focus · HN ↗
You should go to huggingface and maybe create an a/c with a throwaway email and enter your hardware details and that will filter the models for you.