Love pi. I tried to run some local models and pi was the only one that actually worked decently because it didn’t have a gargantuan system prompt that would take minutes to prefill on my scrawny ass laptop.
Been running it almost barebones vanilla for a couple of months. Just a bunch of basic extensions and some skills.
Now, if only they could fix the very annoying bug of the history jumping back at the beginning if I am not a the end while the model is reasoning that would great.
You inspired me to try Pi out - so far it's worked flawlessly. Plugged it into OpenRouter and ~$.50 of Deepseek later I've installed llama.cpp and Llama 3.1. The local model doesn't work with Pi yet (and I know it will be bad and slow even if it does) but I'm curious to see what you can do on an 8GB consumer GPU these days...
If you paid DeepSeek directly, that would have been 1 to 10 cents. OpenRouter has a huge overhead due to their cache logic, I'm surprised they keep business coming in the door for tasks other than system prompt - output pairs.
I thought openrouter just routes you to the same provider for the rest of the session, so that you keep hitting the same cache. Is that not the case?
Also, I’d love to use Deepseek directly (or any of the Chinese providers, at that). Seems only fair to pay the lab that built the model. Unfortunately, any requests to Chinese servers is deeply frowned upon here (Belgium, EU). For personal use: sure. As a token intelligence strategy for the company: absolutely fucking not.
FacelessJim · · focus · HN ↗
Been running it almost barebones vanilla for a couple of months. Just a bunch of basic extensions and some skills.
Now, if only they could fix the very annoying bug of the history jumping back at the beginning if I am not a the end while the model is reasoning that would great.
simpaticoder · · focus · HN ↗
whatshisface · · focus · HN ↗
lemontheme · · focus · HN ↗
Also, I’d love to use Deepseek directly (or any of the Chinese providers, at that). Seems only fair to pay the lab that built the model. Unfortunately, any requests to Chinese servers is deeply frowned upon here (Belgium, EU). For personal use: sure. As a token intelligence strategy for the company: absolutely fucking not.
gigatexal · · focus · HN ↗