I've built something similar, i am hosting it.
You can test Jev like model at 26B parameter count here (built few weeks ago):
<a href="https://gambler-relay-us-west1.leo-fish.ts.net/demo" rel="nofollow">https://gambler-relay-us-west1.leo-fish.ts.net/demo (might not stay up for long)
faangguyindia · · focus · HN ↗
You can test Jev like model at 26B parameter count here (built few weeks ago): <a href="https://gambler-relay-us-west1.leo-fish.ts.net/demo" rel="nofollow">https://gambler-relay-us-west1.leo-fish.ts.net/demo (might not stay up for long)
Typesafe compatible API
This is just running on old hardware.