Thread

Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

415 points · 114 comments · moonikakiss

nullbio · 1785998912 · focus · HN ↗
I (and I imagine many others) would love to use something like this, but can't, because my data is too sensitive to be uploaded to a cloud of which I have no gaurantees of privacy/security.
Is there any way we do this using rented GPUs and open-source software stacks? Paying for the service isn't the issue, I don't care if it's free or if a cut is taken in some capacity, I just don't want the provider to have access to my data.
1. i5heu · 1786000576 · focus · HN ↗
  In OpenRouter there are Zero Retention options. And if you use a EU provider you can be somewhat sure that your privacy is given.
  Other than that there is not really a difference to renting a GPU since the GPU provider can also just steal your data.
  Local GPU(s) are always an option if you have the possibility. It is also not that difficult to run with stuff like “LocalAI”

Unofficial Hacker News client; not affiliated with Y Combinator.