Thread

Beating GPT-5.6 Sol on retrieval with 100x cheaper open models

415 points · 114 comments · moonikakiss

nullbio · 1785998912 · focus · HN ↗
I (and I imagine many others) would love to use something like this, but can't, because my data is too sensitive to be uploaded to a cloud of which I have no gaurantees of privacy/security.
Is there any way we do this using rented GPUs and open-source software stacks? Paying for the service isn't the issue, I don't care if it's free or if a cut is taken in some capacity, I just don't want the provider to have access to my data.
1. kumama · 1786027808 · focus · HN ↗
  castform founder here. unfortunately, we are cloud-hosted at this point.
  but some easy options on the open-source side include huggingface's trl & unsloth.
  you can run our data-generation scripts here: <a href="https://github.com/castform-ai/benchmax" rel="nofollow">https://github.com/castform-ai/benchmax and then hook it up trl/unsloth for training. should be able to do all of this on your own compute.

Unofficial Hacker News client; not affiliated with Y Combinator.