I'm also grateful to the Chinese labs for providing workarounds for the walled gardens that the US based AI companies are attempting to create.
Does anyone know if there are any distillation datasets available? I'd love to see these distributed on BitTorrent. I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
> I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
While I'd like to agree with this, the fact is that pushing the frontier out has always taken (and folks expect to continue to take) hundreds of millions/billions of dollars. Open source and distilled models can follow on for much cheaper, but it's hard to imagine the frontier ever being "democratized" given the huge sums of money required. It was this realization that forced OpenAI to take tons of private investment in the first place.
That's ok. They took a few trillion worth of content and burned a couple of hundred billion of their own money. I really don't see the problem, if they didn't think about this long and hard before they went down that part it should not be on the rest of us to bail them out. The 'frontier' is less important than the democratization process.
The amount of progress that came out of academia and other public sources should not be underestimated either and without all that OpenAI and Anthropic wouldn't even exist.
slowin · · focus · HN ↗
Does anyone know if there are any distillation datasets available? I'd love to see these distributed on BitTorrent. I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
hn_throwaway_99 · · focus · HN ↗
While I'd like to agree with this, the fact is that pushing the frontier out has always taken (and folks expect to continue to take) hundreds of millions/billions of dollars. Open source and distilled models can follow on for much cheaper, but it's hard to imagine the frontier ever being "democratized" given the huge sums of money required. It was this realization that forced OpenAI to take tons of private investment in the first place.
jacquesm · · focus · HN ↗
The amount of progress that came out of academia and other public sources should not be underestimated either and without all that OpenAI and Anthropic wouldn't even exist.