Why is it a problem that the Chinese labs are just distilling down Anthropic’s models? Aren’t Anthropic’s models not just distilling down other people’s work?
Feels like Anthropic crying do as I say not as I do.
What Anthropic is doing requires way more resources than what the Chinese labs are doing. So their complaint is that they do 95% of the work and the Chinese labs do the last 5% and call it their own.
An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.
And the communal work of humanity is orders of magnitude more work than what anthropic pays for their scraping of content. I got no check from them for my contributions
Indeed, if it isn't a crime to train on humanity's data, it isn't a crime to train on capitalism arranged frontier LLM provider models. Is that bad for shareholders and capitalism? Meh, sounds like a suboptimal socioeconomic systems issue. Burn up all the capital the unsophisticated are willing to provide. “We are selling to willing buyers at the current fair market price.”
With my apologies to Brewster Kahle, "Universal Access to All Knowledge."
I use crime in the broad sense of "You shouldn't be allowed to do that" in this context. If you have a better word to capture that thought, let me know, I'll make the edit ("frowned upon" perhaps?). I don't have strong feelings other than "hah AI companies aren't going to be able to create a moat to capture the value they want to capture because we can collectively keep pulling it out of their models in perpetuity through ever improving model distillation methodologies". This is no different than Uber and DoorDash using VC dollars to subsidize services until they try to turn the knob to profitability once they've captured the market, except in this case, there are mechanisms to exfiltrate the model value into open models that can be distributed at very small marginal cost. They can never gate the golden goose money printer, they can only complain it isn't fair they aren't able to.
I have far less of a problem with the Chinese models if they even do this because they are making their models free, whereas the large Western LLM providers throw out a few bits but not their main work product. So not only are they hypocritical, I'm pretty sure that if the Chinese models were not released into the wild they would be making less noise.
Oh yeah, totally agree, I'd even rather pay those building the Chinese models if I didn't think I'd get thrown into a US gulag for felony contempt of business model.
cmiles8 · · focus · HN ↗
Feels like Anthropic crying do as I say not as I do.
jedberg · · focus · HN ↗
An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.
bushbaba · · focus · HN ↗
toomuchtodo · · focus · HN ↗
With my apologies to Brewster Kahle, "Universal Access to All Knowledge."
<a href="https://www.youtube.com/watch?v=RV_ALlJGU_c" rel="nofollow">https://www.youtube.com/watch?v=RV_ALlJGU_c
abcthingx · · focus · HN ↗
toomuchtodo · · focus · HN ↗
"The Spice must flow."
voiceofchoice · · focus · HN ↗
jacquesm · · focus · HN ↗
toomuchtodo · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]