Anthropic has to use this wedge (and future ones) to move regulatory action against the Chinese models or their IPO is going to be really problematic.
(Ironic, though, that I haven't heard of any Chinese models "escaping" which Anthropic and OpenAI both seem to have issues with...)
Like Chinese electric cars, the American producers cannot compete without regulatory action. Yes, I understand that the Chinese government this and that in both the automotive and AI industries.
But reality is what it is as a consumer: it's a cheaper product that's almost as good or better in some cases. And in the case of these open weight models: I can run it on my own infra and not give any data to anyone.
>> And in the case of these open weight models: I can run it on my own infra and not give any data to anyone.
It's worth noting that the overwhelming majority of people who use Chinese models don't do this. Yes, it is nice to have the option, and there are US-based inference providers that claim to not send your data to China and maybe indeed don't, but in the grand scheme of things, we need to remember the adage that became popular during the social media era: if something is free (or, in this case, close to free), you are the product.
I actually do do this. I'm not sure who the 'overwhelming majority' is and where you got the data (link would be appreciated) but everybody that I know that runs these is doing so on their own infra.
Context is useful. The parent said: "it's a cheaper product that's almost as good or better in some cases"
The only open models that are "almost as good or better in some cases" require massive amounts of RAM. I posit that most people cannot afford a decked out Mac Studio, and therefore run the smaller "flash" variants on more normal devices. The issue is that those are nowhere near frontier-level in terms of capability.
Precisely. I have figured out a nice recipe that is quite affordable, 288G of VRAM for a little under 20K, it takes some fiddling though, but once it works it is really neat.
CharlieDigital · · focus · HN ↗
(Ironic, though, that I haven't heard of any Chinese models "escaping" which Anthropic and OpenAI both seem to have issues with...)
Like Chinese electric cars, the American producers cannot compete without regulatory action. Yes, I understand that the Chinese government this and that in both the automotive and AI industries.
But reality is what it is as a consumer: it's a cheaper product that's almost as good or better in some cases. And in the case of these open weight models: I can run it on my own infra and not give any data to anyone.
enraged_camel · · focus · HN ↗
It's worth noting that the overwhelming majority of people who use Chinese models don't do this. Yes, it is nice to have the option, and there are US-based inference providers that claim to not send your data to China and maybe indeed don't, but in the grand scheme of things, we need to remember the adage that became popular during the social media era: if something is free (or, in this case, close to free), you are the product.
jacquesm · · focus · HN ↗
enraged_camel · · focus · HN ↗
The only open models that are "almost as good or better in some cases" require massive amounts of RAM. I posit that most people cannot afford a decked out Mac Studio, and therefore run the smaller "flash" variants on more normal devices. The issue is that those are nowhere near frontier-level in terms of capability.
CharlieDigital · · focus · HN ↗
Not just your local machines.
Enterprises are where you see this adoption. Legal, finance, tax; sensitive context where the data must be contractually opaque to external parties.
jacquesm · · focus · HN ↗
PCIe is incredibly powerful tech.