The moat for OAI and anthropic seems to be very quickly shrinking. Chinese labs are now using RSI-like approaches and even without resorting to heavy distillation they're catching up in a couple of months vs. what would have been 6-12 months a year prior.
And as these models get better the pace of training is quickly speeding up too.
This doesn't bode particularly well for anthropic/OAI after they go public.
token vendors are headed to the same place mobile data vendors went, this is good for everyone but those who thought they could maintain exorbitant prices
when mobile data first arrived, it was expensive and people bragged about their bills (token counts today)
with time, it became commoditized, people now have unlimited plans, and the money is made by the applications that sit on top (token generation is increasingly undifferentiated low-level infra)
This is not to say there has not been significant innovation in the time since, but it's a low margin business (tokens look to be headed this way)
I don't know if we will look at OpenAI and Anthropic as moating on frontier models & selling tokens.
They are banking on the application layer and accumulated business and end user context. They have to quickly make that systems integrated value out weigh the model choice price value in the broader market
alfalfasprout · · focus · HN ↗
And as these models get better the pace of training is quickly speeding up too.
This doesn't bode particularly well for anthropic/OAI after they go public.
verdverm · · focus · HN ↗
pvab3 · · focus · HN ↗
verdverm · · focus · HN ↗
with time, it became commoditized, people now have unlimited plans, and the money is made by the applications that sit on top (token generation is increasingly undifferentiated low-level infra)
This is not to say there has not been significant innovation in the time since, but it's a low margin business (tokens look to be headed this way)
mdale · · focus · HN ↗
They are banking on the application layer and accumulated business and end user context. They have to quickly make that systems integrated value out weigh the model choice price value in the broader market
Unknown if they will be able to pull that off.