Aleph Alpha is just a sad joke by now. The talent isn't there anymore, they never managed to catch up to the other labs, failed to deliver on several projects, and by now are just a cash grab for the investors.
Only if you think in terms of short term gains, but thinking in the long run, doesn't really matter if these models are crap today, all models will eventually be obsolete, unless we plateau hard on every aspect of the tech.
What matters is to have good sovereign models in 5, 10, 20 years time.
I'm curious why you think sovereign models matter that much if open source ones exist (unless that stops at some point). Sovereign model hosting services, yes.
We have lots of capable open weights models, but very few have actual datasets and pipelines open for reproducibility. This is a major issue that even if big companies addressed, policies could well just change, as you also acknowledge.
Society can't depend on the whims of the private sector.
> Society can't depend on the whims of the private sector.
Yeah we can, we do it all the time. Nothing in society works without relying on private actors to keep doing what they have always done. Unless you go full soviet command economy, which in fact works worse.
Society and the private sector cooperates, but any technical dependency is detrimental because of monopolization of interests. You don't need to go full soviet, but knowledge in general must be socialized, that includes access to technology.
We literally grant 20 year monopolies on technologies to their inventors and have for centuries and it's fine. The entire economy runs on chips made by like, 3 foundries using lithography machines made by 1 company. I think you are overstating the necessity.
Autarky would be a national security disaster manifested. You can't be a modern prosperous society and not rely on either trade nor private enterprise.
Yes, society in general shouldn't, what I meant to say is that democratic society can't, otherwise it'll devolve invariably into an oligarchy.
Just open weights are not enough, I think as a nation state you'd at least want a whole sovereign training pipeline. Otherwise you are stuck when Kimi K4 stops releasing their weights.
But I don't see why this needs to be done on a state-by-state level, a sovereign EU model seems to be much more realistic in terms of funding.
While an intriguing concept, that doesn't sound like it could scale to the amount of "cores" needed to do the work. Or am I missing something?
You want to, of course. There are several multi-billion dollar programs to subsidize fabs starting european offshoots. But so far not very successful, and it's not like one endeavour is blocking the other here.
Open weight and -source models aren't without biases. Giving a model away for free would be a good way to distribute an agenda. With a sovereign model you can at least control the agenda and align it with your values.
Like it or not, AI has become part of modern warfare and well as part of the surveillance state (good for crime fighting, not for privacy), so it is an important part of national security, and you'd prefer to be as much self-sufficient for that as you can be.
Not all needs are going to be met by using or finetuning general purpose models, so ability to build your own SOTA ML models (LLMs or not) is also important.
Because if you can't build the model from scratch you don't have a model. Open weights are nice but fixed in time. This isn't a technology that stays static, it improves, and sovereignty means being able to continue to improve independently.
9dev · · focus · HN ↗
gchamonlive · · focus · HN ↗
What matters is to have good sovereign models in 5, 10, 20 years time.
roncesvalles · · focus · HN ↗
hypfer · · focus · HN ↗
The next thought is culture-specific workflows, requirements etc that are best trained into weights by people that actually understand those needs.
gchamonlive · · focus · HN ↗
Society can't depend on the whims of the private sector.
brainwad · · focus · HN ↗
Yeah we can, we do it all the time. Nothing in society works without relying on private actors to keep doing what they have always done. Unless you go full soviet command economy, which in fact works worse.
gchamonlive · · focus · HN ↗
brainwad · · focus · HN ↗
gchamonlive · · focus · HN ↗
brainwad · · focus · HN ↗
gchamonlive · · focus · HN ↗
<a href="https://news.ycombinator.com/item?id=49945572">https://news.ycombinator.com/item?id=49945572
ordersofmag · · focus · HN ↗
gchamonlive · · focus · HN ↗
HotHotLava · · focus · HN ↗
But I don't see why this needs to be done on a state-by-state level, a sovereign EU model seems to be much more realistic in terms of funding.
rixed · · focus · HN ↗
DoctorOetker · · focus · HN ↗
The annoying part of optical matrix multiplication is the conversion from electronic to optic and back.
Why not use electron-optics for matrix multiplication and use dynodes for amplification.
pstuart · · focus · HN ↗
HotHotLava · · focus · HN ↗
stefs · · focus · HN ↗
HarHarVeryFunny · · focus · HN ↗
Not all needs are going to be met by using or finetuning general purpose models, so ability to build your own SOTA ML models (LLMs or not) is also important.
mistrial9 · · focus · HN ↗
tensor · · focus · HN ↗