This is really a microcosm of one particular problem that AI presents to the world: what do people do when their labour is not required any longer?
Here the worry is the social structures of mathematics are eroded such that fewer humans become able to do the work and less well, similarly to how juniors are being recruited less in software engineering, breaking the ladder and leading to fewer seniors in the years to come.
The answer to this really depends strongly on what AI can actually accomplish, but I’ll assume the maximal case and say that AI can do everything economically necessary, and further even those things just desired, such that human labour isn’t required for anything that anyone wants in a practical sense.
Here, we don’t need a human understanding of mathematics to give people a perfect standard of living. We also don’t need humans involved with anything else.
Everything therefore becomes a hobby or a game. People do things because they enjoy them for their own sake, or because they are endeavours used as vehicles to socialise and enjoy others’ company, or because a shared social belief exists and is cultivated such that doing such and such a thing confers social status.
And I think that’s more or less it. I predict we may see some fairly strange sorts of things, such as games where the team structure looks like the descendant of a company org and they compete in an artificial economy. Likewise we may see gamified versions of universities and academia. All of these would be “tamed” such that the rougher parts of the experiences were sanded off.
Sort of like how we evolved in an ancestral environment, and we have certain drives and expectations driven by that environment even though they no longer matter for survival. Our social structures may be derived similarly from those of today, even after they have ceased to serve a real purpose, but changed and repurposed to give meaning and community.
I somewhat disagree. we still need humans to provide direction and context and expanded the concept of... well everything. But we will certainly need less of them. AI will afford the ability to more quickly disseminate state-of-the-art. As soon as a breakthrough is discovered, you no longer need to read 300 whitepapers to hopefully stumble upon it, AI will identify relevancy quicker.
AI didnt break through stavier-nokes until a human set the initial direction. That's not going to change.
>AI didnt break through stavier-nokes until a human set the initial direction. That's not going to change.
Why isn't this going to change?
Right now it's easy to see why don't set AI lose on every problem, someone human or AI has to delegate rare resources between competing interests. But if we look at general compute it was no different in the past. In 1980 you had to ask for permission to get CPU compute time. In 2026 you run it on your own computer, or maybe pay for it on Amazon. The constraints are much different.
If we keep pumping out chips and increasing efficieny someone will just make the LLM into an agentic loop (build the harness in) and set it lose on problems.
Nobody can avoid it. You can RL things in a box until the box is mapped perfectly but that doesn't mean you can go forever on out of distribution data. Our number of boxes will increase but model collapse is an open problem.
Astra goes farther than any model before but it cannot go forever.
Models have been trained on synthetic inputs for over a year now. Collapse only happens in very specific circumstances, it's not a limit in practice.
Chance-Device · · focus · HN ↗
Here the worry is the social structures of mathematics are eroded such that fewer humans become able to do the work and less well, similarly to how juniors are being recruited less in software engineering, breaking the ladder and leading to fewer seniors in the years to come.
The answer to this really depends strongly on what AI can actually accomplish, but I’ll assume the maximal case and say that AI can do everything economically necessary, and further even those things just desired, such that human labour isn’t required for anything that anyone wants in a practical sense.
Here, we don’t need a human understanding of mathematics to give people a perfect standard of living. We also don’t need humans involved with anything else.
Everything therefore becomes a hobby or a game. People do things because they enjoy them for their own sake, or because they are endeavours used as vehicles to socialise and enjoy others’ company, or because a shared social belief exists and is cultivated such that doing such and such a thing confers social status.
And I think that’s more or less it. I predict we may see some fairly strange sorts of things, such as games where the team structure looks like the descendant of a company org and they compete in an artificial economy. Likewise we may see gamified versions of universities and academia. All of these would be “tamed” such that the rougher parts of the experiences were sanded off.
Sort of like how we evolved in an ancestral environment, and we have certain drives and expectations driven by that environment even though they no longer matter for survival. Our social structures may be derived similarly from those of today, even after they have ceased to serve a real purpose, but changed and repurposed to give meaning and community.
superxpro12 · · focus · HN ↗
AI didnt break through stavier-nokes until a human set the initial direction. That's not going to change.
pixl97 · · focus · HN ↗
Why isn't this going to change?
Right now it's easy to see why don't set AI lose on every problem, someone human or AI has to delegate rare resources between competing interests. But if we look at general compute it was no different in the past. In 1980 you had to ask for permission to get CPU compute time. In 2026 you run it on your own computer, or maybe pay for it on Amazon. The constraints are much different.
If we keep pumping out chips and increasing efficieny someone will just make the LLM into an agentic loop (build the harness in) and set it lose on problems.
johnsmith1840 · · focus · HN ↗
If you solve it then yes humans are toast mental usefulness wise
ChickeNES · · focus · HN ↗
johnsmith1840 · · focus · HN ↗
Astra goes farther than any model before but it cannot go forever.
int_19h · · focus · HN ↗
johnsmith1840 · · focus · HN ↗
Sythetic data works in a bounded box we have mapped you can't just make up novel data and it works.
Model collapse is completely unsolved.