> Before approving construction, I would want communities of humans to understand why the design works and what justifies confidence in its safety. I would hope that we all would.
Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny. I would usually catch issues with every response. I'm catching fewer problems these days. Maybe the model is just getting better, and maybe I'm being less careful while under pressure to ship more and more often. But model capability is obviously growing. Even back in March, you could tell it "give me a function that adds two numbers" and you could be 100% confident that it would write the correct function. There was almost no point in looking at the code. Since then, the complexity floor of problems in the category "this is so simple that the model couldn't possibly get it wrong" is rising, and with it, my cognitive surrender to the model is increasing too. Why check it? It's obviously going to be correct.
If AI designs a terawatt fusion plant, then of course we're going to meticulously pore over every detail to ensure safety, reliability, efficiency, whatever. If we find no flaws in the design whatsoever, will we be less careful about the second one? The third one? What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?
Terence Tao is arguing that the human involvement in research is crucial, but doesn't convincingly justify why, in my opinion. He says that "human agency is a value of fundamental importance" and that we will need to build "thriving human communities that can understand [AI ideas] together" - not for the sake of correctness, which AI may surpass us on, but for, I guess, the possibility of reclaiming human meaning and purpose. I don't disagree with this at all, but it's not an argument, it's a statement of values. Unfortunately, the stark reality is that if AI does surpass humans, it will become the economically dominant strategy to not verify them and not double check them, but to just do whatever they say. This seems like a great way to raise p(doom). But as the models get better and better, and as I'm scrutinizing Claude's output less and less... I just hope that there are more Terence Taos out there than people like me.
> What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?
For what it's worth, ten thousand terawatt fusion plants probably approaches the level at which the sheer intensity of energy production would cause significant disruption to the climate (it's roughly 5% of the Earth's entire solar input). Every energy source becomes dirty past a certain point. It would be wiser to learn how to build a utopia within a limited energy budget than find a way to produce enough of it to cook the damn planet, but who am I kidding, we're going to build a million of these things.
We're not going stop doing something locally just because we can do it in space. There are definite physical advantages to producing next to consumption. We will most likely do both -- things that can be done in space will be done in space, and things that are better done locally will ramp up. There is no upper limit to energy demand, none whatsoever. Stopping production anywhere it could theoretically happen is a matter of will.
The more I think about space elevator, the less I believe in it.
The material alone is in a quantity beyond what we can reasonably manufacture.
and the material needs to be perfect. All design we have today have cascade failure mode -- any material failure translates to a total catastrophic failure.
and geostationary does not really meant Geostationary. There are lots of jiggling everywhere. It wear down over time. and let's hope nothing resonance
and we need some maintenance / decommission plan. How can we decommission this when it fail or need upgrade?
It's quite possible that with "infinite" energy we could do mass scale carbon capture to offset the increased temperatures by rolling back the greenhouse effect.
This is basic thermodynamics, not greenhouse gases. If you put some amount of energy into a system with a fixed surface area, you will get a minimum equilibrium temperature.
Greenhouse gases and Earth’s internal nuclear decay engine make the situation worse, but even without them this would boil the oceans.
Even today we're recognizing that compute does not need to happen here.
When we're talking about creating powerplants equal to roughly 5% of the insolation of Earth, I think we're sci-fi enough to discuss orbital datacenters or Mars datacenters or Jupiter fusion candle datacenters.
Compute doesn't need to happen here, but it can, so it will. Building datacenters on Mars or Jupiter doesn't reduce our ability to build datacenters here, so naturally, they will still be built, everywhere that they can be. It's an orthogonal capability.
Maybe, but there are counterexamples today. The processing of raw materials can be done all over the world, but it's not. Modern nations actually disassemble their blast furnaces.
Orbital datacenters suffer from the same problem: limited surface area. You think cooling is a problem now? Try not having conduction or convection. Radiation is all you get. A huge proportion of the mass and size of the ISS is radiators, and that only has to deal with the heat of low-power electronics and a few measly human beings.
Using an entire Mars only nets us about a 2x multiplier for our energy expenditure budget. Then we need four planets to double it again. Exponential growth is a bitch.
Thermodynamics and the tyranny of exponential growth are going to win this battle every time, regardless of the unobtanium technology you try to invent.
Not quite the same problem, because only a small portion of that energy radiated from space will make it back to the environment on Earth. For convection that is 100%.
> Using an entire Mars only nets us about a 2x multiplier for our energy expenditure budget.
That is not the problem being discussed. GPUs can survive at temperatures most organisms cannot. Radiators can be scaled up, but the thermal mass of Earth cannot.
I am quite certain that you are drastically underestimating the scope of the problem of "getting rid of heat" when you're dealing with the discussed levels of power production.
Maybe, but that's a "simple" resource problem. The Earth radiates 20x more heat than that today, and it has not been optimized for purpose. Remember that the initial assumptions were that AI has already designed and manufactured tens of thousands of these plants.
The problem at hand is not just getting rid of heat however, it's getting rid of heat without cooking the Earth. I still fail to see how generating the heat off planet fails to solve that.
pyridines · · focus · HN ↗
Until very recently, I pored over every single line of code Claude generated with razor sharp scrutiny. I would usually catch issues with every response. I'm catching fewer problems these days. Maybe the model is just getting better, and maybe I'm being less careful while under pressure to ship more and more often. But model capability is obviously growing. Even back in March, you could tell it "give me a function that adds two numbers" and you could be 100% confident that it would write the correct function. There was almost no point in looking at the code. Since then, the complexity floor of problems in the category "this is so simple that the model couldn't possibly get it wrong" is rising, and with it, my cognitive surrender to the model is increasing too. Why check it? It's obviously going to be correct.
If AI designs a terawatt fusion plant, then of course we're going to meticulously pore over every detail to ensure safety, reliability, efficiency, whatever. If we find no flaws in the design whatsoever, will we be less careful about the second one? The third one? What about the ten thousandth one? Will "a nuclear fusion plant" become something that models couldn't possibly get wrong?
Terence Tao is arguing that the human involvement in research is crucial, but doesn't convincingly justify why, in my opinion. He says that "human agency is a value of fundamental importance" and that we will need to build "thriving human communities that can understand [AI ideas] together" - not for the sake of correctness, which AI may surpass us on, but for, I guess, the possibility of reclaiming human meaning and purpose. I don't disagree with this at all, but it's not an argument, it's a statement of values. Unfortunately, the stark reality is that if AI does surpass humans, it will become the economically dominant strategy to not verify them and not double check them, but to just do whatever they say. This seems like a great way to raise p(doom). But as the models get better and better, and as I'm scrutinizing Claude's output less and less... I just hope that there are more Terence Taos out there than people like me.
breuleux · · focus · HN ↗
For what it's worth, ten thousand terawatt fusion plants probably approaches the level at which the sheer intensity of energy production would cause significant disruption to the climate (it's roughly 5% of the Earth's entire solar input). Every energy source becomes dirty past a certain point. It would be wiser to learn how to build a utopia within a limited energy budget than find a way to produce enough of it to cook the damn planet, but who am I kidding, we're going to build a million of these things.
itishappy · · focus · HN ↗
breuleux · · focus · HN ↗
itishappy · · focus · HN ↗
breuleux · · focus · HN ↗
cindyllm · · focus · HN ↗
[dead]
stouset · · focus · HN ↗
RupertEisenhart · · focus · HN ↗
You aren't really trying in good faith to think this through are you? This idea is over half a century old. Not getting it by now is willful.
j16sdiz · · focus · HN ↗
The material alone is in a quantity beyond what we can reasonably manufacture.
and the material needs to be perfect. All design we have today have cascade failure mode -- any material failure translates to a total catastrophic failure.
and geostationary does not really meant Geostationary. There are lots of jiggling everywhere. It wear down over time. and let's hope nothing resonance
and we need some maintenance / decommission plan. How can we decommission this when it fail or need upgrade?
CuriouslyC · · focus · HN ↗
itishappy · · focus · HN ↗
stouset · · focus · HN ↗
Greenhouse gases and Earth’s internal nuclear decay engine make the situation worse, but even without them this would boil the oceans.
itishappy · · focus · HN ↗
When we're talking about creating powerplants equal to roughly 5% of the insolation of Earth, I think we're sci-fi enough to discuss orbital datacenters or Mars datacenters or Jupiter fusion candle datacenters.
breuleux · · focus · HN ↗
itishappy · · focus · HN ↗
stouset · · focus · HN ↗
Using an entire Mars only nets us about a 2x multiplier for our energy expenditure budget. Then we need four planets to double it again. Exponential growth is a bitch.
Thermodynamics and the tyranny of exponential growth are going to win this battle every time, regardless of the unobtanium technology you try to invent.
itishappy · · focus · HN ↗
> Using an entire Mars only nets us about a 2x multiplier for our energy expenditure budget.
We don't need to care about cooking Mars.
stouset · · focus · HN ↗
I’m assuming you care about melting the GPUs in your orbital datacenters.
> We don't need to care about cooking Mars.
I’m assuming you care about melting the GPUs in your Martian datacenters.
itishappy · · focus · HN ↗
stouset · · focus · HN ↗
itishappy · · focus · HN ↗
The problem at hand is not just getting rid of heat however, it's getting rid of heat without cooking the Earth. I still fail to see how generating the heat off planet fails to solve that.