One thing about these numbers that's absolutely shocking to me is how low the energy use is:
> That model’s usage was well within our budget ($68, about 4kWh of energy use / 365 grams of carbon emissions).
The energy cost is literally 1% of the total cost. For context, 4kWh of energy would drive you about 15 miles in an EV, about half of the average person's daily driving miles. It's boiling 10 gallons of water.
With the talk of AI data centers' impact on the world, you'd think this would be 10x to 100x the amount of energy in order to get the effects they're using here.
My takeaway: the AI data center buildout is an overbuild probably at least as large as the fiber buildout that left us with so much dark fiber. If not even bigger. The only thing that will save the economy is the inability of NVIDIA and chip fabs to produce enough chips to match the buildout planned.
I agree but it does worry me how fast my usage is increasing. Two months ago I was using 10x less tokens and probably not much more than 5kWh on inference. This month about 30kWh on inference. If it becomes more affordable, is there going to be another jump? Not quite sure
At some point the agents will be good enough that you can tell them "here's $100, go make me money" and they will, maybe not a lot and not all the time, but the EV given the cost of inference will be positive.
I guess the question boils down to, what remote work could a human do to earn money over the internet?
So how many companies/individuals pay a random spam bot contacting them 100€ to upgrade their website? I guess start with targeting gullible and trying to scam them will be the more successful experiment for them. (If run unrestricted)
Those rewards seem like they would be captured by the providers and AI companies though who would use them first to make themselves money. You would be left with whatever they didn’t pursue with their first movers advantage.
Then why would any AI company exist when they could just use their money to buy tokens from another AI company and make money for zero effort? There would be no incentive to be a provider.
Not to mention inflation would grow to match or outpace the rate you could earn on these guaranteed AI gains.
This seems somewhat obviously true, to some degree, but it all depends on the prompt, I think?
In the equilibrium, profits fall to zero in capitalism, but there's never equilibrium, everything is changing. Profit comes from insights that others haven't seen, from access to opportunities that others don't have, or through monopolistic control generating economic rents.
For that AI to make money, it has to have a harness that gives it one of the above things, which does seem possible.
But strictly time-limited, because as soon as someone else figures out what you are doing, there's no moat to prevent them from telling their AI to do it too.
Markets reach equilibrium very fast when everyone has access to the exact same tools and information.
It's a very very very rough approximation, but scaling laws (admittedly weaking over time) yield ~x^2 from a size, ~x^2 from a dynamic power, and more of the data movement is moving from PCB interconnect to on package which looks more similar to chip scaling.
It's a finger in the wind approximation with lots of confounders, but historically it seems to work out that way for most circuit things.
To put that in perspective my house uses an average of 1300kWh per month. Adding 30kWh per month would be an increase of 2.3% at a cost of around $4.20, not enough to notice.
for me it’s more an environmental consideration than financial? Passivhaus Standard for houses is about 50kWh/m²/year, so 30kWh is getting close to 10% of total
Do you live in the US? You should know in that case that the US is already one of the most wasteful energy users on the planet, far ahead of most other developed countries, even.
So potentially increasing energy usage even more, especially with likely token usage acceleration, is hardly reason for celebration.
No one is celebrating and using LLMs isn't necessarily a waste of energy. Where I live isn't relevant, LLMs use the same amount of power regardless of where their users are. You also know nothing about how I use electricity in my home, how I monitor my overall energy use and carbon output, and how I offset it.
> You also know nothing about how I use electricity in my home
I don't
> Where I live isn't relevant,
But where you live does matter. Average monthly household energy usage in the EU is 200-400 kWh. In the US it's more like 800-900kWh. So what you consider "normal energy usage" may vary considerably by where you live.
At 200-400kWh total usage, 30kwh starts to look a lot more significant. So you should at least consider the possibility that it's not that energy usage from AI is insignificant, but that your existing energy usage is wasteful.
You're American and the same 2.3% of energy you use would probably represents 5-10% of what the average Japanese person uses, and likely 15-20% of what the average Indian or Ethiopian uses.
My point isn't to attack you personally (I'd rather attack your country's disastrous energy policies) but to point out that LLMs will constitute a huge increase in our energy consumption at a time where we haven't transitioned to renewable energy and we're probably 20-30 away from it.
> my house uses an average of 1300kWh per month.
And the average European house uses like 3000kwh.... per year. We're already wasting insane amount of energy, adding 2-10% more is definitely huge
epistasis · · focus · HN ↗
> That model’s usage was well within our budget ($68, about 4kWh of energy use / 365 grams of carbon emissions).
The energy cost is literally 1% of the total cost. For context, 4kWh of energy would drive you about 15 miles in an EV, about half of the average person's daily driving miles. It's boiling 10 gallons of water.
With the talk of AI data centers' impact on the world, you'd think this would be 10x to 100x the amount of energy in order to get the effects they're using here.
My takeaway: the AI data center buildout is an overbuild probably at least as large as the fiber buildout that left us with so much dark fiber. If not even bigger. The only thing that will save the economy is the inability of NVIDIA and chip fabs to produce enough chips to match the buildout planned.
ThibWeb · · focus · HN ↗
CuriouslyC · · focus · HN ↗
pphysch · · focus · HN ↗
CuriouslyC · · focus · HN ↗
lukan · · focus · HN ↗
So how many companies/individuals pay a random spam bot contacting them 100€ to upgrade their website? I guess start with targeting gullible and trying to scam them will be the more successful experiment for them. (If run unrestricted)
mannanj · · focus · HN ↗
MisterMunchkin · · focus · HN ↗
CuriouslyC · · focus · HN ↗
monkpit · · focus · HN ↗
Then why would any AI company exist when they could just use their money to buy tokens from another AI company and make money for zero effort? There would be no incentive to be a provider.
Not to mention inflation would grow to match or outpace the rate you could earn on these guaranteed AI gains.
rhyperior · · focus · HN ↗
cyanydeez · · focus · HN ↗
gunalx · · focus · HN ↗
epistasis · · focus · HN ↗
In the equilibrium, profits fall to zero in capitalism, but there's never equilibrium, everything is changing. Profit comes from insights that others haven't seen, from access to opportunities that others don't have, or through monopolistic control generating economic rents.
For that AI to make money, it has to have a harness that gives it one of the above things, which does seem possible.
swiftcoder · · focus · HN ↗
But strictly time-limited, because as soon as someone else figures out what you are doing, there's no moat to prevent them from telling their AI to do it too.
Markets reach equilibrium very fast when everyone has access to the exact same tools and information.
a3dds · · focus · HN ↗
Take a vacation.
rapatel0 · · focus · HN ↗
skew-aberration · · focus · HN ↗
rapatel0 · · focus · HN ↗
It's a finger in the wind approximation with lots of confounders, but historically it seems to work out that way for most circuit things.
233mhz · · focus · HN ↗
My car engine is a miracle of efficiency compared to the stuff we had a hundred years ago, but overall cars pollute more than back then
driverdan · · focus · HN ↗
spiffytech · · focus · HN ↗
driverdan · · focus · HN ↗
ThibWeb · · focus · HN ↗
oblio · · focus · HN ↗
So potentially increasing energy usage even more, especially with likely token usage acceleration, is hardly reason for celebration.
driverdan · · focus · HN ↗
nicoburns · · focus · HN ↗
I don't
> Where I live isn't relevant,
But where you live does matter. Average monthly household energy usage in the EU is 200-400 kWh. In the US it's more like 800-900kWh. So what you consider "normal energy usage" may vary considerably by where you live.
At 200-400kWh total usage, 30kwh starts to look a lot more significant. So you should at least consider the possibility that it's not that energy usage from AI is insignificant, but that your existing energy usage is wasteful.
oblio · · focus · HN ↗
My point isn't to attack you personally (I'd rather attack your country's disastrous energy policies) but to point out that LLMs will constitute a huge increase in our energy consumption at a time where we haven't transitioned to renewable energy and we're probably 20-30 away from it.
[deleted] · · focus · HN ↗
[deleted]
233mhz · · focus · HN ↗
And the average European house uses like 3000kwh.... per year. We're already wasting insane amount of energy, adding 2-10% more is definitely huge