One thing about these numbers that's absolutely shocking to me is how low the energy use is:
> That model’s usage was well within our budget ($68, about 4kWh of energy use / 365 grams of carbon emissions).
The energy cost is literally 1% of the total cost. For context, 4kWh of energy would drive you about 15 miles in an EV, about half of the average person's daily driving miles. It's boiling 10 gallons of water.
With the talk of AI data centers' impact on the world, you'd think this would be 10x to 100x the amount of energy in order to get the effects they're using here.
My takeaway: the AI data center buildout is an overbuild probably at least as large as the fiber buildout that left us with so much dark fiber. If not even bigger. The only thing that will save the economy is the inability of NVIDIA and chip fabs to produce enough chips to match the buildout planned.
I agree but it does worry me how fast my usage is increasing. Two months ago I was using 10x less tokens and probably not much more than 5kWh on inference. This month about 30kWh on inference. If it becomes more affordable, is there going to be another jump? Not quite sure
At some point the agents will be good enough that you can tell them "here's $100, go make me money" and they will, maybe not a lot and not all the time, but the EV given the cost of inference will be positive.
I guess the question boils down to, what remote work could a human do to earn money over the internet?
So how many companies/individuals pay a random spam bot contacting them 100€ to upgrade their website? I guess start with targeting gullible and trying to scam them will be the more successful experiment for them. (If run unrestricted)
epistasis · · focus · HN ↗
> That model’s usage was well within our budget ($68, about 4kWh of energy use / 365 grams of carbon emissions).
The energy cost is literally 1% of the total cost. For context, 4kWh of energy would drive you about 15 miles in an EV, about half of the average person's daily driving miles. It's boiling 10 gallons of water.
With the talk of AI data centers' impact on the world, you'd think this would be 10x to 100x the amount of energy in order to get the effects they're using here.
My takeaway: the AI data center buildout is an overbuild probably at least as large as the fiber buildout that left us with so much dark fiber. If not even bigger. The only thing that will save the economy is the inability of NVIDIA and chip fabs to produce enough chips to match the buildout planned.
ThibWeb · · focus · HN ↗
CuriouslyC · · focus · HN ↗
pphysch · · focus · HN ↗
CuriouslyC · · focus · HN ↗
lukan · · focus · HN ↗
So how many companies/individuals pay a random spam bot contacting them 100€ to upgrade their website? I guess start with targeting gullible and trying to scam them will be the more successful experiment for them. (If run unrestricted)