Comment by kenjackson 18 hours ago
I don't understand the math about how we compute $80b for a gigawatt datacenter. What's the costs in that $80b? I literally don't understand how to get to that number -- I'm not questioning its validity. What percent is power consumption, versus land cost, versus building and infrastructure, versus GPU, versus people, etc...
First, I think it's $80b per 100 GW datacenter. The way you figure that out is a GPU costs $x and consumes y power. The $x is pretty well known, for example an H100 costs $25-30k and uses 350-700 watts (that's from Gemini and I didn't check my work). You add an infrastructure (i) cost to the GPU cost, but that should be pretty small, like 10% or less.
So a 1 gigawatt data center uses n chips, where yn = 1 GW. It costs = xi*n.
I am not an expert so correct me please!
The article says, "Kirshna said that it takes about $80 billion to fill up a one-gigawatt data center."
But thanks for you insight -- I used your basic idea to estimate and for 1GW it comes to about $30b just for enough GPU power to pull 1GW. And of course that doesn't take into account any other costs.
So $80b for a GW datacenter seems high, but it's within a small constant factor.
That said, power seems like a weird metric to use. Although I don't know what sort of metric makes sense for AI (e.g., a flops counterpart for AI workloads). I'd expect efficiency to get better and GPU cost to go down over time (???).
UPDATE: Below someone posted an article breaking down the costs. In that article they note that GPUs are about 39% of the cost. Using what I independently computed to be $30b -- at 39% of total costs, my estimate is $77b per GW -- remarkably close to the CEO of IBM. I guess he may know what he's talking about. :-)
> power seems like a weird metric to use
Because this technology changes so fast, that's the only metric that you can control over several data centers. It is also directly connected to the general capacity of data center, which is limited by available energy to operate.
To expand on rahimnathwani's comment below - the big capital costs of a data center are land, the building itself, the power distribution and the cooling.
You can get a lot of land for a million bucks, and it doesn't cost all that much to build what's basically a big 2-story warehouse, so the primary capital costs are power and cooling. (in fact, in some older estimates, the capital to build that power+cooling cost more per year than the electricity itself)
My understanding is that although power and cooling infrastructure are long-lived compared to computers, they still depreciate faster than the building, so they dominate costs even more than the raw price would indicate.
The state of the art in power and cooling is basically defined by the cost to feed X MW of computing, where that cost includes both capital and operation, and of course lower is better. That means that at a particular SOTA, and at an appropriate scale for that technology, the cost of the facility is a constant overhead on top of the cost of the equipment it houses. To a rough approximation, of course.
https://www.investing.com/news/stock-market-news/how-much-do...