Comment by fxtentacle
Comment by fxtentacle 16 hours ago
Yeah, I can rent an A100 server for roughly the same price as what the electricity would cost me.
Comment by fxtentacle 16 hours ago
Yeah, I can rent an A100 server for roughly the same price as what the electricity would cost me.
~$1.25-1.75/hr at Runpod or vast.ai for an A100
Edit: https://getdeploying.com/reference/cloud-gpu/nvidia-a100
The A100 SXM4 has a TDP of 400 watts, let's say about 800 with cooling etc overhead.
Bulk pricing per KWH is about 8-9 cents industrial. We're over an order of magnitude off here.
At 20k per card all in price (MSRSP + datacenter costs) for the 80GB version, with a 4 year payoff schedule the card costs 57 cents per hour (20,000/24/365/4) assuming 100% utilization.
Because they buy the electricity in bulk so these things are not the same.