Comment by sowbug
I wouldn't mind my own offline Gemini or ChatGPT 5. But even if the hardware and model were free, I don't know how I'd afford the electricity.
I wouldn't mind my own offline Gemini or ChatGPT 5. But even if the hardware and model were free, I don't know how I'd afford the electricity.
I don't think I am. I don't think economies of scale on hardware will drive costs below free, and while subscription providers might be willing to offer services below the cost of the hardware that runs them, I don't think they'll offer services below the cost of the electricity that runs them.
And while data centers might sign favorable contracts, I don't think they are getting electricity that far below retail.
A single machine for personal inference on models of this size isn't going to idle at some point so high that electricity becomes a problem and for personal use it's not like it would be under load often and if for some reason you are able to keep it under heavy load presumably it's doing something valuable enough to easily justify the electricity.
If you can't afford the electricity to afford to run the model on free hardware, you'd certainly never be able to afford the subscription to the same product as a service!
But anyway, the trick is to run it in the winter and keep your house warm.