Comment by jillesvangurp

I look at this as cost savings waiting to happen. Nvidia extorts companies to the extent of tens of thousands for a GPU. Somebody's going to undercut them. At the same time, people are working on optimizations as well. Using cheap CPUs for inference instead of expensive GPUs. Doesn't work for anything but if your model is small enough you can get away with it. Using lower bit quantization makes the models cheaper to run. Using hacks like prompt caching makes subsequent calls more efficient. Etc.

Your base assumption is that it is expensive and therefore these companies will eventually fail when they keep on making less money than they are spending. The reality is that they are indeed spending enormously now and making a lot of very non linear progress. At the same time a lot of that stuff is being widely published and quite a lot of it is open source. At some point you might get consolidation and maybe some companies indeed don't make it. But their core tech will survive. Investors might be crying in a corner. But that won't stop people from continuing to use the tech in some form or another.

I already have a laptop that can some modestly largish models locally. I'm not going to spend 40K or whatever on something that can run a GPT 5 class model. But it's not going to cost that in a few years either. This tech is here to stay. We might pay more or less for it. The current state is the worst it is ever going to be. It's going to be faster, bigger, better, cheaper, more useful, etc. At some point the curves flatten and people might start paying attention to cost more. Maybe don't burn a lot of gas in expensive and inefficient gas generators (as opposed to more efficient gas power plants) and maybe use cheap wind/solar instead. Maybe get some GPUs from a different vendor at a lower price? Maybe take a look at algorithm efficiencies, etc. There is a lot of room for optimization in this market. IMHO surviving companies will be making billions, will be running stuff at scale, and will be highly profitable.

Maybe some investors won't get their money back. Shit happens. That's why it's called venture capital. The web bubble bursting didn't kill the web either.