Comment by chii 7 hours ago
> the secondary market is still alive.
this is the crux. Will these data center cards, if a newer model came out with better efficiency, have a secondary market to sell to?
It could be that second hand ai hardware going into consumers' hands is how they offload it without huge losses.
And the big ones don't even have typical PCIe sockets, they are useless outside of behemoth rackmount servers requiring massive power and cooling capacity that even well-equipped homelabs would have trouble providing!
Data centre cards a don’t have fans and don’t have video out these days.
Why would you do that when you can pay someone else to run the model for you on newer more efficient and more profitable hardware? What makes it profitable for you and not for them?
Ah well yes to a degree that’s possible but at least at the moment you’d still be better off buying a $5k Mac Studio if it’s just inference you’re doing
The GPUs going into data centers aren't the kind that can just be reused by putting them into a consumer PC and playing some video games, most don't even have video output ports and put out FPS similar to cheap integrated GPUs.