Comment by rcarmo 15 hours ago
I would love to try out one of the mini-PCs that ship this, but they seem to be made of either platinum (hugely overpriced in EU) or unobtainium (no retailers carry them here, and getting something direct from China is dicey warranty-wise). ROCm 7 looks to be working already under most Linux distros and having this as a workstation with a local LLM or a “home inference server” with Ollama and a few services seems like a great solution.
I don't think there's any computer hardware that is now economical to buy and use a couple hours a month than to rent
ROCm is making great progress but I’ve had enough hiccups (desktop with RX9070XT) that I’d still recommend those looking for AI capability to continue using an Nvidia or Apple solution for the time being.
Still, I think it’ll be quite equivalent soon.
I think one of the best AI systems in terms of price/performance is still just to build a desktop with dual RTX 3090’s (of course you’ll need an board that supports dual cards) and toss it in a closet.
It depends on what you are doing. A lot of people who want to do local inference want to do it using much larger models than what can be fit onto a RTX3090, and Strix Halo is such a hit because it gives you reasonable (not great, but good enough to not be outright painful) performance with 128GB of memory.
Also, Vulkan is great, and much more stable. Plus tends to work great for new, and even very old, graphics cards.
> ROCm is making great progress
is the progress in the room with us?
Have you looked at Corsair's AI Workstation 300 Desktop PC? [1] It's 2000-2700 EURO depending on model, and taking VAT into consideration it's comparable to the 1700-2300 USD pretax prices.
[1]: https://www.corsair.com/eu/en/c/ai-workstations