Comment by alecco a day ago
Jensen knows what he is doing with the CUDA stack and workstations. AMD needs to beat that more than thinking about bigger hardware. Most people are not going to risk years learning an arcane stack for an architecture that is used by less than 10% of the GPGPU market.
Indeed. the user-facing software stack componentry - pytorch and jax/xla - are owned by meta, and google and open sourced. Further, the open-source models (llama/deepseek) are largely hw agnostic. There is really no user or eco-system lock-in. Also, clouds are highly incentivized to have multiple hardware alternatives.
There already is ROCm support for PyTorch. Then there's stuff like this: https://semianalysis.com/2024/12/22/mi300x-vs-h100-vs-h200-b...
They have improved since that article, by a decent amount from my understanding. But by now, it isn't enough to have "a backend". The historical efforts have spoiled that narrative so badly that it won't be enough to just have a pytorch-rocm pypi package; some of that flak is unfair though not completely unsubstantiated. But frankly they need to deliver better software, across all their offerings, for multiple successive generations before the bad optics around their software stack will start fading. Their competitors are already on their next gen architecture since that article was written.
You are correct that people don't really invoke CUDA APIs much, but that's partially because those APIs actually work and deliver good performance, so things can actually be built on top of them.
What fraction of Nvidia revenue comes from those applications?
Additionally when people discuss CUDA they always think about C, ignoring that has been a C++ first since CUDA 3.0, also has Fortran surpport, and NVidia always embraced having multiple languages being able to play on PTX land as well.
And as of 2025, there is a Python CUDA JIT DSL as well.
Also, even if not the very latest version, the fact that CUDA SDK works on any consumer laptop with NVidia hardware, anyone can slowly get into CUDA, even if their hardware isn't that great.
At this point it looks to me like something is seriously broken internally at AMD resulting in their software stack being lacklustre. They’ve had a lot of time to talk to customers about their problems and spin up new teams, but as far as I’ve heard there’s been very little progress, despite the enormous incentives. I think Lisa Su is a great CEO but perhaps not shaking things up enough in the software department. She is from a hardware background after all.
There used to be a time when hw vendors begudgingly put out sample driver code which contained 1 file with 5000 lines of C code - which just about barely worked. The quality of software was not really a priority, as most of the revenue was from hw sales. That reflected in the quality of hires and incentive structures.
Indeed. The stories I hear about software support for their entry-level hardware aren't great. Having a good on-ramp is essential.
OTOH, by emphasizing datacenter hardware, they can cover a relatively small portfolio and maximize access to it via cloud providers.
As much as I'd love to see an entry-level MI350-A workstation, that's not something that will likely happen.
I'm willing to bet almost nobody you know calls the CUDA API directly. What AMD needs to focus on is getting the ROCm backend going for XLA and PyTorch. That would unlock a big slice of the market right there.
They should also be dropping free AMD GPUs off helicopters, as Nvidia did a decade or so ago, in order to build up an academic userbase. Academia is getting totally squeezed by industry when it comes to AI compute. We're mostly running on hardware that's 2 or 3 generations out of date. If AMD came with a well supported GPU that cost half what an A100 sells for, voila you'd have cohort after cohort of grad students training models on AMD and then taking that know-how into industry.