Comment by sigmoid10

Comment by sigmoid10 5 hours ago

Making the hardware is actually the easy part. Everyone and their uncle who had some cash have tried by now: Microsoft, Meta, Tesla, Huawei, Amazon, Intel - the list goes on and on. But Nvidia is not a chip company. Huang himself said they are mostly a software company. And that is how they were able to build a gigantic moat. Because noone else has even come close on the software side. Google is the only one who has had some success on this side, because they also spent tons of money and time on software refinement by now, while all the other chips vanished into obscurity.

matwood 4 hours ago

Are you saying that Google, Meta, Amazon, etc... can't do software? It's the bread and butter of these companies. The CUDA moat is important to hold off the likes of AMD, but hardware like TPUs for internal use or other big software makers is not a big hurdle.

Of course Huang will lean on the software being key because he sees the hardware competition catching up.

Reply View 2 replies

qdotme 11 minutes ago

Essentially, yes, they haven’t done deep software. Netflix probably comes closest amongst FAANG.
Google, Meta, Amazon do “shallow and broad” software. They are quite fast at capturing new markets swiftly, they frequently repackage OpenSource core and add the large amount of business logic to make it work, but essentially follow the market cycles - they hire and layoff on a few year cycle, and the people who work there typically also will jump around industries due to both transferable skills and relatively competitive competitors.
NVDA is roughly in the same bucket as HFT vendors. They retain talent on a 5-10y timescales. They build software stacks that range from complex kernel drivers and hardware simulators all the way to optimizing compilers and acceleration libraries.
This means they can build more integrated, more optimal and more coherent solutions. Just like Tesla can build a more integrated vehicle than Ford.

Reply View | 0 replies
sigmoid10 2 hours ago

Huang said that many years ago, long before ChatGPT or the current AI hype were a thing. In that interview he said that their costs for software R&D and support are equal or even bigger than their hardware side. They've also been hiring top SWE talent for almost two decades now. None of the other companies have spent even close to this much time and money on GPU software, at least until LLMs became insanely popular. So I'd be surprised to see them catch up anytime soon.

Reply View | 0 replies

sanjayjc 5 hours ago

Genuine question: given LLMs' inexorable commoditization of software, how soon before NVDA's CUDA moat is breached too? Is CUDA somehow fundamentally different from other kinds of software or firmware?

Reply View 3 replies

tomrod 2 hours ago

Current Gen LLMs are not breaching the moat yet.

Reply View | 2 replies
- fzzzy an hour ago
  
  Yeah they are. llama.cpp has had good performance on cpu, amd, and apple metal for at least a year now.
  
  Reply View | 1 reply
  
  tomrod 5 minutes ago
  
  Thw hardware is not the issue. It's the model architectures leading to cascading errors
  
  Reply View | 0 replies