Comment by tough 15 days ago
Doesn't google have TPU's that makes inference of their own models much more profitable than say having to rent out NVDIA cards?
Doesn't OpenAI depend mostly on its relationship/partnership with Microsoft to get GPUs to inference on?
Thanks for the links, interesting book!
The only one who can stop Google is Google.
They’ll definitely have the best model, but there is a chance they will f*up the product / integration into their products.
It would take talent for them to mess up hosting businesses who want to use their TPUs on GCP.
But then again even there, their reputation for abandoning products, lack of customer service, condescension when it came to large enterprises’ “legacy tech” lets Microsoft who is king of hand holding big enterprise and even AWS run rough shod over them.
When I was at AWS ProServe, we didn’t even bother coming up with talking points when competing with GCP except to point out how they abandon services. Was it partially FUD? Probably. But it worked.
DEC, Sun, Intel, Blackberry, Motorola, Nokia: More talented in that area!
> It would take talent for them to mess up hosting businesses who want to use their TPUs on GCP. > But then again even there, their reputation for abandoning products
What are the chances of abandoning TPU-related projects where the company literally invested billions in infrastructure? Zero.
Google will win the LLM game if the LLM game is about compute, which is the common wisdom and maybe true, but not foreordained by God. There's an argument that if compute was the dominant term that Google would never have been anything but leading by a lot.
Personally right now I see one clear leader and one group going 0-99 like a five sigma cosmic ray: Anthropic and the PRC. But this is because I believe/know that all the benchmarks are gamed as hell, its like asking if a movie star had cosmetic surgery. On quality, Opus 4 is 15x the cost and sold out / backordered. Qwen 3 is arguably in next place.
In both of those cases, extreme quality expert labeling at scale (assisted by the tool) seems to be the secret sauce.
Which is how it would play out if history is any guide: when compute as a scaling lever starts to flatten, you expert label like its 1987 and claim its compute and algorithms until the government wises up and stops treating your success persobally as a national security priority. It's the easiest trillion Xi Xianping ever made: pretending to think LLMs are AGI too, fast following for pennies on the dollar, and propping up a stock market bubble to go with the fentanyl crisis? 9-D chess. It's what I would do about AI if I were China.
Time will tell.
I believe Google might win the LLM game simply because they have the infrastructure to make it profitable - via ads.
All the LLM vendors are going to have to cope with the fact that they're lighting money on fire, and Google have the paying customers (advertisers) and with the user-specific context they get from their LLM products, one of the juciest and most targetable ad audiences of all time.
This is one of the best insights after reading 100+ posts here. You are talking about existing demand and existing relationships from their advert business. These customers are happy with Google results. The Google marketing team will be carefully defining new advert products that employ LLMs.
I would only offer one disagreement with your post: There will not be a single winner in LLMs. The landscape is so large that we will have multiple winners in different areas. Example: Google might fail in B2C LLM (chatbot that answers your questions), but will certainly be (wildly?) successful in B2B for adverts.
Hm, yes I agree that winner-take-all is not necessary. A viable monetization strategy is. Meta and Google have this through their ad business, and I think it's possible that some other smaller players might bring in revenue by targeting particular niches like industrial and defense customers who need an LLM product but have data security needs and can afford the product directly.
Everyone seems to forget about Mu Zero which was arguably more important than transformer architecture.
Googles bread and butter is advertising, so they have a huge interest in keeping things in house. Data is more valuable to them than money from hardware sales.
Even then, I think that their primary use case is going to be consumer grade good AI on phones. I dunno why Gemma QAT model fly so low on the radar, but you can basically get full scale Llamma 3 like performance from a single 3090 now, at home.
https://www.cnbc.com/2025/04/09/google-will-let-companies-ru...
Google has already started the process of letting companies self-host Gemini, even on NVidia Blackwell GPUs.
Although imho, they really should bundle it with their TPUs as a turnkey solution for those clients who haven't invested in large scale infra like DCs yet.
It’s my understanding that google makes bulk of ad money from search ads - sure they harvest a ton of data but it isn’t as valuable to them as you’d think. I suspect they know that could change so they’re hoovering up as much as they can to hedge their bets. Meta on the other hand is all about targeted ads.
Right so keeping things in house and seeing what people are asking Gemini would be probably better for them?
Relenting hardware like that would be such a cleansing old-school revenue stream for Google... just imagine...
TPUs are accelerators that accelerate the common operations found in neural nets. A big part is simply a massive number of matrix FMA units to process enormous matrix operations, which comprises the bulk of doing a forward pass through a model. Caching enhancements and massively growing memory was necessary to facilitate transformers, but on the hardware side not a huge amount has changed and the fundamentals from years ago still powers the latest models. The hardware is just getting faster and with more memory and more parallel processing units. And later getting more data types to enable hardware-enabled quantization.
So it isn't like Google designed a TPU for a specific model or architecture. They're pretty general purpose in a narrow field (oxymoron, but you get the point).
The set of operations Google designed into a TPU is very similar to what nvidia did, and it's about as broadly capable. But Google owns the IP and doesn't pay the premium and gets to design for their own specific needs.
Im a research person building models so I can't answer your questions well (save for one part)
That is, as a research person using our GPUs and TPUs I see first hand how choices from the high level python level, through Jax, down to the TPU architecture all work together to make training and inference efficient. You can see a bit of that in the gif on the front page of the book. https://jax-ml.github.io/scaling-book/
I also see how sometimes bad choices by me can make things inefficient. Luckily for me if my code/models are running slow I can ping colleagues who are able to debug at both a depth and speed that is quite incredible.
And because were on HN I want to preemptively call out my positive bias for Google! It's a privilege to be able to see all this technology first hand, work with great people, and do my best to ship this at scale across the globe.
Yes. Google is probably gonna win the LLM game tbh. They had a massive head start with TPUs which are very energy efficient compared to Nvidia Cards.