Comment by 0xbadcafebee

I felt anxious about all the insane valuations and spending around AI lately, and I knew it couldn't last (I mean there's only so much money, land, energy, water, business value, etc). But I didn't really know when it was going to collapse, or why. But recently I've been diving into using local models, and now it's way more clear. There seems to be a specific path for the implosion of AI:

- Nvidia is the most valuable company. Why? It makes GPUs. Why does that matter? Because AI is faster on them than CPUs, ASICs are too narrowly useful, and because first-mover advantage. AMD makes GPUs that work great for AI, but they're a fraction of the value of Nvidia, despite the fact that they make more useful products than Nvidia. Why? Nvidia just got there first, people started building on them, and haven't stopped, because it's the path of least resistance. But if Nvidia went away tomorrow, investors would just pour money into AMD. So Nvidia doesn't have any significant value compared to AMD other than people are lazy and are just buying the hot thing. Nvidia was less valuable than AMD before, they'll return there eventually; all AMD needs is more adoption and investment.

- Every frontier model provider out there has invested billions to get models to the advanced state they're in today. But every single time they advance the state of the art, open weights soon match them. Very soon, there won't be any significant improvement, and open weights will be the same as frontier, meaning there's no advantage to paying for frontier models. So within a few years, there will be no point to paying OpenAI, Anthropic, etc. Again, these were just first-movers in a commodity market. The value just isn't there. They can still provide unique services, tailored polished apps, etc (Anthropic is already doing this by banning users who have the audacity to use their fixed-price plans with non-Anthropic tools). But with AI code tools, anyone can do this. They are making themselves obsolete.

- The final form of AI coding is orchestrated agent-driven vibe-coding with safeguards. Think an insane asylum with a bowling league: you still want 100 people to autonomously (and in parallel) knock the pins knocked over, but you have to prevent the inmates from killing anyone. That's where the future of coding is. It's just too productive to avoid. But with open models and open source interfaces, anyone can do this, whether with hosted models (on any of 50 different providers), or a Beowulf cluster of cobbled together cheap hardware in a garage.

- Eventually, in like 5-10 years (a lifetime away), after AI Beowulfs have been a fad for a while, people will tire of it and move back to the cloud, where they can run any model they want on a K8s cluster full of GPUs, basically the same as today. Difference between now and then is, right now everyone is chasing Anthropic because their tools and models are slightly better. But by then, they won't be. Maybe people will use their tools anyway? But they won't be paying for their models. And it's not just price: one of the things you learn quickly by running models, is they're all good for different things. Not only that, you can tweak them, fine-tune them, and make them faster, cheaper, better than what's served up by frontier models. So if you don't care about the results or cost, you could use frontier, but otherwise you'll be digging deep into them, the same way some companies invest in writing their own software vs paying for it.

- Finally, there's the icing on the cake: LLMs will be cooked in 10 years. I keep reading from AI research experts that "LLMs are a dead end" - and it turns out it's true. LLMs are basically only good because we invest an unsustainable amount of money in the brute-forcing of a relatively dumb form of iteration: download all knowledge, do some mind-bogglingly expensive computational math on it, tweak the reasults, repeat. There's only so many of that loop you can do, because fundamentally, all you're doing is trying to guess your way to an answer from a picture of the past. It doesn't actually learn, the way a living organism learns, from experience, in real-time, going forward; LLMs only look backward. Like taking a snapshot of all the books a 6 year old has read, then doing tweaks to try to optimize the knowledge from those books, then doing it again. There's only so much knowledge, only so many tweaks. The sensory data of the lived experience of a single year of life of a 6 year old is many times more information than everything ever recorded by man. Reinforcement Learning actually gives you progressive, continuously improved knowledge. But it's slow, which is why we aren't doing it much. We do LLMs instead because we can speed-run them. But the game has an end, and it's the total sum of our recorded knowledge and our tweaks.

So LLMs will plateau, frontier models will make no sense, all lines of code will be hands-off, and Nvidia will return to making hardware for video games. All within about 10 years. With the caveat that there might be a shift in global power and economic stability that interrupts the whole game.... but that's where we stand if things keep on course. Personally, I am happy to keep using AI and reap the benefits of all these moronic companies dumping their money into it, because the open weights continue being useful after those companies are dead. But I'm not gonna be buying Nvidia stock anytime soon, and I'm definitely not gonna use just one frontier model company.