Comment by observationist 3 days ago
Moore's law never ceases to amaze (the vulgar version where we're talking compute/dollar, not the transistor count doubling rate.) It won't be too long before phones are running AI models with performance equal to or better than current frontier models running on $100 million dollar clusters. It's hard to even imagine the things that will be running on billion dollar clusters in 10 years.
As state of the art machines continue to chase the latest node, capacity for older nodes has become much less expensive, more openly documented, and actually accessible to individuals. Open source FPGA and ASIC synthesis tools have also immensely improved in quality and capability. The Raspberry Pi Pico RP2350 contains an open source Risc-V core designed by an individual. And 4G cell phones like the https://lilygo.cc/products/t-deck-pro are available on the market built around the very similar ESP32. The latest greatest will always be behind a paywall, but the rising tide floats all boats, and hobbyist projects are growing more sophisticated. Even a $1 ESP32 has dual 240mhz 32bit cores, 8Mb ram, and fast network interfaces which blow away the 8bit micros I grew up with. The state of the open-source art may be a bit behind the state of the proprietary arts, but is advancing as well.
It's really fun to have useful hardware that's easy to program at the bare metal.
Even when technically accessible to individuals it still costs at least 10k$ to get a batch of chips made on a multi project wafer.
chipfoundry.io charges $14,950 for packaged 100 chips. As far as small batch manufacturing goes, that's reasonably affordable. $149 ea. Occasionally I see better deals crop up as part of group buys or for bare dies. Presumably, one would prototype their design on an inexpensive FPGA board first, to verify functionality. So as to be reasonably sure the first batch of chips worked. Folks like Sam Zeloof are working to build new tools for one-off and small batch designs as well, which may further reduce small quantity prices.
It might not be in our lifetimes... the frontier models are using terabytes of RAM. In 10 years iPhones went from ~2GB to ~8GB.
2012 Macbook pros had up to 16gb, 2026 maxes out at 64gb. So 4x increase in 16 years. 1996 Mac desktop had 16MB of ram, so from 1996-2012 there was a 1000x increase.
We won't see gains like we did from the 80s-2000s again.
I don't think you're going to see phones with 512gb VRAM+RAM in your lifetime.
When I was a kid I recall my cousin upgrading his computer to 1 or 2 MB so that we could get some extra features when playing Wing Commander 1. That was 1990.
35 years later, burner phones regularly come with 4 GB of RAM these days. 3 order of magnitude difference, not taking into account miniaturization and speed improvements.
In another 35 years who knows what will happen. Yeah things can't improve at the same pace forever but I would be surprised if anyone back in 1990 could predict the level of technology you can get at every corner store today.
Maybe it's not that everyone gets an RTX 5090 in our pocket, but maybe it's that LLMs now can run on rpi. Realistically it's probably something in the middle.
When I was a kid in Elementary we used DOS computers with maybe 4MB of RAM or few MB and the Play Station wasn't many times powerful. A few years (two or three) later we got Windows 95/98 with 128 times more RAM. A few years later, computers could emulate more or less the PSX and the N64, all within six years.
Phones have as much memory as Android requires, not much more. A low end thinkpad 10 years ago had 8gb memory, and today is same capacity bit more modern and faster. By the same rate we would have a very very fast 8gb memory thinkpad by 2040. Same thing with GPUs. Mid range GPU 10 years ago had 12gb VRAM, mid range AMD GPU last generation (6600xt) had 8gb and 7600xt 16gb, Nvidia 5060 comes at 8gb/16gb.
Phones with 4gb ram is not feasible today because they wouldnt be able to run Android and phone home comfortably, even being a thin client requires running Android and react application on electron. 4gb is not good.
In 2040 phones will came out with the bare minimum to run Android, all the stupid Chinese apps Android distro pushes into consumers, and a react application on electron.
Maybe he fell for the 12 + 12 crap they advertise where half the memory is swap
> ... It won't be too long before phones are running AI models with performance equal to or better than current frontier models running on $100 million dollar clusters.
Maybe, perhaps phones will have the compute power... But not enough memory. If things continue the way they are, that is. Great for AI firms, they'll have their moat.
Well, we live in extraordinary circumstances today. A $40 kit (Patriot Viper Venom DDR5-6000 C36 16GB) is now $199. And that is the cheapest DDR5 I saw. With this year's news of even more allocation towards data centres, Micron exiting the consumer market, and the current inertia of things, I think it will take quite some time for us to see prices back to as they were.
I do hope you're right, but I'm quite skeptical. As mobile devices get more and more locked down, All that memory capacity gets less and less usable. I'm sure it will be accessible to Apple and Google models, but models that obey the user? Not likely