Comment by kingstnap
If you have a market for it, the hardware industry will aggressively dig in to try to deliver. Maximum performance and maximum efficiency. So I can imagine there is still more to go.
I'm sure the relatively clean directed computational graph + massively parallel + massively hungry workload of AI is a breath of fresh air to the industry.
Hardware gains were for the longest time doing very little for consumers because the bottlenecks were not in the hardware but instead in extremely poorly written software running in very poorly designed layers of abstraction that nothing could be done about.
The hardware overhang embodied: that early AI will be inefficiently embodied as a blob of differentiable floating point numbers in order to do gradient descent on them, and shortly after be translated into a dramatically simpler and faster form. An AGI that requires a full rack of H100s to run, suddenly appearing on single video game consoles. https://www.lesswrong.com/w/computing-overhang
Fun fact: Deep Blue was a dedicated chess compute cluster that ran on 30 RS/6000 processors and 480 VLSI chips. If the Stockfish chess program existed in 1997 it would have beaten it with a single 486 CPU: https://www.lesswrong.com/posts/75dnjiD8kv2khe9eQ/measuring-...