Comment by agentultra
Comment by agentultra 4 days ago
I'm not sure I buy The Bitter Lesson, tbh.
Deep Blue wasn't a brute-force search. It did rely on heuristics and human knowledge of the domain to prune search paths. We've always known we could brute-force search the entire space but weren't satisfied with waiting until the heat death of the universe for the chance at an answer.
The advances in machine learning do use various heuristics and techniques to solve particular engineering challenges in order to solve more general problems. It hasn't all come down to Moore's Law.. which stopped bearing large fruit some time ago.
However that still comes at a cost. It requires a lot of GPUs, land, energy, and fresh water, and Freon for cooling. We'd prefer to use less of these resources if possible while still getting answers in a reasonable amount of time.
Deep blue had to use the techniques it did due to the limitations of the hardware of the time. Deep blue would almost certainly lose against AlphaZero, even if you tuned it to modern hardware. All you have to do 'manually' is teach it the rules/give it a loss function, then you just let it do its thing.
It's certainly true that "just throw a bunch of GPUs at it" is wasteful, but it does achieve results.