Comment by no_wizard 2 days ago
see the edit. boils down to the ability to generalize, LLMs can't generalize. I'm not the only one who holds this view either. Francois Chollet, a former intelligence researcher at Google also shares this view.
> A lot of people would say that LLMs generalize pretty well
What do you mean here? The trained model, the inference engine, is the one that makes an LLM for "a lot of people".
> they certainly can understand natural language sequences that are not present in their training data
Keeping the trained model as LLM in mind, I think learning a language includes generalization and is typically achieved by a human, so I'll try to formulate:
Can a trained LLM model learn languages that hasn't been in its training set just by chatting/prompting? Given that any Korean texts were excluded from the training set, could Korean be learned? Does that even work with languages descending from the same language family (Spanish in the training set but Italian should be learned)?
Chollet's argument was that it's not "true" generalization, which would be at the level of human cognition. He sets the bar so high that it becomes a No True Scotsman fallacy. The deep neural networks are practically generalizing well enough to solve many tasks better than humans.
No. His argument is definitely closer to LLMs can't generalize. I think you would benefit from re-reading the paper. The point is that a puzzle consisting of simple reasoning about simple priors should be a fairly low bar for "intelligence" (necessary but not sufficient). LLMs performs abysmally because they have a very specific purpose trained goal that is different from solving the ARC puzzles. Humans solve these easily. And committees of humans do so perfectly. If LLMs were intelligent they would be able to construct algorithms consisting of simple applications of the priors.
Training to a specific task and getting better is completely orthogonal to generalized search and application of priors. Humans do a mix of both search of the operations and pattern matching of recognizing the difference between start and stop state. That is because their "algorithm" is so general purpose. And we have very little idea how the two are combined efficiently.
At least this is how I interpreted the paper.
He is setting a bar, saying that that is the "true" generalization.
Deep neural networks are definitely performing generalization at a certain level that beats humans at translation or Go, just not at his ARC bar. He may not think it's good enough, but it's still generalization whether he likes it or not.
I'm not convinced either of your examples is generalization. Consider Go. I don't consider a procedural chess engine to be "generalized" in any sense yet a decent one can easily beat any human. Why then should Go be different?
> Francois Chollet, a former intelligence researcher at Google also shares this view.
Great, now there are two of you.
Are you able to formulate "generalization" in a concrete and objective way that could be achieved unambiguously, and is currently achieved by a typical human? A lot of people would say that LLMs generalize pretty well - they certainly can understand natural language sequences that are not present in their training data.