Comment by wasabi991011
Comment by wasabi991011 a day ago
That's not what they are saying. SOTA models include much more than just language, and the scale of training data is related to its "intelligence". Restricting the corpus in time => less training data => less intelligence => less ability to "discover" new concepts not in its training data
Could always train them on data up to 2015ish and then see if you can rediscover LLMs. There's plenty of data.