Comment by SilverElfin 8 hours ago
This seems like the same exact talk LeCun has been giving for years, basically pushing JEPA, world models, and attacking contemporary LLMs. Maybe he’s right but it also seems like he’s wrong in terms of timing or impact. LLMs have been going strong for longer than he expected, and providing more value than expected.
>LLMs have been going strong for longer than he expected
Have they? They still seem to be a dead end toward AGI.
Yeah, he was quite vocal in his opinion that they would plateau earlier than they did and that little value would be derived from them because they're just stochastic parrots. Agree with him that they're probably not sufficient for AGI, but, at least in my experience, they're adding a lot of value and they're continuously performing better in a range of tasks that he wasn't expecting them to.
This is also my read; JEPA is a genuinely interesting concept, but he's been hawking it for several years, and nothing has come of it in the domains in which LLMs are successful. Hoping that changes at some point!