Comment by MontyCarloHall

Comment by MontyCarloHall 13 hours ago

>I don't see why these companies can't just stop training at some point.

Because training isn't just about making brand new models with better capabilities, it's also about updating old models to stay current with new information. Even the most sophisticated present-day model with a knowledge cutoff date of 2025 would be severely crippled by 2027 and utterly useless by 2030.

Unless there is some breakthrough that lets existing models cheaply incrementally update their weights to add new information, I don't see any way around this.

fennecbutt 3 hours ago

Ain't never hearda rag

Reply View 1 reply

MontyCarloHall 2 hours ago

There is no evidence that RAG delivers equivalent performance to retraining on new data. Merely having information in the context window is very different from having it baked into the model weights. Relying solely on RAG to keep model results current would also degrade with time, as more and more information would have to be incorporated into the context window the longer it's been since the knowledge cutoff date.

Reply View | 0 replies