Comment by oersted

Relying on the model’s own “memory” to answer factual queries is almost always a mistake. Fine-tuning is almost always a more complex, more expensive and less effective method to give a model access to a knowledge base.

However using the model as a multi-hop search robot, leveraging it’s general background knowledge to guide the research flow and interpret findings, works exceedingly well.

Training with RL to optimize research tool use and reasoning is the way forward, at least until we have proper Stateful LLMs that can effectively manage an internal memory (as in Neural Turing Machines, and such).