Comment by sillysaurusx

Sure, not every model is an autoregressive transformer. And even a GPT could give some useful predictions if you stuff the context window with things it's been fine tuned to predict. We did that to get GPT to play chess a few years ago.

Specifically, I could imagine throwing current weather data at the model and asking it what it thinks the next most likely weather change is going to be. If it's accurate at all, then that could be done on any given day without further training.

The problems happen when you start throwing data at it that it wasn't trained on, so it'll be a cat and mouse game. But it's one I think the cat can win, if it's persistent enough.