Comment by fennecbutt

I've always thought that LLMs are still just statistical machines and that their output is very similar to the superpermutation problem, though not exactly.

I just like to think of it as a high dimensional view of the relationships between various words and that the output is the result of continuing the path taken through that high dimensional space, where each point's probability of selection changes with each token in the sequence.

Unfortunately there's no thought or logic really going on there in the simplest cases as far as I can understand it. Though for more complex models/different architectures anything that fundamentally changes the way that the model explores a path through space like that could be implementing thought/logic I suppose.

It's why they need to outsource mathematics for the most part.