Comment by percentcer 6 hours ago
Isn't this just a form of next token prediction? i.e. you'll keep your options open for a potential rhyme if you select words that have many associated rhyming pairs, and you'll further keep your options open if you focus on broad topics over niche
It doesn't really make explain it because then you'd expect lots of nonsensical lines trying to make a sentence that fits with the theme and rhymes at the same time.
In the same way that human brains are just predicting the next muscle contraction.
To be fair, we don't actually know how the human mind works.
The most sure things we know is that it is a physical system, and that does feel like something to be one of these systems.
Assuming the task remains just generating tokens, what sort of reasoning or planning would say is the threshold, before it's no longer "just a form of next token prediction?"