Comment by drdeca
Sure. But that isn’t a reason to conflate the two?
OP wasn’t suggesting looking for an alternative/successor to MLPs, but for an alternative/successor to transformers (while presumably still using MLPs) in the same way that transformers are an alternative/successor to LSTMs.
And that sort of proves my original point which is that we're probably gonna keep riding the same wave as far as it will go!! i.e. keep the tech stack mostly with just what we know works best.