Comment by make3

Architecture difference wrt vanilla transformers and between modern transformers are a tiny part of what makes a model nowadays