Comment by didip

I am super curious about tensor parallelism and the mechanisms behind how some models can activate only some of their attentions.