Comment by liuliu

Comment by liuliu 3 months ago

1 reply

The router is manually designed (see their cem function). Also, the experts are not separate weights, just different scales of it's singular values.

ghc 3 months ago

Thank you, I was missing that second part.