supermatt 19 hours ago

Probably naive questions:

Does that also mean that Gemini-3 (the top ranked model) loses to mistral 3 40% of the time?

Does that make Gemini 1.5x better, or mistral 2/3rd as good as Gemini, or can we not quantify the difference like that?

Reply View 3 replies
  • esafak 19 hours ago

    Yes, of course.

    Reply View 2 replies
    • uejfiweun 11 hours ago

      Wow. If all the trillions only produces that small of a diff... that's shocking. That's the sort of knowledge that could pop the bubble.

      Reply View 1 reply
      • JustFinishedBSG 2 hours ago

        I wouldn't trust LMArena results much. They measure user preference and users are highly skewed by style, tone etc.

        You can litteraly "improve" your model on LMArena by just adding a bunch of emojis.

        Reply View 0 replies