Comment by Youden

Comment by Youden 20 hours ago

They mentioned LMArena, you can get the results for that here: https://lmarena.ai/leaderboard/text

Mistral Large 3 is ranked 28, behind all the other major SOTA models. The delta between Mistral and the leader is only 1418 vs. 1491 though. I *think* that means the difference is relatively small.

jampekka 19 hours ago

1491 vs 1418 ELO means the stronger model wins about 60% of the time.

Reply View 4 replies

supermatt 19 hours ago

Probably naive questions:
Does that also mean that Gemini-3 (the top ranked model) loses to mistral 3 40% of the time?
Does that make Gemini 1.5x better, or mistral 2/3rd as good as Gemini, or can we not quantify the difference like that?

Reply View | 3 replies
- esafak 19 hours ago
  
  Yes, of course.
  
  Reply View | 2 replies
  
  uejfiweun 11 hours ago
  
  Wow. If all the trillions only produces that small of a diff... that's shocking. That's the sort of knowledge that could pop the bubble.
  
  Reply View | 1 reply
  
  JustFinishedBSG 2 hours ago
  
  I wouldn't trust LMArena results much. They measure user preference and users are highly skewed by style, tone etc.
  You can litteraly "improve" your model on LMArena by just adding a bunch of emojis.
  
  Reply View | 0 replies