Comment by supermatt

Comment by supermatt 20 hours ago

3 replies

View on Hacker News

Probably naive questions:

Does that also mean that Gemini-3 (the top ranked model) loses to mistral 3 40% of the time?

Does that make Gemini 1.5x better, or mistral 2/3rd as good as Gemini, or can we not quantify the difference like that?

esafak 20 hours ago

Yes, of course.

Reply View 2 replies
  • uejfiweun 11 hours ago

    Wow. If all the trillions only produces that small of a diff... that's shocking. That's the sort of knowledge that could pop the bubble.

    Reply View 1 reply
    • JustFinishedBSG 3 hours ago

      I wouldn't trust LMArena results much. They measure user preference and users are highly skewed by style, tone etc.

      You can litteraly "improve" your model on LMArena by just adding a bunch of emojis.

      Reply View 0 replies