Comment by mgraczyk

Comment by mgraczyk 16 hours ago

Figure 4

I think you are assuming we are talking about swapping API usage from one model to another. That is not what happened. A specific product doing a specific thing uses less energy now.

To clarify: the way models become more efficient is usually by training a new one with a new architecture, quantization, etc.

This is analogous to making a computer more efficient by putting a new CPU in it. It would be completely normal to say that you made the computer more efficient, even though you've actually swapped out the hardware.

sigilis 16 hours ago

Don’t they call all their LLM models Gemini? The paper indicates that they specifically used all the AI models to come up with this figure when they describe the methodology. It looks like they even include classification and search models in this estimate.

I’m inclined to believe that they are issuing a misleading figure here, myself.

Reply View 5 replies

mgraczyk 16 hours ago

They reuse the word here for a product, not a model. It's the name of a specific product surface. There is no single model and the models used change over time and for different requests

Reply View | 3 replies
- immibis 16 hours ago
  
  So it includes both tiny models and large models?
  
  Reply View | 2 replies
  
  mgraczyk 16 hours ago
  
  I would assume so. One important trend is that models have gotten more intelligent for the same size, so for a given product you can use a smaller model.
  Again this is pretty similar to how CPUs have changed
  
  Reply View | 1 reply
  
  immibis 7 hours ago
  
  So it's not a specific product doing a specific thing, but the average across different things?
  
  Reply View | 0 replies
simianwords 11 hours ago

“Gemini App” would be the specific Gemini App in the App Store. Why would it be anything different?

Reply View | 0 replies

esperent 16 hours ago

> Figure 4: Median Gemini Apps text prompt emissions over time—broken down by Scope 2 MB emissions (top) and Scope 1+3 emissions (bottom). Over 12 months, we see that AI model efficiency efforts have led to a 47x reduction in the Scope 2 MB emissions per prompt, and 36x reduction in the Scope 1+3 emissions per user prompt—equivalent to a 44x reduction in total emissions per prompt.

Again, it's talking about "median Gemini" while being very careful not to name any specific numbers for any specific models.

Reply View 8 replies

logicprog 11 hours ago

You're grouping those words wrong. As another commenter pointed out to you, which you ignored, it's median (Gemini Apps) not (median Gemini) Apps. Gemini Apps is a highly specific thing — with a legal definition even iirc — that does not include search, and encompasses a list of models you can actually see and know.

Reply View | 0 replies
simianwords 11 hours ago

What do you think the Gemini app means? It can only mean the consumer facing actually existing Gemini App that exposes 2 models.

Reply View | 0 replies
mgraczyk 16 hours ago

That isn't what that means. Look at the paragraph above that where they explain.
This is the median model used to serve requests for a specific product surface. It's exactly analogous to upgrading the CPU in a computer over time

Reply View | 5 replies
- esperent 15 hours ago
  
  I can't copy text from that pdf on my phone, but the paragraph above says exactly what you'd expect: they're using a "median" value from a "typical user" across all Gemini models. While being very careful not to list the specific models which are used to calculate this median, because it almost certainly includes the tiny model used to show AI summaries on google.com, which would massively skew the median value. As someone above said, it's like adding 8 extra meals of a single lettuce leaf and then claiming you reduced the median caloric intake of your meals.
  
  Reply View | 3 replies
  
  simianwords 11 hours ago
  
  This doesn’t check out. It is not reasonable to interpret “Gemini app” as also including a functionality that is embedded in google searches.
  Gemini app is a specific thing: the Gemini App that actually exists.
  How can Gemini App also include their internal augmented functionality on search which itself is not an application?
  
  Reply View | 2 replies
- tovej 14 hours ago
  
  The median does not move if the upper tail shifts, it only moves if the median moves.
  The fact that they do not report the mean is concerning. The mean captures the entire distribution and could actually be used to calculate the expected value of energy used.
  The median only tells you which point separates the upper half from the lower half, if you don't know anything else about the distribution you cannot use it for any kind of analysis.
  
  Reply View | 0 replies