Comment by bombcar

> but how is the result "stored"

Like this: https://huggingface.co/docs/safetensors/index

Yes, serving requires infra, too. But you can use infra optimized for serving; nvidia GPUs are not the only game in town.

Theoretically it would be much less expensive to just continue to run the existing models, but ofc none of the current leaders are going to stop training new ones any time soon.