Comment by tarruda
Thanks for your work, it is really an amazing small LM.
Can you share what kind of hardware is necessary to train it, and how long it took?
Thanks for your work, it is really an amazing small LM.
Can you share what kind of hardware is necessary to train it, and how long it took?
Thank you!
The Gemma3 technical report contains many details on training setup https://arxiv.org/pdf/2503.19786
This was released with the initial batch of Gemma3 so it doesn't contain the 270m details, nonetheless you'll get a good idea of what it takes to build these models.