It's interesting that DeepMind still publishes this stuff. OpenAI doesn't publish anything of that sort anymore. DeepMind is more research/publication focused, but this is a disadvantage in a competitive landscape where OpenAI and Anthropic can just apply the results of your paper without giving anything back to the research community.
They feel behind because they didn't have the smart guy with a new idea a few years back, and HE decided to work at a place which started as open.
Playing catch up and trying to attract talent from the hot-new-thing OpenAI requires incentives beyond lots of money. I contend actually being open helps.
I'm sure that's one reason Facebook has an open source model, scientists can care about ethics and could be attracted to openness.
> They feel behind because they didn't have the smart guy with a new idea a few years back, and HE decided to work at a place which started as open.
The "Attention Is All You Need" guys all worked at Google. Google is where they are despite having the smart guys with a new idea a few years back.
Of course, IMHO it wouldn't have have helped Google if they'd kept the transformer architecture secret. They'd have fumbled it because they didn't realise what they had.
> but this is a disadvantage in a competitive landscape
Or it's a unique advantage because this stuff doesn't happen without good researches who may want:
1) Their name in scientific papers
2) They might actually care about the openess of AI