Comment by rfv6723
See ML research papers from Apple. Their researchers prefered small models over LLM. So they thought researchers' effort would make up the lack of compute. Then the scale law hit them hard.
See ML research papers from Apple. Their researchers prefered small models over LLM. So they thought researchers' effort would make up the lack of compute. Then the scale law hit them hard.