Comment by nostrademons

I've heard that the general transformer architecture (not specifically LLMs, which imply a language model, but applied to sensory perceptions and outputting motor commands) has actually been fairly successful when applied to robotics. You want your overall assembly line to have a tiny, repeatable instruction set, but inside each of those individual instructions is oftentimes a complex motion that's very dependent upon chaotic physical realities. Think of being able to orient a part or deal with a stuck bolt, for example. AI Transformers potentially would allow us to replace several steps in the assembly that currently require human workers with robots, and that in turn makes the rest of the assembly much more reproducible (and cheaper).

Training these models takes a bunch more time, because you first need to build special hardware that allows a human to do these motions while having a computer record all the sensor inputs and outputs, and then you need to have the human do them a few thousand times, while LLMs just scrape all the content on the Internet. But it's potentially a lot more impactful, because it allows robots to impact the physical world and not just the printed word.