Comment by jimbohn

I wonder if the failures to pretrain are the result of our understanding of neural networks being more akin to alchemy rather than chemistry