Comment by janalsncm

It’s not that crazy to add a couple of domain-specific prediction heads to a BERT-family pretrained model and then do a quick fine tuning. By volume that’s less common but I would guess most people are just using things off the shelf and might not even consider themselves AI engineers. I have no frame of reference though.