Comment by nerdponx

Because if you work on a team you need to use a language that the whole team can work with. If I'm the one R guy at a Python shop, it's not going to work out well. It depends a lot on org structure of course. But I think it's telling that the jobs you highlight are mostly academic jobs where the practitioner would be expected to be a highly competent individual working largely alone, or in a very small group, carrying out research on behalf of some stakeholders, and not likely to have to put anything "into production" any time soon.

For example, I used R (data.table) when I was a solo data scientist working on a consulting project where I needed to work with a dataset on the order of a few billion rows. I had nobody around to constrain my choice of tools, so I went with whatever felt convenient, familiar, and ergonomic for getting the job done.

Today, I am on a team of 5 other people, none of which know a lick of R, and my code needs to run in production pipelines that need to at least in theory be debuggable, auditable, fixable, etc. by people other than me. Therefore I use Python, because we are a Python team and that's the language that we use, end of story. (Python also happens to be a good choice on our team for other reasons, but that's not the point here).

Maybe the best industry where you are likely to find people doing "production" work in R is some form of insurance. But even back in 2017-2020, things were shifting towards Python at the one P&C company I worked for.