Comment by simianwords

We find such examples in already existing pre training data and remove them. Do you not think it will work?