Comment by kevindamm

I was actually referring to the data ingestion for training LLMs, I don't know what filtering or weighting might be done with wordfreq.