Comment by kevindamm
I was actually referring to the data ingestion for training LLMs, I don't know what filtering or weighting might be done with wordfreq.
I was actually referring to the data ingestion for training LLMs, I don't know what filtering or weighting might be done with wordfreq.