Comment by tanewishly
Comment by tanewishly 6 months ago
My point was that there is no test (or classifier) that can always guarantee that one definition of fairness by itself, irrespective of the base rate. If the classifier acts the same independent of base rate, there are always base rates (ie occurrence rates in the rates population) for which the classifier will fail the given definition.
That illustrates that the given definition cannot hold universally, irrespective of what classifier you dream up. Unless your classifier is not independent from the base rate - that is, a classifier that gets more lenient if there's more fraud in the group. That seems undesirable when considering fairness as a goal.
> Unless your classifier is not independent from the base rate - that is, a classifier that gets more lenient if there's more fraud in the group.
If I understand correctly, that was the purpose of their "reweighting". To remove "bias" from their new model, they looked at the aggregate false positive rates by nationality, then applied explicit penalty to people with Western nationality to bring that FPR equal to non-Western.
I'd certainly agree that's undesirable, and my point was also to highlight that (since the article was reasonably clear on the type of fairness they achieved, but much less clear on the fairness they sacrificed).