So if I want to make a model to recommend inkjet printers then a quarter of all recommendations should be for HP printers? After all, a quarter of all sold printers are HP.
As you say, that would be a crappy model. But in my opinion that would also be hardly a fair or unbiased model. That would be a model unfairly biased in favor of HP, who barely sell anything worth recommending
I know a cop, they do public searchings for weapons or drugs. Our law dictates fairness. So every now and then they search an elderly couple. You know how this goes and what the results are.
Any model would be unfair, age-wise but also ethnically.
To be most effective the model would have to be unfair. It would suck to be a law abiding young specific ethnic minority.
But does it help to search elderly couples?
I’m Genuinely curious what would be fair and effective here. You can’t be a Bayesian.
Yes, well there's the irony.
"Unbiased" and "fair" are quite overloaded here, to borrow a programming term.
I think it's one of those times where single words should expressly NOT be used to describe the intent.
The intent of this is to presume that the rate of the thing we are trying to detect is constant across subgroups. The definition of a "good" model therefore is one that approximates this.
I'm curious if their data matches that assumption. Do subgroups submit bad applications at the same rate?
It may be that they don't have the data and therefore can't answer that.