MGMT 8825M Week 3 Ethics Analysis Example

Fair to Which Group, Measured How? An Ethics Analysis of a Lender's Pledge That Its Credit Model Treats Applicant Groups Fairly, Showing That Three Readings of Fairness Cannot All Hold When Default Rates Differ and Arguing for Calibration With a Named Cost

Student Name

Walden University

MGMT 8825M: Ethics of Artificial Intelligence and Autonomous Systems

Instructor Name

Month Day, Year

Fair to Which Group, Measured How? An Ethics Analysis of a Lender's Pledge That Its Credit Model Treats Applicant Groups Fairly, Showing That Three Readings of Fairness Cannot All Hold When Default Rates Differ and Arguing for Calibration With a Named Cost

The Stated Value

Halsey Lending, a composite online lender making unsecured personal loans of $2,000 to $25,000, states on its website and in its investor materials that "our credit model treats every applicant group fairly." The model places each applicant on a 100-point scale on which a higher number signals a smaller predicted chance of default, and Halsey approves applicants who score 60 or above. For fair lending monitoring, Halsey estimates applicants' race from surname and address, as many lenders do, because it does not collect race on applications.

The pledge sounds clear until one asks what it means. In Halsey's 2025 applicant pool, applicants in the group the analysis will call Group B defaulted, among those approved under an earlier, looser policy, at about 11 percent, compared with 6 percent for Group A. Those base rates differ for reasons that include differences in income stability and in access to credit history, many of them rooted in past discrimination. When base rates differ, the pledge admits at least three readings, and they conflict.

Three Readings of Fairness

The first reading is equal approval rates: the model is fair if it approves the same share of each group. This is sometimes called demographic or statistical parity. The second is equal error rates: the model is fair if creditworthy applicants in each group are denied at the same rate, and applicants who will default are approved at the same rate. The third is calibration: the model is fair if a given score means the same risk of default whatever the applicant's group, so that applicants scored 70 default at the same rate in Group A and Group B.

Why They Cannot All Hold

Two papers published in 2017 showed that these readings collide. Kleinberg et al. (2017) proved that, except in special cases such as perfect prediction or equal base rates, no risk score can be calibrated within groups and also equalize the two kinds of error across groups. Chouldechova (2017) showed a closely related result for a score turned into a yes-or-no decision: if base rates differ, a calibrated or equally predictive tool must produce different false positive and false negative rates across groups.

The public form of this collision was the 2016 dispute over the COMPAS recidivism tool. Angwin et al. (2016), writing for ProPublica, reported that Black defendants in Broward County, Florida, who did not go on to reoffend were nearly twice as likely as white defendants to have been labeled higher risk. Northpointe, the tool's developer, replied that the scores were equally predictive across groups: a given score corresponded to similar reoffense rates for Black and white defendants. Both claims could be true at once, and the 2017 papers explained why.

What Each Reading Forbids

At Halsey's cutoff of 60, an internal backtest of the current model on 2024 applicants found that 14 percent of Group B applicants who repaid their earlier loans would have been denied, compared with 8 percent of Group A applicants who repaid. The model was well calibrated: applicants scored 60 to 65 defaulted at about 9 percent in both groups. So Halsey's model satisfies the third reading and fails the second.

Table 1

What Each Reading of Fairness Rules Out for Halsey's Model

The table assumes base rates continue to differ between groups, as they do in Halsey's data. If base rates were equal, the readings could be satisfied together.

Which Reading Halsey Should Adopt

Halsey should adopt calibration as its primary meaning of fairness, for three reasons. First, calibration is what a score promises the individual: that it means what it says about her. A lender that adjusted scores so creditworthy applicants in each group were denied at equal rates would be telling some applicants that their risk was lower than it is and others that it was higher. Second, calibration keeps loan pricing honest; Halsey prices loans by score, and miscalibrated scores would push borrowers in one group into loans they are more likely to struggle to repay. Third, calibration is what a borrower or regulator can check against outcomes.

Equal approval rates are the weakest candidate. Achieving them with one model would require approving Group B applicants at much lower scores, which would raise their default rate further and expose borrowers to debt they cannot carry. Hellman (2020) argued that balance in error rates carries more moral weight than its critics allow, because it reflects how the burden of mistaken predictions falls on each group. That argument deserves weight here, and it shapes the cost section below.

Who Bears the Cost

Choosing calibration means that creditworthy Group B applicants near the cutoff bear the cost: they are denied loans they would have repaid at a higher rate than comparable Group A applicants. In Halsey's backtest, that is roughly 6 additional denials per 100 creditworthy Group B applicants. Halsey cannot erase that cost without giving up calibration, but it can reduce it. It can seek data that improves prediction for thin-file applicants, such as rent and utility payment histories, which would narrow the error gap without miscalibrating scores. It can offer applicants denied near the cutoff a smaller first loan. And it should revise its pledge to say what it actually measures: that scores mean the same risk for every group, and that the lender publishes and works to reduce the gap in denials of creditworthy applicants.

References

Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153-163. https://doi.org/10.1089/big.2016.0047

Hellman, D. (2020). Measuring algorithmic fairness. Virginia Law Review, 106(4), 811-866.

Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017). Inherent trade-offs in the fair determination of risk scores. In C. H. Papadimitriou (Ed.), 8th Innovations in Theoretical Computer Science Conference (ITCS 2017) (pp. 43:1-43:23). Schloss Dagstuhl. https://doi.org/10.4230/LIPIcs.ITCS.2017.43