Opening the Box: Examining Fairness and Impact of "Black Box" Machine Learning Algorithms

Abstract

"Black box" machine learning algorithms are becoming increasingly integrated into daily life, performing complex tasks and decisions traditionally left only to humans in fields like healthcare, finance, and criminal justice. Although these models excel at deciphering patterns in historical data to make predictions, they pose an immense danger to society: their decision-making process is completely unknown to observers. This can lead to models becoming inadvertently biased against protected categories by law (sex, age, race, etc.) which is concealed from decision-makers, leaving affected individuals without recourse.

Through an in-depth study of the COMPAS recidivism algorithm, a black box model used to predict criminal recidivism throughout the United States, this project develops new methods to judge the fairness and impact of black box models in real-world settings. The paper explores interpretable modeling as a technique to examine how protected categories affect the predictability of a black box model. Additionally, the project offers evaluation techniques such as propensity score matching and treatment effect analysis to judge an algorithm’s impact on a decision-making process before and after its implementation.