Operationalizing Responsible Machine Learning: From Equality Towards Equity

Abstract

With the widespread proliferation of machine learning, there arises both the opportunity for societal benefit as well as the risk of harm. Approaching responsible machine learning is challenging because technical approaches may prioritize convenience and employ mathematical definitions of fairness that abstract away relevant notions of real-world fairness. Conversely, social approaches and theories may produce findings that are too abstract to effectively translate into practice. In this thesis, we will describe research throughout the machine learning pipeline that bridges these approaches and utilizes social implications to guide technical work. First, we will discuss what it means for a dataset to be biased, specifically considering visual image datasets. Then, we will consider the implications of this bias during model training both in the context of finetuning and when incorporating intersectionality. Finally, we will address the complicated problem of fairness measurement. We will do so by covering the complexities of measuring bias amplification, thinking about the actual harms arising from stereotyping, and showcasing how we can measure a multiplicity of representational harms in the task of image captioning. Across these works, we will show how despite the technically convenient approach of considering equality acontextually, a stronger engagement with societal context allows us to operationalize a more equitable formulation. Overall, we will explore how we can expand a narrow focus on equality in responsible machine learning to encompass a broader understanding of equity that more substantively engages with societal context.