Fairness and Robustness: Investigating the Effects of Label Noise on Bias in Machine Learning for Medical Imaging

Abstract

Using deep learning models to predict diseases from chest X-ray images has become an increasingly successful area of research in recent years, with some models approaching the accuracy of human radiologists. However, there are still several challenges that prevent these techniques from being used in real-world settings. One such challenge is model bias: classifiers trained on X-ray images have been shown to be significantly less accurate for some racial minorities and other demographic groups, and attempts to increase the fairness of machine learning models have had limited success in medical contexts. Another challenge is label noise: medical imaging datasets are extremely difficult and costly to annotate accurately due to the domain expertise required, so chest X-ray datasets such as CheXpert resort to using semi-automated labeling methods that are inherently noisy and thus increase the difficulty of training accurate models. While these two obstacles have been studied separately in prior works, their potential relationship with each other in medical settings has not yet been investigated. In this paper, we seek to understand the potential effects that label noise in a dataset can have on the fairness of a resulting model. We first analyze the presence of label noise in the CheXpert dataset with respect to several demographic groups, and we then use this analysis to study how increased levels of label noise affect the fairness of trained X-ray classification models. Furthermore, we analyze the performance of various noise mitigation methods in order to test whether these methods increase classifier biases with respect to demographic groups in the CheXpert dataset. We find that both the accuracies and fairness of standard chest X-ray classifiers are robust to moderate amounts of training label noise. We also show that the selected noise mitigation strategies successfully improve classification accuracy without introducing additional biases, which indicates that these methods can help create more reliable and equitable models for medical imaging analysis.