Manipulation of sources of bias in AI device development

The increased use of artificial intelligence (AI)-enabled medical devices in clinical practice has driven the need for better understanding of bias in AI applications. While there are many bias mitigation methods, determining which method is appropriate in each situation is not trivial due to a lack of objective bias mitigation assessment approaches. This study presents an approach to manipulate sources of bias in order to facilitate the evaluation of bias mitigation methods during AI development. This approach amplifies sources of bias in the data by varying disease prevalence between subgroups to promote associations between patient subgroups and specific disease classes. The approach can be adjusted to affect the degree to which bias is amplified. In this study, bias amplification was used to vary the bias of COVID-19 classification models using chest X-ray data between patient subgroups defined by sex (female or male) or race (black or white). Analysis of subgroup sensitivity shows that the proposed bias amplification approach results in the sensitivity of one subgroup increasing while the sensitivity of the other subgroup is decreasing, despite a consistent difference in subgroup area under the receiver operating characteristic curve. For example, the amplification of sex-related bias increased the sensitivity for COVID-19 classification of the female subgroup from 0.65 +/- 0.04 to 0.90 +/- 0.02 while decreasing the sensitivity of the male subgroup from 0.66 +/- 0.03 to 0.18 +/- 0.02. This shows that the bias amplification approaches promote bias by causing the model to enhance existing correlations between a patient's subgroup and their COVID-19 status, which results in biased performance in a clinical setting. The approach presented in this study was found to systematically amplify sources of bias for our dataset and AI model, and can thus be used to evaluate the effectiveness of potential bias mitigation methods.