Adversarial learning with optimism for bias reduction in machine learning

Recently, machine learning has gained enormous momentum and been extensively applied to decision making. However, biases in machine learning models have raised serious concerns. Efforts have been made to reduce biases inherent to training datasets. Unfortunately, data pre-processing aiming to eliminate inherent biases may inadvertently introduce implicit biases that could deflect model training. To address this issue, an in-processing approach, Adversarial Debiasing, has been proposed. Adversarial Debiasing aims to mitigate both dataset biases and implicit biases through its algorithmic process. Nevertheless, model training may not always converge. In this study, we successfully accelerate the convergence of an Adversarial Debiasing model trained on the CelebFaces Attributes dataset using the optimistic Adam optimizer. We also give some discussion on the convergence rate of our proposed framework. We show that given some learning rates, our proposed method adapting the optimistic Adam leads to the convergence of an Adversarial Debiasing model, while one with the Adam optimizer may fail to converge.