Boundary-guided Black-box Fairness Testing

Although deep learning models have achieved outstanding performance in many applications, there are still concerns about their fairness. A series of fairness testing methods, which evaluate the fairness of deep learning models by generating discriminatory samples, have been proposed. However, these methods either neglect the naturalness of discriminatory samples or roughly select natural discriminatory samples, leading to a decrease in efficiency. In this paper, we introduce a boundary-guided black-box fairness testing method to effectively generate individual discriminatory samples with high efficiency and enhanced naturalness. Our boundary-guided method involves a global exploration phase, which explores multiple paths from the initial samples to the surrogate decision boundary of the target model, imitated from the semantic latent space of a generative adversarial network (GAN). Then, a local perturbation phase explores the nearby space around a given sample for identifying potential discriminatory samples. Extensive experiments on various datasets demonstrate that our approach outperforms state-of-the-art methods in terms of efficiency and effectiveness while maintaining high naturalness.