Accuracy Comparison across Face Recognition Algorithms: Where Are We on Measuring Race Bias?

被引：88
作者：
Cavazos J.G. [1 ]
Phillips P.J. [2 ]
Castillo C.D. [3 ]
O'Toole A.J. [1 ]
机构：
[1] School of Behavioral and Brain Sciences, University of Texas at Dallas, Richardson, 75080, TX
[2] Information Access Division, National Institute of Standards and Technology, Gaithersburg, 20899, MD
[3] Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, 21218, MD
来源：
IEEE Transactions on Biometrics, Behavior, and Identity Science | 2021年 / 3卷 / 01期
关键词：
deep convolutional neural networks; Face recognition algorithm; race bias; the other-race effect;
D O I：
10.1109/TBIOM.2020.3027269
中图分类号：
学科分类号：
摘要：
Previous generations of face recognition algorithms differ in accuracy for images of different races (race bias). Here, we present the possible underlying factors (data-driven and scenario modeling) and methodological considerations for assessing race bias in algorithms. We discuss data-driven factors (e.g., image quality, image population statistics, and algorithm architecture), and scenario modeling factors that consider the role of the 'user' of the algorithm (e.g., threshold decisions and demographic constraints). To illustrate how these issues apply, we present data from four face recognition algorithms (a previous-generation algorithm and three deep convolutional neural networks, DCNNs) for East Asian and Caucasian faces. First, dataset difficulty affected both overall recognition accuracy and race bias, such that race bias increased with item difficulty. Second, for all four algorithms, the degree of bias varied depending on the identification decision threshold. To achieve equal false accept rates (FARs), East Asian faces required higher identification thresholds than Caucasian faces, for all algorithms. Third, demographic constraints on the formulation of the distributions used in the test, impacted estimates of algorithm accuracy. We conclude that race bias needs to be measured for individual applications and we provide a checklist for measuring this bias in face recognition algorithms. © 2019 IEEE.
引用
收藏
页码：101 / 111
页数：10
相关论文
共 47 条
  • [21] Grother P., Ngan M., Hanaoka K., Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects, Inst. Stand. Technol, (2019)
  • [22] Drozdowski P., Rathgeb C., Dantcheva A., Damer N., Busch C., IEEE Trans. Technol. Soc, 1, 2, pp. 89-103, (2020)
  • [23] Cavazos J.G., Jeckeln G., Hu Y., O'Toole A.J., Deep Learning- Based Face Analytics
  • [24] Wiskott L., Fellous J.-M., Kruger N., Malsburg C.V.D., Face recognition by elastic bunch graph matching, IEEE Trans. Pattern Anal. Mach. Intell, 19, 7, pp. 775-779, (1997)
  • [25] Turk M.A., Pentland A.P., Face recognition using Eigenfaces, Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit, pp. 586-591, (1991)
  • [26] Moghaddam B., Pentland A., Probabilistic visual learning for object representation, IEEE Trans. Pattern Anal. Mach. Intell, 19, 7, pp. 696-710, (1997)
  • [27] Okada K., Et al., Face Recognition: From Theory to Applications, pp. 186-205, (1998)
  • [28] Beveridge J.R., Givens G.H., Phillips P.J., Draper B.A., Factors that influence algorithm performance in the face recognition grand challenge, Comput. Vis. Image Understand, 113, 6, pp. 750-762, (2009)
  • [29] Krizhevsky A., Sutskever I., Hinton G.E., ImageNet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems, pp. 1097-1105, (2012)
  • [30] Cook C.M., Howard J.J., Sirotin Y.B., Tipton J.L., Vemury A.R., Demographic effects in facial recognition and their dependence on image acquisition: An evaluation of eleven commercial systems, IEEE Trans. Biometrics Behav. Identity Sci, 1, 1, pp. 32-41, (2019)
12345