Masked face recognition using frontal and profile faces with multiple fusion levels

The main goal of this work is to construct a face recognition system that can overcome the limitation of masked face by exploiting the unmasked parts of frontal and profile faces. This study mainly depends on the upper part of frontal face that contains the eyes and forehead beside of the unmasked parts of profile face such as ear. Many experiments are conducted to measure the efficiency and reliability of face recognition system in case of pandemic where the people have to cover most of their face by mask which makes the process of face recognition harder. Binarized statistical image features approaches is used as texture-based descriptors for feature extraction process beside of convolutional neural networks. Additionally, four fusion approaches are implemented, namely sensor-level, feature-level, score-level and decision-level fusion. Images of frontal and profile face under different conditions from ND-TWINS-2009-2010 and CFPW datasets are used in the experiments. The experimental results show that the proposed method is more reliable and accurate in the case of masked face than using profile or frontal face images as a standalone system. The maximum performance of the proposed method for recognizing masked face as recognition rate is 99.83%. Moreover, the proposed system is better than the state-of-the-art CNN-based and texture-based systems using both frontal and profile faces with and without mask on the faces.