Face recognition in unconstrained environments with multimodal 2D/3D BiLSTM-CNN parallel model

Face recognition (FR) is a part of many modern real-life applications. However, FR in unconstrained environments remains a challenging research area. Facial images in these environments have some issues that may affect the results, such as facial expressions, occlusions, low resolution, noise, lighting, and pose changes. In this study, a novel multimodal 2D/3D BiLSTM-CNN parallel architecture for FR was developed. To enhance recognition performance against various variations that the face may experience in real-world situations, the proposed parallel CNN architecture is made up of three subnetworks for feature extraction that can fully exploit the 2D/3D features extracted from the face, including the Local Binary Pattern image (LBP), 3D mesh-LBP, and the face image itself. To select the most important data from the feature vectors generated by parallel CNNs, a BiLSTM model is proposed, followed by two fully connected layers for the FR. Using four face datasets, the results demonstrate the effectiveness of the multimodal 2D/3D BiLSTM-CNN parallel model in recognizing faces in uncontrolled environments while undergoing various facial transformations.