A Bimodal Biometric Verification System Based on Deep Learning

In order to improve the limitation of single-mode biometric identification technology, a bimodal biometric verification system based on deep learning is proposed in this paper. A modified CNN architecture is used to generate better facial feature for bimodal fusion. The obtained facial feature and acoustic feature extracted by the acoustic feature extraction model are fused together to form the fusion feature on feature layer level. The fusion feature obtained by this method are used to train a neural network of identifying the target person who have these corresponding features. Experimental results demonstrate the superiority and high performance of our bimodal biometric in comparison with single-mode biometrics for identity authentication, which are tested on a bimodal database consists of data coherent from TED-LIUM and CASIA-WebFace. Compared with using facial feature or acoustic feature alone, the classification accuracy of fusion feature obtained by our method is increased obviously.