Secure deep multimodal biometric authentication using online signature and face features fusion

In Indian banking systems, signatures and faces are the two traits of biometrics that are used for personal identification. The automation of this identification system requires the use of a multimodal verification system. Although many researchers are working in the field of multimodal biometrics, the research involving online signatures and face images is very sparse because it involves the handling of two different types of databases. As the online signature data is in sequence form while the face image data is in image form. Authentication through an online signature requires the generation of a strong feature vector and authentication through a face database requires improving the active area of the face image. The dimensionality of the feature vector generated through the face image is generally large, it needs to be minimized. In this paper, the multimodal biometrics verification method involving online signatures and face images is presented. This is performed by forming a fusion feature vector combining extracted features from online signatures and face images. To extract the features from the face images a modified context aware (MCA) algorithm and Tangential discrimination analysis (TDA) algorithm for dimensionality reduction of feature vector are proposed. The fusion feature vector is used to train a modified mixed sequence deep neural network (MMS-DNN). The proposed system provides an improvement in verification performance in terms of equal error rate (EER). The proposed system achieves the EER of 0.5%, which shows a large improvement from existing work. The proposed system also provides security to the biometric data involved as only a fusion feature vector is used in training and verification algorithms as opposed to raw online signature and face image data.