Parallel-Structure-based Transfer Learning for Deep NIR-to-VIS Face Recognition

This paper considers a heterogeneous face recognition problem, i.e., matching near-infrared (NIR) to visible (VIS) face images. The significant domain gap between the NIR and VIS modalities poses great challenges to accurate face recognition. To overcome the domain gap problem, previous works usually adopted a series structure to transfer high-level features. This paper proposes a Parallel- Structure-based Transfer learning method (PST), which fully utilizes multi-scale feature map information. Specifically, PST consists of two parallel streams of network, i.e., a source stream (S-stream) and a transfer stream (T-stream). S-stream is pre-trained on a large-scale VIS database, and its parameters are fixed. It preserves the discriminative ability learned from the large-scale source dataset. T-stream absorbs multi-scale feature maps from S-stream and transfers the NIR and VIS face embeddings to a unique feature space, which is agnostic to the input image modality. The proposed PST method achieves state-of-the-art performance on CASIA NIR-VIS 2.0 Database, the largest near-infrared face database.