Transfer Learning of Spatio-Temporal Information using 3D-CNN for Person Re-identification

Video based person re-identification has received significant attention in the recent past due to advancement in deep learning techniques. However, it is still a challenging problem because of inherent dynamic nature of videos. Additionally, lack of sufficient annotated dataset may lead to overfitting issues while training deep networks. In this paper, we propose a spatio-temporal transfer learning approach using 3D-CNN for video based person re-identification. To address the issue of insufficient labelled data and transfer the knowledge, we use a pre-trained 3D-CNN model of Sports-1M dataset and perform fine-tuning on multiple domain datasets such as PRID-2011, iLIDS-VID, MARS and an aerial video dataset simultaneously. Learning features from multiple domain data is of significant value because of large variation which otherwise is not possible to obtain from small individual datasets. In our experiments, we show that the fine-tuned transferred features encode robust representations and enhance the re-identification accuracy. Further, to boost the performance, we apply XQDA metric learning. Experiments conducted on all the four datasets show that the proposed framework outperforms the popular methods by an average improvement of 4% or more. In addition, we analyse the network's robustness against adversarial examples and show that the proposed 3D-CNN network has better resilience compared to 2D-CNN used in most of the existing algorithms.