An Encoder Generative Adversarial Network for Multi-modality Image Recognition

This paper is concerned with the multi-modality image recognition which is a crucial technique used in industrial applications. The paper proposes a novel algorithm based on the deep generative adversarial network to learn a common feature space between different modalities. These abstract features are robust to the modality discrepancy and can be used to train a cross- modality classifier which will achieve excellent performance on all modalities. The comparative experiments on standard multi- modality image recognition benchmark are employed to validate the effectiveness of our proposed algorithm. The results demonstrate that the proposed network is efficient to deal with the multi-modality recognition challenge, especially improve the performance on the modalities with limited training samples.