A Unified Multiview Spectral Feature Learning Framework for Hyperspectral Image Classification

Recent progress in spectral classification is dominated by the use of deep learning models. While various learning architectures have been developed, they all extract spectral features from a single-view input. In this article, we investigate a different perspective and develop a unified multiview spectral feature learning framework, which extracts discriminative spectral features from multiple views of inputs. To our knowledge, this is the first reported multiview spectral feature learning method based on deep learning. In this framework, we introduce a multiview spectrum construction method by transforming the input spectral vector into multiple 3-D image patches with different sizes, termed multiview spectrum. This multiview spectrum is fed to a well-designed triple-stream architecture, where global and two local spectral feature learning networks operate in parallel, thus capturing both global and local spectral contextual features simultaneously. Another important contribution of this work is a novel interactive attention mechanism to identify the most informative spectral contextual features. The model is trained in an end-to-end fashion from scratch with a joint loss. Experimental results on four datasets demonstrate excellent performance compared to the current state of the art.