Self-supervised Deep Correlational Multi-view Clustering

In conventional unsupervised multi-view clustering (MVC), learning of representations from heterogeneous multi-view data and its subsequent clustering are often separately optimized. The disparate optimization would lead to suboptimal performance because multi-view representation learning is not goal-directed. In this paper, we unify unsupervised multi-view learning and deep clustering in a novel discriminative Self-supervised Deep Correlational Multi-view Clustering (SDC-MVC) network. A new unified loss function is proposed to incorporate consensus information into discriminative representations, in which, the former is learnt by maximizing the canonical correlation among multi-view representations projected by neural networks, and the later is achieved through using confident clustering assignments as supervision. Further, multi-view representations are harnessed by our proposed Deep Serial Feature-level (DSF) Fusion layer. Experiments on three public datasets demonstrated that our method outperforms six state-of-the-art correlation-based MVC algorithms in terms of three evaluation metrics.