Clustering by sparse orthogonal NMF and interpretable neural network

Employing differentiable reconstruction to interpret the clustering layers of neural networks presents a potent solution to the interpretability challenges encountered when applying neural networks within the multimedia sector. However, most of the existing approaches proposed for interpretability of machine learning make post hoc interpretations of neural networks. While existing approaches have made progress in post hoc interpretation, they still fall short in providing clear explanations for black-box models. There are two main challenges: the first is how we should design an interpretable neural network using sparse orthogonal non-negative matrix factorization (NMF) instead of performing post hoc interpretation. The second is how to implement sparse orthogonal NMF and perform clustering using the designed neural network. To solve these two problems, we address this issue by designing a New Interpretable Neural Network (NINN) that uses sparse orthogonal NMF as a basis for its architecture. We innovate in three aspects: first, NINN is a transparent neural network model with greater interpretability in the clustering layer. Second, NINN automatically performs gradient-based optimization of the parameters in the program and has the advantage of end-to-end learning. Third, since NINN is a differentiable reformulation of sparse orthogonal NMF, NINN inherits many properties of both NMF and neural networks, such as parallel solving, online clustering, dimensionality reduction, and unsupervised learning. Experimental evidence demonstrates that the NINN outperforms a myriad of conventional clustering algorithms, such as k-means and NMF, as well as some of the latest models, in terms of clustering performance.