Deep, Flexible Data Embedding with Graph-Based Feature Propagation for Semi-supervised Classification

Graph-based data representation has recently received much attention in the fields of machine learning and cognitive computation. Deep architectures and the semi-supervised learning paradigm are very closely related to natural cognitive systems. In this paper, and in the context of semi-supervised learning, we will be addressing deep graph-based data representation using a cascade of flexible embedding based on feature propagation over graphs. Inspired by connectionist models, we developed a deep architecture that performs data representation. In each layer, a graph is created over the current representation of the data. This graph is used to aggregate the current features of the input data and provide a layer-specific linear and non-linear representation. The semi-supervised scheme presented simultaneously satisfies several desired properties. These include graph-based regularization of the data structure - a geometrically motivated criterion, flexible non-linear projection (i.e., linear and non-linear projections are jointly estimated), graph-based feature propagation (providing a low-pass filter of the features in each layer), and deep architecture. Our work's main innovative aspect stems from the fact that each layer employs feature propagation (aggregation) before solving the layer-by-layer projection transformations. The proposed model can be learned layer by layer. In each layer, the non-linear data representation and linear regression are jointly estimated with a closed form solution. The proposed method was evaluated using semi-supervised classification tasks with six image datasets. These experiments demonstrated the effectiveness of the proposed approach, which can compete with a variety of competing semi-supervised methods. Compared to a flexible scheme for data representation, the introduced method improved the performance by 8.5% on average. Compared to a recent deep scheme for data representation, the introduced feature propagation improved the performance by 1.3% on average. The use of feature propagation in each layer can improve the flexible model's performance.