Hierarchical Pruning Discriminative Extreme Learning Machine

The extreme learning machine (ELM) provides efficient unified solutions for generalized single hidden layer feed-forward neural networks. Hierarchical learning based on ELM has now attracted lots of interests. This paper presents a hierarchical pruning discriminative ELM (H-PDELM) for feature learning and classification. The ELM pruning auto-encoder (ELM-PAE) is developed for unsupervised feature learning by promoting the output weights matrix to be row-sparse based on l(2), (1)-norm regularization. ELM-PAE can naturally distinguish and prune useless neurons in hidden layer to determine the structure of AE. Besides, we learn a flexible output weights matrix for super-vised feature classification by relaxing the strict regression label matrix of ELM into a slack one for better generalization performance. H-PDELM performs layer-wise unsupervised feature learning using ELM-PAE, and conducts decision making by the flexible output weights matrix. The network of H-PDELM is compact with good generalization ability. Preliminary experiments on visual dataset show its effectiveness.