Discriminative neural network pruning in a multiclass environment: A case study in spoken emotion recognition

Deep learning has become one of the most widely accepted paradigms regarding machine learning. It focuses on the use of hierarchical data models and builds upon the notion that in order to learn about high level data representations, a better understanding of intermediate level representation is needed. Restricted Boltzmann Machines and deep belief networks are two main types of deep learning algorithms commonly used in a wide array of classification and pattern recognition tasks. Examples of these tasks are natural language recognition, neuroimaging studies, forecasting time series, parametric voice synthesis, and speech emotion recognition among others. Recent machine learning studies suggest that deep learning networks can help map feature problems into a more advantageous position, hence improving the classification process. However, selecting a suitable Deep learning architecture in response to a specific problem can be difficult. In this study, we intend to investigate whether discriminative measures, such as Anova, Pearsonas Correlation, Fisher score, Gain ratio, ReliefF, OneR among others, could offer pointers to identify useful neural nods in a Deep learning network. This is due to the fact that normally not all hidden neurons provide insightful information for a classification task. Our approach consists in using some of these discriminative measures to rank the hidden neurons based on their output values, and then prune them in accordance to their position within said ranking. Our results indicate that this approach is also helpful in multiclass classification problems and the pruning process seems to have a positive effect in diminishing the resulting error rate.