Kernel Optimization-Based Multiclass Support Vector Machine Feature Selection

Support vector machine (SVM) is a popular classification paradigm in machine learning and has achieved great success in real applications. There has been considerable interest in feature selection for SVM, but the previous works are usually for binary classification. This paper considers feature selection in a multiclass classification scenario where the goal is to determine a subset of available features which is most discriminative and informative for all the classes simultaneously. Based on the data distributions of classes in the feature space, this paper first presents a model selection criterion named multiclass kernel polarization (MKP) to evaluate the goodness of a kernel in multiclass classification scenario, and then optimizes the scale factors assigned to each feature in a kernel by maximizing this criterion to identify the more relevant features. Since MKP is differentiable with respect to the scale factors, the gradient-based search techniques can be used to solve this maximizing problem efficiently. Experimental study on some UCI machine learning benchmark examples demonstrates the effectiveness of the proposed approach.