Comparison of Embedded and Wrapper Approaches for Feature Selection in Support Vector Machines

Feature selection methods are generally divided into three categories: filter, wrapper and embedded approaches. In terms of learning performance, the filter approach is typically inferior compared to the other two because it does not use the target learning algorithm. The embedded and wrapper approaches are both considered high-performing. In this paper we compare the embedded and the wrapper approaches in the context of Support Vector Machines (SVMs). In the wrapper category, we compare well-known algorithms such as Genetic Algorithm (GA), Forward and Backward selection, and a new binary Particle Swarm Optimization (PSO) algorithm. For an embedded approach we devise a new heuristic algorithm based on Multiple Kernel Learning.