Feature selection for classification models via bilevel optimization

Selecting model features that would ensure adequate out-of-sample classification is difficult in real life applications of classification often because there is a large number of candidate features. We propose a bilevel programming approach to feature selection problem for classification and develop a novel genetic algorithm as a solution approach. We implement the proposed framework in three different case studies where we classify influenza strains based on antigenic variety, distinguish between good and bad quality colposcopy images, and identify splice junction sites in genetic sequences. As a benchmark for the proposed genetic algorithm, we use a derivative-free optimization method to solve the bilevel feature selection problems in these case studies. The computational experiments show that the proposed bilevel framework improves the overall classification performance while selecting the most important features for the model. (C) 2018 Elsevier Ltd. All rights reserved.