An Empirical Study on Wrapper-based Feature Selection for Software Engineering Data

Software metrics give valuable information for understanding and predicting the quality of software modules, and thus it is important to select the right software metrics for building software quality classification models. In this paper we focus on wrapper-based feature (metric) selection techniques, which evaluate the merit of feature subsets based on the performance of classification models. We seek to understand the relationship between the internal learner used inside wrappers and the external learner for building the final classification model. We perform experiments using four consecutive releases of a very large telecommunications system, which include 42 software metrics (and with defect data collected for every program module). Our results demonstrate that (1) the best performance is never found when the internal and external learner match; (2) the best performance is usually found by using NB (Naive Bayes) inside the wrapper unless SVM (Support Vector Machine) is external learner; (3) LR (Logistic Regression) is often the best learner to use for building classification models regardless of which learner was used inside the wrapper.