Data-driven decision-making in classification algorithm selection

The selection of the appropriate classification algorithm for a given data-set is an important and complex issue, full of research challenges. In this paper, we present a developed meta-analysis-based framework to improve decision-making in the selection of classification algorithms based on data-set characteristics. We study the effectiveness of our proposed framework with 32 data-sets. Three classification algorithms - neural networks, decision trees, and k-nearest neighbours - were trained and applied to data-sets with different characteristics, aiming to review the performance of algorithms in the presence of noise in the data, the interaction between features, as well as a small or a large ratio between the number of instances and the number of features. Our results show that feature noise is the most important predictor of the decision regarding the choice of the classification algorithm, and data-driven classification is found to be useful in this scenario.