Cascades of Evolutionary Support Vector Machines

Support vector machines (SVMs) have been widely applied to binary classification, but their real-life applications are limited due to high time and memory complexities of training coupled with high sensitivity to the hyperparameters of the classifier. To train SVMs from large datasets, numerous techniques were proposed which select a subset out of all the data presented for training. However, it is challenging to determine the appropriate size of such a subset which may lead to sub-optimal performance. In this paper, we propose a new approach to building a cascade of SVMs, each of which is optimized using a memetic algorithm that selects a small subset of the training data and tunes the hyperparameters. The optimization at each level of the cascade is aimed at creating competence regions that altogether cover complementary parts of the input space. Our experiments performed over 12 synthesized datasets and 24 benchmarks revealed that our method outperforms other classifiers, including SVMs trained with the whole set as well as with a reduced set selected using other techniques. Furthermore, our cascade identifies the high-confidence regions in the input space, and the results confirm that they are characterized with increased classification accuracy obtained for the test data.