Exploiting diversity of margin-based classifiers

An experimental comparison among Support Vector Machines, AdaBoost and a recently proposed model for maximizing the margin with Feed-forward Neural Networks has been made on a real-world classification problem, namely Text Categorization. The results obtained when comparing their agreement on the predictions show that similar performance does not imply similar predictions, suggesting that different models can be combined to obtain better performance. As a consequence of the study, we derived a very simple confidence measure of the prediction of the tested margin-based classifiers. This measure is based on the margin curve. The combination of margin-based classifiers with this confidence measure lead to a marked improvement on the performance of the system, when combined with several well-known combination schemes.