Several novel evaluation measures for rank-based ensemble pruning with applications to time series prediction

Ensemble pruning is a desirable and popular method to overcome the deficiency of high computational costs of traditional ensemble learning techniques. Among various of ensemble pruning methods, rank-based pruning is conceptually the simplest and possesses performance advantage. While four evaluation measures for rank-based ensemble pruning specifically for time series prediction are proposed by us in this paper. The first one, i.e. Complementarity measure for time series prediction (ComTSP), is properly modified from Complementarity measure (COM) for classification. The design idea of ComTSP is, if the error made by the subensemble for a pruning sample is larger than that by the candidate predictor to a certain extent, it is assumed that the predictor is complementary to the subensemble. And the predictor which minimizes the error rate of subensemble on the pruning set will be selected at each selection step. The second one, i.e. Concurrency thinning for time series prediction (ConTSP), is correctly transformed from Concurrency measure (CON) for classification. With ConTSP, a predictor is rewarded for obtaining a good performance, and rewarded more for obtaining a good performance when the subensemble performs badly. A predictor is penalized when both the subensemble and itself perform poorly. The measure ReTSP-Value is specifically designed for Reduce Error (RE) pruning for time series prediction. However, ReTSP-Value and ComTSP have the same flaw that, they could not guarantee the remaining predictor which supplements the subensemble the most will be selected. The cause of this flaw is that the predictive error in time series prediction is directional. It is not reasonable for these measures to take reducing error as the only goal while ignore the error direction. While our finally proposed measure ReTSP-Trend overcomes this defect, taking into consideration the trend of time series and the direction of forecasting error. It could indeed guarantee that the remaining predictor which supplements the subensemble the most will be selected. The comparison experiments on four benchmark financial time series datasets show that the measure ReTSP-Trend outperforms the other measures, which can remarkably improve the predictive ability and promote the generalization capability of the pruned ensembles for time series forecasting. (C) 2014 Elsevier Ltd. All rights reserved.