Reducing Meta-Level Data in Stacking Ensembles for Classification

Stacking as an ensemble framework relies on metalearning for combining the outputs of multiple base classifiers. What exactly the outputs represent and how they are used for meta-learning is an important aspect of stacking. Commonly, the prediction probabilities produced by the base classifiers at the class level are used for training the meta-classifier. The dimensionality of this meta-level data is, therefore, not negligible. Reducing it would result in the need for a less complex metaclassifier, and would thereby simplify the process of metalearning. This paper presents a straightforward approach for aggregating the prediction probabilities of the base classifiers. By calculating the mean and standard deviation at the class level, a substantial reduction in the dimensionality of the meta-level data is achieved. The experimental analysis, conducted on multiple diverse datasets, suggests that the aggregation preserves sufficient information for meta-learning since better or highly competitive performance was attained with respect to the common stacking ensemble framework (utilising the full meta-level data).