The power of ensemble learning in sentiment analysis

An ensemble of models is a set of learning models whose individual predictions are combined in such a way that component models compensate for each other's weaknesses. Although there has been a growing interest in ensemble learning techniques in the general machine learning community, the use of ensembles in sentiment classification is still limited. Moreover, much of the research activity on ensemble learning is centred around homogeneous ensembles, although heterogeneous ensembles may prove very useful when combining pre-trained models, which are often readily available. In this paper, several techniques for constructing heterogeneous ensembles are applied and comparatively evaluated in respect of benchmark sentiment classification data sets across four different domains. Median performance improvements of up to 5.53% over the best individual model are observed for several ensemble configurations in respect of all four validation data sets, and clear trends are identified that may prove useful to other researchers in the field. Furthermore, a novel ensemble selection approach is proposed that avoids the storage of individual predictions, as well as the costly retraining of all candidate models for an ensemble, that are often required by other similar approaches.