Selective Feature Bagging of one-class classifiers for novelty detection in high-dimensional data

Novelty detection in high-dimensional data is a challenging task due to the masking effect of irrelevant attributes. A common solution is to discover feature subspace, of which attributes are relevant to novelties. Due to the high uncertainty of novelties in practical applications, ensemble models that combine results from multiple subspaces are proved to be more effective than single models. According to the theory of bias-variance tradeoff, existing ensembles are often developed based on variance reduction. However, it is argued that the combination of poor detectors will deteriorate the performance of ensembles. To this end, this paper proposes an ensemble detector that takes into account variance and bias reduction simultaneously. Our ensemble is referred to as Selective Feature Bagging (SFB) since it is developed on the basis of Feature Bagging (FB). In order to improve the accuracy without deterioration of diversity of base detectors in FB, we resort to the notion of dynamic classifier selection which is proved be effective in classification. During the ensemble generation phase, base detectors are produced and categorized into different groups that are distinguished by the dimensionality of subspace used for training. The purpose of such a design is to maintain the diversity. During the generation phase, the most competent base detector from each of groups is dynamically selected and used to make decision on the test pattern. The purpose of such a design is to enhance the accuracy. We verify the effectiveness of SFB on 15 data sets from KEEL repository. Experimental results have shown that SFB can statistically outperform FB. In addition, several state-of-the-art have also been outperformed by SFB.