ReliefF Based Pruning Model for Multi-Label Classification

Multi-label classification (MLC) is a machine learning problem in which models are sought that assign a subset of labels to each instance. MLC is receiving increased attention and is relevant to many domains such as text categorization, classification of music and videos, semantic annotation of images and many more. Recently, many studies are looking for efficient and accurate algorithms to cope with multi-label classification challenge. They are usually partitioned into two main categories: algorithm adaptation and problem transformation. In multi-label classification problem the labels will not occur independent of each other; instead, there are statistical dependencies between them. Nowadays, it is commonly accepted that exploiting dependencies between the labels is the key of improving the performance of multi-label classification problem. In this paper, we divide the utilizing methods of label dependency into two groups from the perspective of different ways of problem transformation: label grouping model and feature space extending model. Label grouping model normally groups labels into several label subsets based on certain strategies or criteria to incorporate label dependences. While feature space extending model usually extends the feature space of the binary classifiers to let them discover existing label dependence by themselves. We find out that the common difficulty for both kinds of models is how to accurately measure the dependences between labels. In particular, for feature space extending model, how to choose proper labels to extend the original feature space is the key to improve classification performance. On the basis of this, we propose a ReliefF based pruning model for multi-label classification (ReliefF based Stacking, RFS). RFS measures the dependencies between labels in a feature selection perspective, and then selects the more relative labels into the original feature space. And we use a stacking based algorithm during training and prediction. The key contribution of this algorithm is threefold: (1) It provides a new method to measure the dependences between labels. Unlike existing methods measuring pair-wise label dependences, our method related to the ReliefF algorithm takes into account the effect of all interacting labels. (2) Instead of extending the original feature space with all labels, we choose the closely related labels. Thus, we can reduce noise in the data and avoid adverse effects caused by irrelevant labels. (3) In the feature selection phase, we design a brand new strategy that treats original features and label features as the same features and select together. Our empirical study is divided into two parts: a systematic study on parameters of our algorithm and a comparative study between our proposal and other multi-label classification algorithms. The effects of parameters, feature selection strategies and base classifiers on RFS are discussed in the first part of experiments. In the second part, experiment results based on 6 evaluating measures on 9 multi-label benchmark datasets show that RFS is more effective compared to other advanced multi-label classification algorithms. © 2019, Science Press. All right reserved.