R2CI: Information theoretic-guided feature selection with multiple correlations

Information theoretic-guided feature selection approaches (ITFSs), which exploit the uncertainty of information to measure the correlation of features, aim to select the most informative features. However, most previous approaches suffer from two drawbacks. 1) Complementarity and interaction are not valued, leading to features with potential discriminatory information for learning tasks such as classification not being excavated and affecting the effectiveness of learning. 2) The various correlations that exist between features for the class have not been fully considered, and their differentiation and relationships have not been well reflected. To address the former issue, guided by information theory, the complementarity and interaction between features are studied. For the latter, firstly, some ITFSs are reviewed and analyzed in terms of feature correlation. The analysis reveals that considering feature multi-correlation is absent in the selection process. Motivated by this problem, a feature selection algorithm with class-based relevance, redundancy, complementarity, and interaction (R2CI) is designed for the first time. Moreover, the distinctions and connections among different correlations are also explored. The results of comparisons and hypothesis test against competitive algorithms show that R2CI has significant advantages in most cases.(c) 2022 Elsevier Ltd. All rights reserved.