Feature selection considering weighted relevancy

Feature selection plays an important role in pattern recognition and machine learning. Feature selection based on information theory intends to preserve the feature relevancy between features and class labels while eliminating irrelevant and redundant features. Previous feature selection methods have offered various explanations for feature relevancy, but they ignored the relationships between candidate feature relevancy and selected feature relevancy. To fill this gap, we propose a feature selection method named Feature Selection based on Weighted Relevancy (WRFS). In WRFS, we introduce two weight coefficients that use mutual information and joint mutual information to balance the importance between the two kinds of feature relevancy terms. To evaluate the classification performance of our method, WRFS is compared to three competing feature selection methods and three state-of-the-art methods by two different classifiers on 18 benchmark data sets. The experimental results indicate that WRFS outperforms the other baselines in terms of the classification accuracy, AUC and F1 score.