Feature selection using a set based discrete particle swarm optimization and a novel feature subset evaluation criterion

In many machine learning or patter recognition tasks such as classification, datasets with a large number of features are involved. Feature selection aims at eliminating the redundant and irrelevant features which would bring computational burden and degrade the performance of learning algorithms. Particle swarm optimization (PSO) has been widely used in feature selection due to its global search ability and computational efficiency. However, PSO was originally designed for continuous optimization problems and the discretization of PSO in feature selection is still a problem which needs further investigation. This paper develops a novel feature selection algorithm based on a set based discrete PSO (SPSO). SPSO employs a set based encoding scheme which makes it able to characterize the discrete search space in feature selection problem. It also redefines the velocity term and the corresponding arithmetic operators which enables it to search for the optimal feature subset in the discrete space. In addition, a novel feature subset evaluation criterion based on contribution rate is proposed as the fitness function in SPSO. The proposed criterion does not need any pre-determined parameter to keep the balance between relevance and redundancy of the feature subset. The proposed method is compared with six filter approaches and four wrapper approaches on ten well known UCI dataset and the experimental results demonstrate the proposed method is promising.