An Improved Particle Swarm Optimization for Feature Selection

被引:211
作者
Liu, Yuanning [1 ,2 ]
Wang, Gang [1 ,2 ]
Chen, Huiling [1 ,2 ]
Dong, Hao [1 ,2 ]
Zhu, Xiaodong [1 ,2 ]
Wang, Sujing [1 ,2 ]
机构
[1] Jilin Univ, Coll Comp Sci & Technol, Changchun 130012, Peoples R China
[2] Jilin Univ, Minist Educ, Key Lab Symbol Computat & Knowledge Engn, Changchun 130012, Peoples R China
基金
中国国家自然科学基金;
关键词
particle swarm optimization; feature selection; data mining; support vector machines; CONVERGENCE ANALYSIS; STABILITY;
D O I
10.1016/S1672-6529(11)60020-6
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Particle Swarm Optimization (PSO) is a popular and bionic algorithm based on the social behavior associated with bird flocking for optimization problems. To maintain the diversity of swarms, a few studies of multi-swarm strategy have been reported. However, the competition among swarms, reservation or destruction of a swarm, has not been considered further. In this paper, we formulate four rules by introducing the mechanism for survival of the fittest, which simulates the competition among the swarms. Based on the mechanism, we design a modified Multi-Swarm PSO (MSPSO) to solve discrete problems, which consists of a number of sub-swarms and a multi-swarm scheduler that can monitor and control each sub-swarm using the rules. To further settle the feature selection problems, we propose an Improved Feature Selection (IFS) method by integrating MSPSO, Support Vector Machines (SVM) with F-score method, The IFS method aims to achieve higher generalization capability through performing kernel parameter optimization and feature selection simultaneously. The performance of the proposed method is compared with that of the standard PSO based, Genetic Algorithm (GA) based and the grid search based methods on 10 benchmark datasets, taken from UCI machine learning and StatLog databases. The numerical results and statistical analysis show that the proposed IFS method performs significantly better than the other three methods in terms of prediction accuracy with smaller subset of features.
引用
收藏
页码:191 / 200
页数:10
相关论文
共 20 条
[1]  
[Anonymous], 1999, The Nature Statist. Learn. Theory
[2]   Multiswarms, exclusion, and anti-convergence in dynamic environments [J].
Blackwell, Tim ;
Branke, Juergen .
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, 2006, 10 (04) :459-472
[3]   Particle swarms and population diversity [J].
Blackwell, TM .
SOFT COMPUTING, 2005, 9 (11) :793-802
[4]  
Boser B. E., 1992, Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory, P144, DOI 10.1145/130385.130401
[5]  
Chen Y W, 2006, EXPERT SYSTEMS APPL, V37, P315
[6]   The particle swarm - Explosion, stability, and convergence in a multidimensional complex space [J].
Clerc, M ;
Kennedy, J .
IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, 2002, 6 (01) :58-73
[7]  
Eberhart R., 1995, MHS 95, P39, DOI [DOI 10.1109/MHS.1995.494215, 10.1109/MHS.1995.494215]
[8]  
Guyon I., 2003, Journal of Machine Learning Research, V3, P1157, DOI 10.1162/153244303322753616
[9]   A distributed PSO-SVM hybrid system with feature selection and parameter optimization [J].
Huang, Cheng-Lung ;
Dun, Jian-Fan .
APPLIED SOFT COMPUTING, 2008, 8 (04) :1381-1391
[10]   Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm [J].
Jiang, M. ;
Luo, Y. P. ;
Yang, S. Y. .
INFORMATION PROCESSING LETTERS, 2007, 102 (01) :8-16