Single-objective and multi-objective mixed-variable grey wolf optimizer for joint feature selection and classifier parameter tuning

Feature selection plays an essential role in data preprocessing, which can extract valuable information from extensive data, thereby enhancing the performance of machine learning classification. However, existing feature selection methods primarily focus on selecting feature subsets without considering the impact of classifier parameters on the optimal subset. Different from these works, this paper considers jointly optimizing the feature subset and classifier parameters to minimize the number of features and achieve a low classification error rate. Since feature selection is an optimization problem with binary solution space while classifier parameters involve both continuous and discrete variables, our formulated problem becomes a complex multi- objective mixed-variable problem. To address this challenge, we consider a single-objective optimization method and a multi-objective optimization approach. Specifically, in the single-objective optimization method, we adopt the linear weight method to convert our multiple objectives into a fitness function and then propose a mixed-variable grey wolf optimizer (MGWO) to optimize the function. The proposed MGWO introduces Chaos-Faure initialization, Log convergence factor adjustment, and optimal solution adaptive update operators to enhance its adaptability and balance the global and local search of the algorithm. Subsequently, an improved multi-objective grey wolf optimizer (IMOGWO) is introduced to directly address the problem. The proposed IMOGWO introduces improved initialization, local search, and binary variable mutation operators to balance its exploration and exploitation abilities, making it more suitable for our mixed-variable problem. Extensive simulation results show that our MGWO and IMOGWO outperform recent and classic baselines. Moreover, we also find that jointly optimizing classifier parameters can significantly improve classification accuracy.