Two-stage Feature Selection Method for Text Classification

Dimension reduction is the process of reducing the number of random features under consideration, and can be divided into the feature selection and the feature extraction. A two-stage feature selection method based on the Regularized Least Squares-Multi Angle Regression and Shrinkage (RLS-MARS) model is proposed in this paper: In the first stage, a new weighting method, the Term Frequency Inverse Document and Category Frequency Collection normalization (TF-IDCFC) is applied to measure the features, and select the important features by using the category information as a factor. In the second stage, the RLS-MARS model is used to select the relevant information, while the Regularized Least Squares (RLS) with the Least Angle Regression and Shrinkage (LARS) can be viewed as an efficient approach. The experiments on Fudan University Chinese Text Classification Corpus and 20 Newsgroups, both of those datasets demonstrate the effectiveness of the new feature selection method for text classification in several classical algorithms: KNN and SVMLight.