Feature Selection in Multiple Linear Regression Problems with Fewer Samples Than Features

Feature selection is of utmost importance when it comes to problems with large p (number of features) and small n (number of samples). Using too many features for a final model will most probably result in overfitting. There are many possibilities to select a subset of features to represent the data, this paper illustrates correlation filters, forward selection and genetic algorithm for feature selection and PCA and PLS as transformation methods. The methods are tested on three artificial data sets and one data set from an ultrasound study. Results show that no method excels for all problems and every method gives different insights into the data. The greedy style forward selection usually overfits and shows the largest difference between training and testing data, the PLS and PCA perform worse on the artificial data, but better for the ultrasound data.