Feature extraction algorithms for pattern classification

Feature extraction is often an important preprocessing step in classifier design, in order to overcome the problems associated with having a large input space. A common way of doing this is to use Principle Component Analysis to find the most important features. However, it has been recognised that this may not produce an optimal set of features in some problems since the method relies on the second order statistics (covariance structure) of the data. In this paper a method called projection pursuit is presented, which is capable of extracting features based on higher order statistics of the distribution. The original projection pursuit algorithm performs a full d-dimensional search (where d is the number of features sought) that is impractical when d is large. Instead, a simple stepwise approach is suggested in which the computations only grow linearly with d. Some simulations on six publicly available data sets are shown which shows how it may be superior to PCA on some tasks in pattern classification.