Hierarchical Feature Extraction using Partial Least Squares Regression and Clustering for Image Classification

In this paper, we propose an image classification method using Partial Least Squares regression (PLS) and clustering. PLSNet is a simple network using PLS for image classification and obtained high accuracies on the MNIST and CIFAR-10 datasets. It crops a lot of local regions from training images as explanatory variables, and their class labels are used as objective variables. Then PLS is applied to those variables, and some filters are obtained. However, there are a variety of local regions in each class, and intra-class variance is large. Therefore, we consider that local regions in each class should be divided and handled separately. In this paper, we apply clustering to local regions in each class and make a set from a cluster of all classes. There are some sets whose number is the number of clusters. Then we apply PLSNet to each set. By doing the processes, we obtain some feature vectors per image. Finally, we train SVM for each feature vector and classify the images by voting the result of SVM. Our PLSNet obtained 82.42% accuracy on the CIFAR-10 dataset. This accuracy is 1.69% higher than PLSNet without clustering and an attractive result of the methods without CNN.