Dimensionality reduction of near infrared spectral data using global and local implementations of principal component analysis for neural network calibrations

The artificial neural network (ANN) learning algorithm is a strong alternative to traditional linear calibration methods used in near infrared (NIR) spectroscopy. The generalisation capacity of ANN can be effectively employed only when the ratio of available training samples to a number of neuron interconnection weights and biases (unknown regression parameters) is large. Traditionally, this ratio is increased by compressing X data to fewer variables using principal component analysis (PCA). An alternative method, local PCA compression, has been described in the literature. This approach overcomes the global linearity of PCA by performing dimensionality reduction in two steps: division of the data space into clusters and local compression of each cluster using PCA. This study compared global and local implementations of PCA compression in NIR calibration problems solved with ANN regression. Three data sets were used for development of control,(based on PCA) and experimental (based on local PCA) ANN calibrations. The results demonstrated that local PCA could outperform traditional global PCA compression. However, the best dimensionality reduction method was case-dependent. Performance of local PCA-based calibrations degraded rapidly as compression rate increased, while global PCA allowed higher compression at minimal cost of prediction accuracy.