Spectral filter design based on in-field hyperspectral imaging and machine learning for mango ripeness estimation

Hyperspectral imaging (HSI) is a powerful technology already used for many objectives in agriculture. Applications include disease monitoring, plant phenotyping, yield estimation or fruit composition and ripeness. However, the cost of hyperspectral sensors is typically an order of magnitude higher than simpler RGB cameras, which can be prohibitive. Given that in HSI processing the spectral data often contains redundancies, the full spectra are not always required for a specific application and there is an opportunity to design a lower cost multi spectral sensing system by dimensionality reduction. In past work, HSI dimensionality reduction has been applied in the form of band selection to achieve faster computation times. If, however, the objective is to design a lower cost multi-spectral camera system, band selection is poorly suited because real-world sensor and optical filter responses do not typically replicate the individual bands of a hyperspectral sensor. The objective of this paper is to develop a new methodology for filter selection by simulating several imaging devices with different real-world optical filters, to use a high cost HSI device to design a lower cost multi-spectral solution for a specific application. In this paper, we apply the technique to the specific task of mango fruit maturity estimation (dry matter), which was recently shown to be possible using HSI. Mango HSI acquired under field conditions from an UGV was used as input for the experiments. These involved the simulation of imaging devices, using support vector machines for modelling, and testing several filter combinations by brute force or optimisation with genetic algorithms. The mango prediction performance of the simulations was compared to the best performance obtained with full HSI data, which had an R-2 of 0.74. The best values came from the simulation of a four-sensor device with four distinct filters, achieving R-2 up to 0.69 for mango dry matter estimation. The results showed that genetic algorithms, when compared to brute force approaches, were able to obtain the best solution in an efficient way, and that a good performance for mango ripeness estimation can be achieved from the combination of four spectral filters that would allow to implement them into a low-cost, custom-made multi-spectral sensor. The methods exposed in this paper are more broadly applicable to applications beyond mango maturity estimation.