Applying multimodal data fusion based on manifold learning with nuclear magnetic resonance (NMR) and near infrared spectroscopy (NIRS) to maize haploid identification

The fusion of multi-source data obtained by multimodal sensors has recently attracted attention. The manifold learning method has been applied to data fusion problems because of its ability to extract the underlying structure of data, and a new data fusion method has been developed called alternating diffusion maps. In practical application, some shortcomings of this method were discovered. Firstly, it is based on diffusion process, which is more suitable for clustering tasks than classification tasks because of its clustering characteristics; secondly, in possible data overlapping areas the diffusion process between different classes of samples makes the performance of the algorithm decline rapidly; finally, the lack of explicit mapping makes it difficult to extend to new samples. In response to these problems, this paper proposes a new kernel width selection method with discrimination effect, which can be applied to classification tasks and possible overlapping area. In addition, Nystrom method is extended to solve out-of-sample problem. The improved framework to the identification of maize haploids and proposed for the first time to carry out fusion analysis on the data obtained by NMR and NIRS measurement equipment. The experimental results show that our method has a significant improvement in the classification task of unclear class boundaries of maize kernel, up to about 9%, which confirmed the effectiveness of the fusion of NMR and NIRS data for classification and the superiority of our proposed framework. (c) 2021 IAgrE. Published by Elsevier Ltd. All rights reserved.