Multimodal Data Fusion using Signal/Image Processing Methods for Multi-Class Machine Learning

As the world progresses further into the digital era, we see a growing utility for combining datasets gathered on different devices and receivers as well as on varying time ranges, for use in machine learning. However, machine learning classification introduces a requirement for standardized data, which in turn hampers the ability to utilize diverse sets of data at a given timestamp. In this paper, we investigate the application of various signal pre-processing techniques (Daubecheis wavelet, discrete cosine and discrete fourier transform among others) for multi-modal, multi-class machine learning. Following the pre-processing, the multi-faceted signals are represented solely by features generated from first order statistics, eigen decomposition, and linear discriminant. Utilizing these generated features, as opposed to the signals themselves, these diverse datasets may now be combined as input to machine learning methods. Furthermore, we apply Fisher's linear discriminant ratio and Random Forest feature importance metrics for feature ranking and feature space reduction followed by a comparison of the approaches. Our work demonstrates that dissimilar datasets with common classes may be combined using the proposed methods with a classification accuracy >= 95%. This paper demonstrates that the feature space may be reduced by approximately 60% with <= 5% loss in classification accuracy, and in some cases, a slight increase in classification accuracy.