Multimodal Data Fusion-Moving From Domain-Specific Algorithms to Transdomain Understanding for Accelerated Solution Development

The exploding availability of new data streams now enables insights to be garnered through the integration (fusion) of multiple data sources (modalities); however, currently, it remains difficult to predict a priori which multimodal data fusion (MMDF) methods and architectures will be best suited for a novel application, leading to trial-and-error approaches that are inefficient in both time and cost. Although MMDF strategies are being applied ad hoc in many different fields (e.g., healthcare, autonomous navigation, robotics, manufacturing, transportation, etc.), results are reported in different journals using different terminology for different target audiences. These applications have diverse needs (assessment, prediction, detection, etc.), resources (number and quality of available data streams), and constraints (social, economic, and physics); however, they share many properties when these lists are evaluated at the metalevel. The authors propose that a shared MMDF framework is urgently needed to allow aggregation of insights across applications, with the end goal of developing an MMDF recommendation tool that overcomes the need for trial-and-error approaches to expedite implementation of new solutions across a wide range of disciplines. To provide context for this Special Issue and provide an overview of key related issues, this editorial presents an abbreviated summary of the current state of MMDF usage from multiple perspectives, based on input from a March 2018 workshop, and finally presents four key research priorities for the field of MMDF.