Sample Intercorrelation-Based Multidomain Fusion Network for Aquatic Human Activity Recognition Using Millimeter-Wave Radar

To address the issue that the existing multidomain fusion methods do not consider data correlation within mini-batch data, the first attempt is made in this letter to propose a method based on a sample intercorrelation learning multidomain fusion network (SIMFNet), which aims to accommodate multivariate domain data and further enhance the aquatic human activity recognition (AHAR) performance. To fully use the radar multidimensional information, the three-branch convolution neural network (CNN) feature extractor is first used to extract domain-specific features from the time-range map (TRM), time-Doppler map (TDM), and cadence velocity diagram (CVD). Then, the multidomain features are fused and fed into the graph construction layer (GCL) to generate instance graphs. Next, a graph aggregation layer (GAL) is applied to aggregate node information from various-hop neighborhood domains. Finally, node-level classification is used to achieve AHAR. The experimental results evaluated on the built AHAR dataset demonstrate that the proposed SIMFNet has better generalization performance than the state-of-the-art multidomain fusion methods.