Marine Mammal Call Classification Using a Multi-Scale Two-Channel Fusion Network (MT-Resformer)

The classification of high-frequency marine mammal vocalizations often faces challenges due to the limitations of acoustic features, which are sensitive to mid-to-low frequencies but offer low resolution in high-frequency ranges. Additionally, single-channel networks can restrict overall classification performance. To tackle these challenges, we introduce MT-Resformer, an innovative dual-channel model with a multi-scale framework designed for classifying marine mammal vocalizations. Our approach introduces a feature fusion strategy that combines the constant-Q spectrogram with Mel filter-based spectrogram features, effectively overcoming the low resolution of Mel spectrograms in high frequencies. The MT-Resformer model incorporates two key components: a multi-scale parallel residual network (MResNet) and a Transformer network channel. The model employs a multi-level neural perceptron (MLP) to dynamically regulate the weighting of the two channels, enabling flexible feature fusion. Experimental findings validate the proposed approach, yielding classification accuracies of 99.17% on the Watkins dataset and 95.22% on the ChangLong dataset. These results emphasize its outstanding performance.