Multimodal Sentiment Analysis Network Based on Distributional Transformation and Gated Cross-Modal Fusion

Multimodal sentiment analysis aims to synthesize text, audio, and video modalities to extract sentiment information. Existing research focuses on representation learning and feature fusion, but due to inter-modal data distribution differences, fusion models are often difficult to effectively capture inter-modal correlations, especially ignoring unimodal distribution differences, which affects fusion effectiveness. In addition, text features are more important in multimodal sentiment analysis, which increases the challenge of fusing verbal and non-verbal modal information. To address these issues, we propose the Cross-Modal Joint Representation Interaction Network (CMJN), which quantifies the distributional differences between modalities through a Distributed Transformation Layer (DTL) and learns joint representations of verbal and non-verbal using a Gated Cross-Modal Transformer (GCT) to capture inter-modal coherence and complementarity. Experimental results show that CMJN significantly improves multimodal sentiment analysis on CMU-MOSI and CMU-MOSEI datasets.