Aspect Based Sentiment Analysis on Multimodal Data: A Transformer and Low-Rank Fusion Approach

Video has become the main way for users to share their daily lives and express their opinions, most of which reflect their emotional information. Sentiment analysis of videos can be used to understand user behavior and thus provide better improvements and services. However, due to various reasons, most of the existing sentiment analysis is based on unimodal, and the accuracy of unimodal-based sentiment analysis is low and prone to ambiguity. Some existing multimodal sentiment analysis methods lack effective interaction between modalities. This paper proposes a model called Aspect Based Sentiment Analysis on Multimodal Data: A Transformer and Low-Rank Fusion Approach (ABSA-TLRF). Specifically, ABSA-TLRF utilizes a cross-modal alignment module based on the cross-modal attention mechanism and an efficient method based on low-rank fusion to effectively integrate information within and between modalities. This achieves global-local information interaction, ultimately yielding more accurate emotion fusion results. The model performs sentiment classification, enabling high-level multimodal sentiment analysis. Experimental results indicate that our model outperforms several state-of-the-art methods on three commonly used datasets. Our research suggests that combining a cross-modal alignment module based on the cross-modal attention mechanism and an efficient method based on low-rank fusion can enhance our understanding of multimodal content, thereby improving sentiment analysis performance.