Combining transfer and ensemble learning models for image and text aspect-based sentiment analysis

Multimodal Aspect-Based Sentiment Analysis (MABSA) is a rapidly evolving field, essential for understanding emotions across different data types like text and images. By analyzing sentiments from multiple sources, MABSA holds great potential for diverse real-world applications such as social media monitoring and customer feedback analysis. This study introduces a novel approach that leverages both machine learning and deep learning techniques to improve sentiment interpretation at a fine-grained level, enabling more precise emotional insights from multimodal data. Our approach integrates a Light Gradient Boosting Machine with advanced models, including Transformer-XL Network (XLNet), Bidirectional Encoder Representations from Transformers (BERT), and its optimized variant, RoBERTa. This hybrid model significantly enhances the accuracy and robustness of aspect-based sentiment analysis. Evaluations on the Twitter 2015 dataset achieved an accuracy of 80.52% and an F1-measure of 76.42%. Further testing on the Twitter 2017 dataset resulted in an accuracy of 73.85% and an F1-measure of 72.68%. These results demonstrate the effectiveness of our method, highlighting its potential for more comprehensive sentiment analysis across multiple modalities.