Visual sentiment analysis using data-augmented deep transfer learning techniques

被引:0
作者
Zhiguo Jiang
Waneeza Zaheer
Aamir Wali
S. A. M. Gilani
机构
[1] Anhui Vocational College of Defense Technology,School of Electrical Technology
[2] National University of Computer and Emerging Sciences,Department of Computer Science
来源
Multimedia Tools and Applications | 2024年 / 83卷
关键词
Visual sentiment analysis; Transfer learning; VGG16; Data augmentation;
D O I
暂无
中图分类号
学科分类号
摘要
There has been a growing trend among users of social media platforms to express their emotions using visual content. Visual sentiment analysis is the process of understanding the emotional polarity of images or videos and is still considered a challenging problem in artificial intelligence. Most of the existing models are based on robust machine learning or deep learning techniques. The idea of using deep transfer learning techniques for visual sentiment analysis is fairly new. In this paper, we propose a new approach using data-augmented-transfer learning architecture consisting of a pre-trained VGG16 model that is fine-tuned using SVM with augmented training data. For fine-tuning and evaluation, we initially use two Twitter image datasets. We further validated the proposed model on a third dataset. The commonly used geometric augmentation methods such as rotation, zoom range, width shift, height shift, shear range and horizontal flip were are used. We compare our proposed VGG16-SVM model with 3 other state-of-the-art deep models commonly used for transfer learning and 4 machine learning models (besides SVM) used for fine-tuning. The results show that VGG16-SVM produces the overall best accuracy (94%) and recall (96%) among all transfer learning and machine learning pairs. We also show that our proposed model outperforms all previous studies that use the same dataset.
引用
收藏
页码:17233 / 17249
页数:16
相关论文
共 50 条
[1]  
Ain QT(2017)Sentiment analysis using deep learning techniques: a review Int J Adv Comput Sci Appl 8 424-22
[2]  
Campos V(2017)From pixels to sentiment: fine-tuning CNNs for visual sentiment prediction Image Vis Comput 65 15-611
[3]  
Jou B(2022)Visual sentiment analysis using deep learning models with social media data Appl Sci 12 1030-6967
[4]  
Giro-i Nieto X(2016)Survey of visual sentiment prediction for social media analysis Front Comp Sci 10 602-235
[5]  
Chandrasekaran G(2019)A survey on sentiment analysis and opinion mining for social multimedia Multimed Tools Appl 78 6939-230
[6]  
Antoanela N(2014)Predicting ad liking and purchase intent: large-scale analysis of facial responses to ads IEEE Trans Affect Comput 6 223-4385
[7]  
Andrei G(2017)Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis Neurocomputing 261 217-2525
[8]  
Monica C(2021)Discovering sentimental interaction via graph convolutional network for visual sentiment prediction Appl Sci 11 1404-undefined
[9]  
Hemanth J(2020)Sentiment analysis using deep learning architectures: a review Artif Intell Rev 53 4335-undefined
[10]  
Ji R(2018)Visual sentiment prediction based on automatic discovery of affective regions IEEE Trans Multimedia 20 2513-undefined