A comprehensive survey on deep learning-based approaches for multimodal sentiment analysis

Sentiment analysis is an important natural language processing issue that has many applications in various fields. The increasing popularity of social networks and growth and development of their related tools and technologies has led to share the users' multimodal content and opinions in a hybrid form of different media, including texts, images, videos, audio and emojis. The increasing interest of users to share their content using a combination of several media has significantly increased the amount of multimodal data. Most of the comments that users post in the social media have emotional aspects and provide useful indicators for many purposes. Compared to single-modal data, such as text-only or image-only comments, multimodal data contain more useful information and leads to better understanding of the real sentiments of users. Many studies have been conducted in this area, each of which deals with one or some of the various common challenges of multimodal sentiment analysis methods, including incomplete data, heterogeneity of modals, fusion method of the results, interactions between modals, and existence of unrelated, insufficient and redundant data and information. The emergence of deep neural networks and the evolution of deep learning tools and techniques has led to the development of deep learning-based approaches to multimodal sentiment analysis to address its challenges and constraints. This paper is a comprehensive comparative survey of sentiment analysis approaches, challenges, applications, and trends, with a special focus on deep learning-based multimodal sentiment analysis methods. Examining the limitations of the recent studies, describing possible future solutions and evaluating existing challenges are also taken into consideration and future direction of the methods are evaluated.