Target-oriented multimodal sentiment classification by using topic model and gating mechanism

Multimodality sentiment classification of social media attracts increasing attention, whose main purpose is to predict the sentiment of the target mentioned in the posts. Current research mainly focuses on integrating the multimodal data, but fails to consider the impacts on the target. In this work, we tend to propose a target-oriented multimodal sentiment classification model. Specifically, our model starts with exploiting the target-oriented topic within the text. Then, a multi-head attention network is established to learn the multimodal interaction among textual, visual and topic information, based on which the target-oriented representations of the topic, the text and the image are obtained. Moreover, a gating unit to fuse the multimodal information is also built up. On the task of target-oriented multimodal sentiment classification, experiments on multimodal samples are carried out on manually annotated the dataset. Experimental results reveal that our method significantly reduces the gap over each given target, which sets a foundation to achieve the state-of-arts sentiment classification results.