I-Topic: An Image-text Topic Modeling Method Based on Community Detection

Multimodal content, including images, audio, and videos, has occupied a rising proportion of online information. Rich semantics and emotions are expressed through the interplay between different data modes. The existing multimodal topic models mainly focus on the semantic alignment between different data modes, while we argue that other data modes should play an equally important role with text. Thus, we proposed an image-text topic modeling method (I-Topic) to preliminarily explore the interplay between different data modes. Multi-modal pre-trained models and community detection are integrated into I-Topic. A new image-text dataset composed of visual memes is constructed for evaluation. Experimental results on three datasets suggested the effectiveness of I-Topic. Manual observation proved that visual elements could help improve the diversity of topics.