SEPM: Multiscale semantic enhancement-progressive multimodal fusion network for fake news detection

Our lives are negatively impacted and harmed by fake news. Academics and industries both are paying more and more attention to multimodal fake news detection as a result of the growth of the internet and social media. However, current methods tend to use encoders to directly encode news content, which results in the lack of a large amount of text and image high-level semantic information. At the same time, these methods focus on independently extracting individual modal features and then fusing them, which has achieved some results but is difficult to narrow the modal gap between multimodal information. Furthermore, the forgery forms of fake news are becoming more diverse, increasing the difficulty of detection. Therefore, we propose a multiscale semantic enhancement-progressive multimodal fusion network (SEPM) for detecting fake news, which enhances image and text semantic features while fusing multimodal information in a staged and hierarchical fashion, and is suitable for detecting various forms of fake news. Specifically, we get descriptions of the entities to enhance text semantic understanding with contextual associations, and we propose an image semantic enhancement module that combines the global and local features of the image. To further enhance the performance of model, we also create a progressive and fusion method to fuse the multimodal cues that are obtained. Extensive studies on two public datasets, Twitter and Weibo, illustrate that our method surpasses existing latest methods on the two datasets by 3.5 % and 2 %, respectively, indicating the effectiveness of the SEPM model in recognizing fake news.