MCAN: multimodal cross-aware network for fake news detection by extracting semantic-physical feature consistency

Social platforms are vital for information dissemination but also contribute to the spread of fake news, causing confusion and misinformation. To combat this, advancements in detection technology are crucial, particularly for posts that combine text and images, as they often present misleading information. However, current research often overlooks the extraction of key features from both modalities, missing critical elements like writing styles and image manipulations, hampering detection accuracy. In response, this work introduces the MCAN (Multimodal Cross-Aware Network), which freezes the parameters of BERT and ResNet50 to extract semantic features from text and images. It includes a text vocabulary network to analyze writing style differences and employs error level analysis to detect image manipulations. By integrating these features through a flexible multimodal fusion subnetwork with Bimodal Cross-Attention Blocks, MCAN effectively identifies fake news. Experimental results on two popular datasets demonstrate that MCAN outperforms existing baseline models in predictive accuracy.