A Dynamic Weighted Multimodal Fusion Fake Information Detection Method

Regarding the task of recognizing fake information, it is difficult to perform accurate recognition based on single-modal fake information detection models in the face of combined graphic and textual fake information. To address the problem wherein current multimodal detection models usually use splicing for multimodal fusion, which leads to a redundancy of modal information during feature fusion and cannot effectively combine the advantages of different modalities, this paper proposes a dynamic weighted multimodal fusion network for feature fusion based on the attention mechanism. To address the problem of inadequate text content extraction, extracting text abstract features using the TextRank algorithm, and the abstract features are introduced into the detection model as an independent modality. The dynamic weighted multimodal disinformation detection model (DWMF) proposed by text uses BERT and vision transformer to extract text and image features, respectively, after which the text, image, and abstract features are fused using a fusion network and then classified. The model achieves an accuracy rate of 98.1% with the MCG-FNeWS public dataset, and it F1- score and accuracy are better than those of existing multimodal models.