Can Fake News Detection Models Maintain the Performance through Time? A Longitudinal Evaluation of Twitter Publications

被引:2
|
作者
Guimaraes, Nuno [1 ,3 ]
Figueira, Alvaro [1 ]
Torgo, Luis [2 ]
机构
[1] Univ Porto, CRACS INESCTEC, P-4169007 Porto, Portugal
[2] Dalhousie Univ, Fac Comp Sci, Halifax, NS B3H 1W5, Canada
[3] Rua Campo Alegre S-N Porto, P-4150180 Porto, Portugal
基金
加拿大自然科学与工程研究理事会;
关键词
fake news detection; social networks; false information; machine learning; data mining;
D O I
10.3390/math9222988
中图分类号
O1 [数学];
学科分类号
0701 ; 070101 ;
摘要
The negative impact of false information on social networks is rapidly growing. Current research on the topic focused on the detection of fake news in a particular context or event (such as elections) or using data from a short period of time. Therefore, an evaluation of the current proposals in a long-term scenario where the topics discussed may change is lacking. In this work, we deviate from current approaches to the problem and instead focus on a longitudinal evaluation using social network publications spanning an 18-month period. We evaluate different combinations of features and supervised models in a long-term scenario where the training and testing data are ordered chronologically, and thus the robustness and stability of the models can be evaluated through time. We experimented with 3 different scenarios where the models are trained with 15-, 30-, and 60-day data periods. The results show that detection models trained with word-embedding features are the ones that perform better and are less likely to be affected by the change of topics (for example, the rise of COVID-19 conspiracy theories). Furthermore, the additional days of training data also increase the performance of the best feature/model combinations, although not very significantly (around 2%). The results presented in this paper build the foundations towards a more pragmatic approach to the evaluation of fake news detection models in social networks.
引用
收藏
页数:27
相关论文
共 14 条
  • [1] Performance Evaluation of Transformer-based NLP Models on Fake News Detection Datasets
    Babu, Raveen Narendra
    Lung, Chung-Horng
    Zaman, Marzia
    2023 IEEE 47TH ANNUAL COMPUTERS, SOFTWARE, AND APPLICATIONS CONFERENCE, COMPSAC, 2023, : 316 - 321
  • [2] An Experimental Evaluation of Data Classification Models for Credibility Based Fake News Detection
    Ramkissoon, Amit Neil
    Mohammed, Shareeda
    20TH IEEE INTERNATIONAL CONFERENCE ON DATA MINING WORKSHOPS (ICDMW 2020), 2020, : 93 - 100
  • [3] Ensemble based high performance deep learning models for fake news detection
    Almandouh, Mohammed E.
    Alrahmawy, Mohammed F.
    Eisa, Mohamed
    Elhoseny, Mohamed
    Tolba, A. S.
    SCIENTIFIC REPORTS, 2024, 14 (01):
  • [4] Enhancing Fake News Detection with Large Language Models Through Multi-agent Debates
    Jeptoo, Korir Nancy
    Su, Chengjie
    NATURAL LANGUAGE PROCESSING AND CHINESE COMPUTING, PT II, NLPCC 2024, 2025, 15360 : 474 - 486
  • [5] Fake news detection using enhanced features through text to image transformation with customized models
    Furqan Rustam
    Wajdi aljedaani
    Anca Delia Jurcut
    Sultan Alfarhood
    Mejdl Safran
    Imran Ashraf
    Discover Computing, 27 (1)
  • [6] A Review of Fake News Detection Models: Highlighting the Factors Affecting Model Performance and the Prominent Techniques Used
    Hamed, Suhaib Kh.
    Aziz, Mohd Juzaiddin Ab
    Yaakub, Mohd Ridzwan
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2023, 14 (07) : 379 - 390
  • [7] Fake news detection: comparative evaluation of BERT-like models and large language models with generative AI-annotated data
    Raza, Shaina
    Paulen-Patterson, Drai
    Ding, Chen
    KNOWLEDGE AND INFORMATION SYSTEMS, 2025, : 3267 - 3292
  • [8] An evaluation of the run-time and task-based performance of event detection techniques for Twitter
    Weiler, Andreas
    Grossniklaus, Michael
    Scholl, Marc H.
    INFORMATION SYSTEMS, 2016, 62 : 207 - 219
  • [9] Performance Evaluation of Driving Behavior Identification Models through CAN-BUS Data
    Azadani, Mozhgan Nasr
    Boukerche, Azzedine
    2020 IEEE WIRELESS COMMUNICATIONS AND NETWORKING CONFERENCE (WCNC), 2020,
  • [10] Vae-Clip: Unveiling Deception through Cross-Modal Models and Multi-Feature Integration in Multi-Modal Fake News Detection
    Zhou, Yufeng
    Pang, Aiping
    Yu, Guang
    ELECTRONICS, 2024, 13 (15)