Towards Automated Fact-Checking: An Exploratory Study on Identifying Check-worthy Phrases for Verification

In today's information-saturated social media environment, it is essential to prioritize the verification of potentially false or misleading claims. This need has led to the development of fact-checking, a process dedicated to verifying the truthfulness of statements. Given the limited human resources available to scrutinize all online claims, it is crucial to identify the most critical ones to verify. Therefore, a (semi-)automated system capable of detecting the most urgent and relevant claims for verification is needed. To address this challenge, we evaluate an approach based on Natural Language Processing and Machine Learning techniques. We explore lexical features, embedding models, LLMs, and traditional classification techniques to develop an automated system to classify statements according to their relevance for verification (i.e., their check-worthiness). Our evaluation is based on data collections including checkable statements extracted from tweets and political speeches. Embedding and LLM-based techniques showed great potential to improve the performance of the verification process by effectively prioritizing the most critical and relevant statements for verification.