Hindi fake news detection using transformer ensembles

In the past few decades, due to the growth of social networking sites such as Whatsapp and Facebook, information distribution has been at a level never seen before. Knowing the integrity of information has been a long-standing problem, even more so for the regional languages. Regional languages, such as Hindi, raise challenging problems for fake news detection as they tend to be resource constrained. This limits the amount of data available to efficiently train models for these languages. Most of the existing techniques to detect fake news is targeted towards the English language or involves the manual translation of the language to the English language and then proceeding with Deep Learning methods. Pre-trained transformer based models such as BERT are fine-tuned for the task of fake news detection and are commonly employed for detecting fake news. Other pre-trained transformer models, such as ELECTRA and RoBERTa have also been shown to be able to detect fake news in multiple languages after suitable fine-tuning. In this work, we propose a method for detecting fake news in resource constrained languages such as Hindi more efficiently by using an ensemble of pre-trained transformer models, all of which are individually fine-tuned for the task of fake news detection. We demonstrate that the use of such a transformer ensemble consisting of XLM-RoBERTa, mBERT and ELECTRA is able to improve the efficiency of fake news detection in Hindi by overcoming the drawbacks of individual transformer models.