Leveraging Emoji to Improve Sentiment Classification of Tweets

Recent advances in the Natural Language Processing field have brought good results to a number of interesting tasks, for instance, Linguistic Acceptability, Question Answering, Reading Comprehension, Natural Language Inference, and Sentiment Analysis. Methods, such as ULMFiT, ELMo, BERT, and their derivatives, have achieved increasing success with these tasks, but often requiring substantial amounts of pre-training data and computational resources. We propose a novel methodology to classify the sentiment of tweets, based on BERT but focusing on emojis, treating them as an important source of sentiment as opposed to considering them simple input tokens. Additionally, it is possible to use a previously pre-trained BERT model to warm start ours, greatly reducing the training time required. Experiments on two Brazilian Portuguese datasets - TweetSentBR and 2000-tweets-BR - show that our methodology produces better results than BERT and outperforms the previously published results for both datasets, thus establishing new state-of-the-art results on TweetSentBR with accuracy of 0.7577 (4.8 percentage points absolute improvement) and F-1 score of 0.7395 (8.4 percentage points absolute improvement); and on 2000-tweets-BR with accuracy of 0.8316 (15.2 percentage points absolute improvement) and F-1 score of 0.8151 (24.5 percentage points absolute improvement).