SCoEmbeddings: Encoding Sentiment Information into Contextualized Embeddings for Sentiment Analysis

Contextualized word representations such as ELMo embeddings, can capture rich semantic information and achieve impressive performance in a wide variety of NLP tasks. However, as problems found in Word2Vec and GloVe, we found that ELMo word embeddings also lack enough sentiment information, which may affect sentiment classification performance. Inspired by previous embedding refinement method with sentiment lexicon, we propose an approach that combines contextualized embeddings (ELMo) of the pre-trained model with sentiment information of lexicon to generate sentiment-contextualized embeddings, called SCoEmbeddings. Experimental results show that our SCoEmbeddings achieve higher accuracy than ELMo embeddings, Word2Vec embeddings, and refined Word2Vec embeddings on the SST-5 dataset. Meanwhile, we also visualize embeddings and weights of SCoEmbeddings, demonstrating the effectiveness of our SCoEmbeddings.