Combining Explicit and Implicit Semantic Similarity Information for Word Embeddings

In this paper, we propose a new framework that combines both explicit and implicit semantic similarity information for training word embeddings. While the former determines the similarity degree between two words explicitly, the latter reflects word similarities implicitly through contextual and relational similarity. We also propose a novel concept called relative similarity in vocabulary, which deliberately utilizes explicit semantic similarity information (word's definition in particular) for word embeddings. We conduct experimental studies on various word similarity and word categorization datasets. The results show that our framework compares favorably to a number of state-of-the-art approaches for word embeddings.