Recurrent Neural Network Language Model with Part-of-speech for Mandarin Speech Recognition

Recurrent neural network language models (RNNLMs) have been successfully applied in a variety of language processing applications ranging from speech recognition to machine translation. They can fight the curse of dimensionality by learning a distributed representation (word vector). The components of these vectors measure the co-occurrence of the word with context features over a corpus. However, RNNLMs ignore the fact that the meaning of word can vary substantially in different contexts (e.g., for polysemous words). In this paper, we investigate part-of-speech information to address this issue to some extent on the basis of information about the meaning of a word they could provide. Experimental results on Mandarin speech recognition task show that a significant character error reduction of 1.18% absolute (7.72% relative) was obtained when using recurrent neural network language model with part-of-speech.