Combining weighted category-aware contextual information in convolutional neural networks for text classification

Convolutional neural networks (CNNs) are widely used in many natural language processing tasks, which employ some convolutional filters to capture useful semantic features of a text. However, a small window size convolutional filter is short of the ability to capture contextual information, simply increasing the window size may bring the problems of data sparsity and enormous parameters. To capture the contextual information, we propose to use the weighted sum operation to obtain contextual word representation. We present one implicit weighting method and two explicit category-aware weighting methods to assign the weights of the contextual information. Experimental results on five text classification datasets show the effectiveness of our proposed methods.