Multilingual sequence to sequence convolutional machine translation

In this paper, to improve the translation quality of a sentence, Convolutional sequence to sequence architecture has been applied to English-Punjabi, Punjabi-English, Hindi-Punjabi, Punjabi-Hindi language pairs. The Convolution architecture consists of Gated Linear Unit (GLU) and a Multi-Hop attention mechanism. The GLU mechanism controls the information flow through hidden units to produce good translations. The Multi-Hop attention is an enhanced version of the attention mechanism that allows the model to make repeated gaze on the sentence than looking at the sentence only once. As compared to the Recurrent Neural Network (RNN), the Convolution based technique provides better accuracy, captures long-range dependencies between the words, capture local context, and allows parallelization over every element in a sequence without depending on the computations of the previous time steps. In this work, we have applied the architecture on four Indian language pairs and compared the performance of our model with conventional models using BLEU, METEOR and WER as evaluation metrics. Our model outperforms recurrent neural networks by 2 BLEU points on English-Punjabi, 2.5 BLEU points on Punjabi-English, 3.5 BLEU points on Hindi-Punjabi translations, and 1.5 BLEU points on Punjabi-Hindi.