Doubly-Attentive Decoder for Multi-modal Neural Machine Translation

被引：78

作者：

Calixto, Iacer ^{[1
]}

Liu, Qun ^{[1
]}

Campbell, Nick ^{[2
]}

机构：

[1] Dublin City Univ, Sch Comp, ADAPT Ctr, Dublin, Ireland

[2] Trinity Coll Dublin, Speech Commun Lab, ADAPT Ctr, Dublin 2, Ireland

来源：

PROCEEDINGS OF THE 55TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2017), VOL 1 | 2017年

基金：

爱尔兰科学基金会;

关键词：

D O I：

10.18653/v1/P17-1175

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

We introduce a Multi-modal Neural Machine Translation model in which a doubly-attentive decoder naturally incorporates spatial visual features obtained using pre-trained convolutional neural networks, bridging the gap between image description and translation. Our decoder learns to attend to source-language words and parts of an image independently by means of two separate attention mechanisms as it generates words in the target language. We find that our model can efficiently exploit not just back-translated in-domain multi-modal data but also large general-domain text-only MT corpora. We also report state-of-the-art results on the Multi30k data set.

引用

页码：1913 / 1924

页数：12