DiaBLa: a corpus of bilingual spontaneous written dialogues for machine translation

被引：4
作者：
Bawden, Rachel [1 ]
Bilinski, Eric [2 ]
Lavergne, Thomas [3 ]
Rosset, Sophie [2 ]
机构：
[1] Univ Edinburgh, Sch Informat, Edinburgh, Midlothian, Scotland
[2] Univ Paris Saclay, LIMSI, CNRS, Orsay, France
[3] Univ Paris Sud, LIMSI, CNRS, Univ Paris Saclay, Orsay, France
来源：
LANGUAGE RESOURCES AND EVALUATION | 2021年 / 55卷 / 03期
关键词：
Machine translation; Corpus; Dataset; Evaluation; Bilingual; Dialogue; Chat;
D O I：
10.1007/s10579-020-09514-4
中图分类号：
TP39 [计算机的应用];
学科分类号：
081203 ; 0835 ;
摘要：
We present a new English-French dataset for the evaluation of Machine Translation (MT) for informal, written bilingual dialogue. The test set contains 144 spontaneous dialogues (5700+ sentences) between native English and French speakers, mediated by one of two neural MT systems in a range of role-play settings. The dialogues are accompanied by fine-grained sentence-level judgments of MT quality, produced by the dialogue participants themselves, as well as by manually normalised versions and reference translations produced a posteriori. The motivation for the corpus is twofold: to provide (i) a unique resource for evaluating MT models, and (ii) a corpus for the analysis of MT-mediated communication. We provide an initial analysis of the corpus to confirm that the participants' judgments reveal perceptible differences in MT quality between the two MT systems used.
引用
收藏
页码：635 / 660
页数：26
相关论文
共 22 条
  • [1] [Anonymous], 2017, P SOFTW DEM 15 C EUR, DOI DOI 10.18653/V1
  • [2] [Anonymous], 2007, International Journal of Computational Linguistics & Chinese Language Processing
  • [3] Bahdanau D, 2016, Arxiv, DOI [arXiv:1409.0473, DOI 10.48550/ARXIV.1409.0473]
  • [4] Bawden R., 2018, THESIS
  • [5] Bawden R., 2018, P C N AM CHAPT ASS C, V1, P1304
  • [6] On the interpretation of x(2) from contingency tables, and the calculation of P
    Fisher, RA
    [J]. JOURNAL OF THE ROYAL STATISTICAL SOCIETY, 1922, 85 : 87 - 94
  • [7] Higashinaka R, 2016, LREC 2016 - TENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, P3146
  • [8] Isabelle Pierre, 2017, P 2017 C EMP METH NA, P2486, DOI [10.18653/v1/d17-1263, DOI 10.18653/V1/D17-1263]
  • [9] Junczys-Dowmunt M., 2018, ARXIV180400344CS
  • [10] King Margaret, 1990, 13 INT C COMPUTATION
123