Robust cross-lingual knowledge base question answering via knowledge distillation

被引:1
作者
Wang, Shaofei [1 ]
Dang, Depeng [1 ]
机构
[1] Beijing Normal Univ, Sch Artificial Intelligence, Beijing, Peoples R China
基金
中国国家自然科学基金;
关键词
Knowledge base question answering; Cross-lingual; Knowledge distillation; Noise-generating model; Robustness; Machine translation;
D O I
10.1108/DTA-12-2020-0312
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Purpose Previous knowledge base question answering (KBQA) models only consider the monolingual scenario and cannot be directly extended to the cross-lingual scenario, in which the language of questions and that of knowledge base (KB) are different. Although a machine translation (MT) model can bridge the gap through translating questions to the language of KB, the noises of translated questions could accumulate and further sharply impair the final performance. Therefore, the authors propose a method to improve the robustness of KBQA models in the cross-lingual scenario. Design/methodology/approach The authors propose a knowledge distillation-based robustness enhancement (KDRE) method. Specifically, first a monolingual model (teacher) is trained by ground truth (GT) data. Then to imitate the practical noises, a noise-generating model is designed to inject two types of noise into questions: general noise and translation-aware noise. Finally, the noisy questions are input into the student model. Meanwhile, the student model is jointly trained by GT data and distilled data, which are derived from the teacher when feeding GT questions. Findings The experimental results demonstrate that KDRE can improve the performance of models in the cross-lingual scenario. The performance of each module in KBQA model is improved by KDRE. The knowledge distillation (KD) and noise-generating model in the method can complementarily boost the robustness of models. Originality/value The authors first extend KBQA models from monolingual to cross-lingual scenario. Also, the authors first implement KD for KBQA to develop robust cross-lingual models.
引用
收藏
页码:661 / 681
页数:21
相关论文
共 43 条
  • [1] [Anonymous], 2014, ARXIV14063676
  • [2] Berant J., 2013, P 2013 C EMP METH NA, P1533
  • [3] Bollacker K, 2008, P 2008 ACM SIGMOD IN, P1247, DOI 10.1145/1376616.1376746
  • [4] Bordes A., 2015, CoRR abs/1506.02075
  • [5] Bordes A., 2013, ADV NEURAL INFORM PR, V26, P2787, DOI DOI 10.5555/2999792.2999923
  • [6] Buzaaba Happy., 2021, SN Computer Science, V2, P1, DOI DOI 10.1007/S42979-020-00421-7
  • [7] Chen GB, 2017, ADV NEUR IN, V30
  • [8] Cimiano P, 2013, LECT NOTES COMPUT SC, V8138, P321, DOI 10.1007/978-3-642-40802-1_30
  • [9] Clark K, 2019, 57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019), P5931
  • [10] Knowledge and data mining for recent and advanced applications using emerging technologies
    Dhyani, Vijaya
    [J]. DATA TECHNOLOGIES AND APPLICATIONS, 2020, 54 (01) : 64 - 65