POS-based Word Alignment for Small Corpus

被引:0
作者
Srivastava, Jyoti [1 ]
Sanyal, Sudip [1 ]
机构
[1] Indian Inst Informat Technol, Informat Technol, Allahabad, Uttar Pradesh, India
来源
PROCEEDINGS OF 2015 INTERNATIONAL CONFERENCE ON ASIAN LANGUAGE PROCESSING | 2015年
关键词
Word alignment; Statistical Machine Translation; POS tagger; Small Corpus;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
A good quality word alignment system needs a high quality, domain specific and large size of the parallel corpus for training. Finding a high quality parallel corpus for a particular language pair in a specific domain is expensive and hard to find. Conversely, using a small corpus has some advantages like less training time and low memory requirement. It can be created or corrected manually. So this paper is an effort to achieve good quality word alignment with a small size of parallel corpus. This paper describes use of part of speech (POS) tag to improve the performance of statistical word alignment. This approach works well with small size of the corpus. Experiments were conducted on TDIL sample tourism corpus of 1000 sentences for English-Hindi language pair. Out of these 1000 sentences 950 sentences are used for training and 50 sentences are used for testing. F-measure is increased by approximately 4% and Alignment Error Rate (AER) decreased by approximately 4 % in comparison to baseline system for word alignment GIZA++.
引用
收藏
页码:37 / 40
页数:4
相关论文
共 13 条
[1]  
[Anonymous], 2005, P 43 ANN M ASS COMP
[2]  
[Anonymous], STAT APPROACH FACTOR
[3]  
[Anonymous], ACL 44
[4]  
Aswani Niraj., 2005, Proceedings of the ACL Workshop on Building and Using Parallel Texts - ParaText'05, P57, DOI DOI 10.3115/1654449.1654458
[5]  
Bodrumlu Tugba., 2009, Proceedings of the NAACL/HLT Workshop on Integer Programming for Natural Language Processing, P28
[6]  
Brown P. F., 1993, Computational Linguistics, V19, P263
[7]   Alignment-based extraction of multiword expressions [J].
Caseli, Helena de Medeiros ;
Ramisch, Carlos ;
Volpe Nunes, Maria das Gracas ;
Villavicencio, Aline .
LANGUAGE RESOURCES AND EVALUATION, 2010, 44 (1-2) :59-77
[8]  
Gale WilliamA., 1991, PROC DARPA WORKSHOP, P152
[9]  
Och FJ, 2003, COMPUT LINGUIST, V29, pc, DOI 10.1162/089120103321337421
[10]  
Smadja F, 1996, COMPUT LINGUIST, V22, P1