ATBBC: Named entity recognition in emergency domains based on joint BERT-BILSTM-CRF adversarial training

被引:5
作者
Cai, Buqing [1 ]
Tian, Shengwei [1 ]
Yu, Long [2 ]
Long, Jun [3 ]
Zhou, Tiejun [4 ]
Wang, Bo [1 ]
机构
[1] Univ Xinjiang, Sch Software, Urumqi, Xinjiang, Peoples R China
[2] Univ Xinjiang, Coll Network Ctr, Urumqi, Xinjiang, Peoples R China
[3] Univ Cent South, Inst Big Data Res, Changsha, Peoples R China
[4] Internet Informat Secur Ctr, Urumqi, Xinjiang, Peoples R China
基金
中国国家自然科学基金;
关键词
Named Entity Recognition; BERT; BILSTM; CRF; Adversarial Training;
D O I
10.3233/JIFS-232385
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
With the rapid growth of Internet penetration, identifying emergency information from network news has become increasingly significant for emergency monitoring and early warning. Although deep learning models have been commonly used in Chinese Named Entity Recognition (NER), they require a significant amount of well-labeled training data, which is difficult to obtain for emergencies. In this paper, we propose anNERmodel that combines bidirectional encoder representations from Transformers (BERT), bidirectional long-short-term memory (BILSTM), and conditional random field (CRF) based on adversarial training (ATBBC) to address this issue. Firstly, we constructed an emergency dataset (ED) based on the classification and coding specifications of the national emergency platform system. Secondly, we utilized the BERT pretraining model with adversarial training to extract text features. Finally, BILSTM and CRF were used to predict the probability distribution of entity labels and decode the probability distribution into corresponding entity labels.Experiments on the ED show that our model achieves an F1-score of 85.39% on the test dataset, which proves the effectiveness of our model.
引用
收藏
页码:4063 / 4076
页数:14
相关论文
共 27 条
[1]  
[Anonymous], 2014, Journal of Computational Information Systems
[2]   Augmenting Open-Domain Event Detection with Synthetic Data from GPT-2 [J].
Ben Veyseh, Amir Pouran ;
Minh Van Nguyen ;
Min, Bonan ;
Thien Huu Nguyen .
MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, ECML PKDD 2021: RESEARCH TRACK, PT III, 2021, 12977 :644-660
[3]  
Biswas S., 2009, HYBRID ORIYA NAMED E, P639
[4]  
Dai Z., 2019, 2019 12 INT C IM SIG, P1, DOI [DOI 10.1109/CISP-BMEI48845.2019.8965823, 10.1109/CISP-BMEI48845.2019.8965823]
[5]  
Devlin J., ARXIV
[6]  
[杜志强 Du Zhiqiang], 2020, [武汉大学学报. 信息科学版, Geomatics and Information Science of Wuhan University], V45, P1344
[7]  
Graves A, 2005, LECT NOTES COMPUT SC, V3697, P799
[8]  
Hochreiter S, 1997, NEURAL COMPUT, V9, P1735, DOI [10.1162/neco.1997.9.8.1735, 10.1007/978-3-642-24797-2, 10.1162/neco.1997.9.1.1]
[9]  
Lafferty J., 2001, INT C MACHINE LEARNI
[10]   UD_BBC: Named entity recognition in social network combined BERT-BiLSTM-CRF with active learning [J].
Li, Wei ;
Du, Yajun ;
Li, Xianyong ;
Chen, Xiaoliang ;
Xie, Chunzhi ;
Li, Hui ;
Li, Xiaolei .
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2022, 116