BioWordVec, improving biomedical word embeddings with subword information and MeSH

被引:276
作者
Zhang, Yijia [1 ,2 ]
Chen, Qingyu [1 ]
Yang, Zhihao [2 ]
Lin, Hongfei [2 ]
Lu, Zhiyong [1 ]
机构
[1] NIH, NCBI, NLM, Bethesda, MD 20894 USA
[2] Dalian Univ Technol, Sch Comp Sci & Technol, Dalian 116023, Liaoning, Peoples R China
关键词
DRUG INTERACTION EXTRACTION;
D O I
10.1038/s41597-019-0055-0
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Distributed word representations have become an essential foundation for biomedical natural language processing (BioNLP), text mining and information retrieval. Word embeddings are traditionally computed at the word level from a large corpus of unlabeled text, ignoring the information present in the internal structure of words or any information available in domain specific structured resources such as ontologies. However, such information holds potentials for greatly improving the quality of the word representation, as suggested in some recent studies in the general domain. Here we present BioWordVec: an open set of biomedical word vectors/embeddings that combines subword information from unlabeled biomedical text with a widely-used biomedical controlled vocabulary called Medical Subject Headings (MeSH). We assess both the validity and utility of our generated word embeddings over multiple NLP tasks in the biomedical domain. Our benchmarking results demonstrate that our word embeddings can result in significantly improved performance over the previous state of the art in those challenging tasks.
引用
收藏
页数:9
相关论文
共 37 条
[1]  
[Anonymous], P 52 ANN M ASS COMP, DOI DOI 10.3115/VL/P14-1146
[2]   A neural probabilistic language model [J].
Bengio, Y ;
Ducharme, R ;
Vincent, P ;
Jauvin, C .
JOURNAL OF MACHINE LEARNING RESEARCH, 2003, 3 (06) :1137-1155
[3]  
Bojanowski Piotr, 2017, Trans. Assoc. Comput. Linguist., V5, P135, DOI DOI 10.1162/TACL_A_00051
[4]   Comparative experiments on learning information extractors for proteins and their interactions [J].
Bunescu, R ;
Ge, RF ;
Kate, RJ ;
Marcotte, EM ;
Mooney, RJ ;
Ramani, AK ;
Wong, YW .
ARTIFICIAL INTELLIGENCE IN MEDICINE, 2005, 33 (02) :139-155
[5]   A Cost-Efficient Communication Framework for Battery-Switch-Based Electric Vehicle Charging [J].
Cao, Yue ;
Yang, Shusen ;
Min, Geyong ;
Zhang, Xing ;
Song, Houbing ;
Kaiwartya, Omprakash ;
Aslam, Nauman .
IEEE COMMUNICATIONS MAGAZINE, 2017, 55 (05) :162-169
[6]  
Cer D., 2017, P 11 INT WORKSH SEM, P1, DOI DOI 10.18653/V1/S17-2001
[7]  
Chen Q., 2018, P BIOCREATIVE OHNLP, P5
[8]   Sentence Similarity Measures Revisited: Ranking Sentences in Pubmed Documents [J].
Chen, Qingyu ;
Kim, Sun ;
Wilbur, W. John ;
Lu, Zhiyong .
ACM-BCB'18: PROCEEDINGS OF THE 2018 ACM INTERNATIONAL CONFERENCE ON BIOINFORMATICS, COMPUTATIONAL BIOLOGY, AND HEALTH INFORMATICS, 2018, :531-532
[9]  
Chiu B., 2016, How to train good word embeddings for biomedical NLP, P166, DOI [DOI 10.18653/V1/W16-2922, 10.18653/v1/w16-2922]
[10]  
Collado-Vides, 2017, DATABASE, V2017