Automated classification of clinical trial eligibility criteria text based on ensemble learning and metric learning

被引:8
作者
Zeng, Kun [1 ]
Xu, Yibin [1 ]
Lin, Ge [2 ]
Liang, Likeng [3 ]
Hao, Tianyong [3 ]
机构
[1] Sun Yat Sen Univ, Sch Data & Comp Sci, Guangzhou, Peoples R China
[2] Sun Yat Sen Univ, Natl Engn Res Ctr Digital Life, Guangzhou, Peoples R China
[3] South China Normal Univ, Sch Comp Sci, Guangzhou, Peoples R China
基金
中国国家自然科学基金;
关键词
Eligibility criteria classification; Metric learning; Focal loss; Ensemble learning; Clinical trial; INFORMATION;
D O I
10.1186/s12911-021-01492-z
中图分类号
R-058 [];
学科分类号
摘要
Background Eligibility criteria are the primary strategy for screening the target participants of a clinical trial. Automated classification of clinical trial eligibility criteria text by using machine learning methods improves recruitment efficiency to reduce the cost of clinical research. However, existing methods suffer from poor classification performance due to the complexity and imbalance of eligibility criteria text data. Methods An ensemble learning-based model with metric learning is proposed for eligibility criteria classification. The model integrates a set of pre-trained models including Bidirectional Encoder Representations from Transformers (BERT), A Robustly Optimized BERT Pretraining Approach (RoBERTa), XLNet, Pre-training Text Encoders as Discriminators Rather Than Generators (ELECTRA), and Enhanced Representation through Knowledge Integration (ERNIE). Focal Loss is used as a loss function to address the data imbalance problem. Metric learning is employed to train the embedding of each base model for feature distinguish. Soft Voting is applied to achieve final classification of the ensemble model. The dataset is from the standard evaluation task 3 of 5th China Health Information Processing Conference containing 38,341 eligibility criteria text in 44 categories. Results Our ensemble method had an accuracy of 0.8497, a precision of 0.8229, and a recall of 0.8216 on the dataset. The macro F1-score was 0.8169, outperforming state-of-the-art baseline methods by 0.84% improvement on average. In addition, the performance improvement had a p-value of 2.152e-07 with a standard t-test, indicating that our model achieved a significant improvement. Conclusions A model for classifying eligibility criteria text of clinical trials based on multi-model ensemble learning and metric learning was proposed. The experiments demonstrated that the classification performance was improved by our ensemble model significantly. In addition, metric learning was able to improve word embedding representation and the focal loss reduced the impact of data imbalance to model performance.
引用
收藏
页数:10
相关论文
共 37 条
  • [1] [Anonymous], 2002, Advances in Neural Information Processing Systems
  • [2] An approach for transgender population information extraction and summarization from clinical trial text
    Chen, Boyu
    Jin, Hao
    Yang, Zhiwen
    Qu, Yingying
    Weng, Heng
    Hao, Tianyong
    [J]. BMC MEDICAL INFORMATICS AND DECISION MAKING, 2019, 19 (Suppl 2)
  • [3] Classifying Eligibility Criteria in Clinical Trials Using Active Deep Learning
    Chuan, Ching-Hua
    [J]. 2018 17TH IEEE INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND APPLICATIONS (ICMLA), 2018, : 305 - 310
  • [4] Devlin J, 2019, 2019 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES (NAACL HLT 2019), VOL. 1, P4171
  • [5] Fuzzy C-Means Clustering With Local Information and Kernel Metric for Image Segmentation
    Gong, Maoguo
    Liang, Yan
    Shi, Jiao
    Ma, Wenping
    Ma, Jingjing
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2013, 22 (02) : 573 - 584
  • [6] Modernizing Clinical Trial Eligibility: Recommendations of the American Society of Clinical Oncology-Friends of Cancer Research Minimum Age Working Group
    Gore, Lia
    Ivy, S. Percy
    Balis, Frank M.
    Rubin, Eric
    Thornton, Katherine
    Donoghue, Martha
    Roberts, Samantha
    Bruinooge, Suanna
    Ersek, Jennifer
    Goodman, Nancy
    Schenkel, Caroline
    Reaman, Gregory
    [J]. JOURNAL OF CLINICAL ONCOLOGY, 2017, 35 (33) : 3781 - 3787
  • [7] Is that you? Metric Learning Approaches for Face Identification
    Guillaumin, Matthieu
    Verbeek, Jakob
    Schmid, Cordelia
    [J]. 2009 IEEE 12TH INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2009, : 498 - 505
  • [8] Extractive summarization of clinical trial descriptions
    Gulden, Christian
    Kirchner, Melanie
    Schuettler, Christina
    Hinderer, Marc
    Kampf, Marvin
    Prokosch, Hans-Ulrich
    Toddenroth, Dennis
    [J]. INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS, 2019, 129 : 114 - 121
  • [9] Clustering clinical trials with similar eligibility criteria features
    Hao, Tianyong
    Rusanov, Alexander
    Boland, Mary Regina
    Weng, Chunhua
    [J]. JOURNAL OF BIOMEDICAL INFORMATICS, 2014, 52 : 112 - 120
  • [10] He Z, 2014, AMIA 2014 ANN S AMIA, P777