Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury

被引:8
作者
Connor, Skylar [1 ]
Li, Ting [1 ]
Roberts, Ruth [2 ,3 ]
Thakkar, Shraddha [4 ]
Liu, Zhichao [1 ]
Tong, Weida [1 ]
机构
[1] US FDA, Natl Ctr Toxicol Res, Jefferson, AR 20993 USA
[2] ApconiX Ltd, Macclesfield, England
[3] Univ Birmingham, Dept Biosci, Birmingham, Warwickshire, England
[4] US FDA, Ctr Drug Evaluat & Res, Silver Spring, MD USA
来源
FRONTIERS IN ARTIFICIAL INTELLIGENCE | 2022年 / 5卷
关键词
adaptability; AI; deep learning; drug-induced liver injury (DILI); drug safety; risk assessment; regulatory science;
D O I
10.3389/frai.2022.1034631
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Artificial intelligence (AI) has played a crucial role in advancing biomedical sciences but has yet to have the impact it merits in regulatory science. As the field advances, in silico and in vitro approaches have been evaluated as alternatives to animal studies, in a drive to identify and mitigate safety concerns earlier in the drug development process. Although many AI tools are available, their acceptance in regulatory decision-making for drug efficacy and safety evaluation is still a challenge. It is a common perception that an AI model improves with more data, but does reality reflect this perception in drug safety assessments? Importantly, a model aiming at regulatory application needs to take a broad range of model characteristics into consideration. Among them is adaptability, defined as the adaptive behavior of a model as it is retrained on unseen data. This is an important model characteristic which should be considered in regulatory applications. In this study, we set up a comprehensive study to assess adaptability in AI by mimicking the real-world scenario of the annual addition of new drugs to the market, using a model we previously developed known as DeepDILI for predicting drug-induced liver injury (DILI) with a novel Deep Learning method. We found that the target test set plays a major role in assessing the adaptive behavior of our model. Our findings also indicated that adding more drugs to the training set does not significantly affect the predictive performance of our adaptive model. We concluded that the proposed adaptability assessment framework has utility in the evaluation of the performance of a model over time.
引用
收藏
页数:9
相关论文
共 29 条
[1]   Emerging technologies and their impact on regulatory science [J].
Anklam, Elke ;
Bahl, Martin Iain ;
Ball, Robert ;
Beger, Richard D. ;
Cohen, Jonathan ;
Fitzpatrick, Suzanne ;
Girard, Philippe ;
Halamoda-Kenzaoui, Blanka ;
Hinton, Denise ;
Hirose, Akihiko ;
Hoeveler, Arnd ;
Honma, Masamitsu ;
Hugas, Marta ;
Ishida, Seichi ;
Kass, George En ;
Kojima, Hajime ;
Krefting, Ira ;
Liachenko, Serguei ;
Liu, Yan ;
Masters, Shane ;
Marx, Uwe ;
McCarthy, Timothy ;
Mercer, Tim ;
Patri, Anil ;
Pelaez, Carmen ;
Pirmohamed, Munir ;
Platz, Stefan ;
Ribeiro, Alexandre J. S. ;
Rodricks, Joseph, V ;
Rusyn, Ivan ;
Salek, Reza M. ;
Schoonjans, Reinhilde ;
Silva, Primal ;
Svendsen, Clive N. ;
Sumner, Susan ;
Sung, Kyung ;
Tagle, Danilo ;
Tong, Li ;
Tong, Weida ;
Van den Eijnden-van-Raaij, Janny ;
Vary, Neil ;
Wang, Tao ;
Waterton, John ;
Wang, May ;
Wen, Hairuo ;
Wishart, David ;
Yuan, Yinyin ;
Slikker, William Jr Jr .
EXPERIMENTAL BIOLOGY AND MEDICINE, 2022, 247 (01) :1-75
[2]  
[Anonymous], 2022, 117 C 2021 2022
[3]  
[Anonymous], 2021, 117 C 2021 2022
[4]  
[Anonymous], 2020, 116 C 2019 2020
[5]   The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation [J].
Chicco, Davide ;
Totsch, Niklas ;
Jurman, Giuseppe .
BIODATA MINING, 2021, 14 (01) :1-22
[6]   The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation [J].
Chicco, Davide ;
Jurman, Giuseppe .
BMC GENOMICS, 2020, 21 (01)
[7]  
FDA, 2021, ADV REG SCI FDA FOC
[8]  
Groce A., 2002, LECT NOTES COMPUTER, P2280, DOI [10.1007/3-540-46002-0_25, DOI 10.1007/3-540-46002-0_25]
[9]   Artificial intelligence to deep learning: machine intelligence approach for drug discovery [J].
Gupta, Rohan ;
Srivastava, Devesh ;
Sahu, Mehar ;
Tiwari, Swati ;
Ambasta, Rashmi K. ;
Kumar, Pravir .
MOLECULAR DIVERSITY, 2021, 25 (03) :1315-1360
[10]   Advancing Regulatory Science [J].
Hamburg, Margaret A. .
SCIENCE, 2011, 331 (6020) :987-987