Superhuman performance on sepsis MIMIC-III data by distributional reinforcement learning

被引:7
作者
Boeck, Markus [1 ]
Malle, Julien [1 ]
Pasterk, Daniel [1 ]
Kukina, Hrvoje [1 ]
Hasani, Ramin [2 ]
Heitzinger, Clemens [1 ,3 ]
机构
[1] Tech Univ Wien TU Wien, Vienna, Austria
[2] MIT, 77 Massachusetts Ave, Cambridge, MA 02139 USA
[3] TU Wien, CAIML Ctr Artificial Intelligence & Machine Learn, Vienna, Austria
基金
奥地利科学基金会;
关键词
D O I
10.1371/journal.pone.0275358
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
We present a novel setup for treating sepsis using distributional reinforcement learning (RL). Sepsis is a life-threatening medical emergency. Its treatment is considered to be a challenging high-stakes decision-making problem, which has to procedurally account for risk. Treating sepsis by machine learning algorithms is difficult due to a couple of reasons: There is limited and error-afflicted initial data in a highly complex biological system combined with the need to make robust, transparent and safe decisions. We demonstrate a suitable method that combines data imputation by a kNN model using a custom distance with state representation by discretization using clustering, and that enables superhuman decision-making using speedy Q-learning in the framework of distributional RL. Compared to clinicians, the recovery rate is increased by more than 3% on the test data set. Our results illustrate how risk-aware RL agents can play a decisive role in critical situations such as the treatment of sepsis patients, a situation acerbated due to the COVID-19 pandemic (Martineau 2020). In addition, we emphasize the tractability of the methodology and the learning behavior while addressing some criticisms of the previous work (Komorowski et al. 2018) on this topic.
引用
收藏
页数:18
相关论文
共 34 条
[1]   A Review of Biomarkers and Physiomarkers in Pediatric Sepsis [J].
Alqahtani, Mashael F. ;
Marsillio, Lauren E. ;
Rozenfeld, Ranna A. .
CLINICAL PEDIATRIC EMERGENCY MEDICINE, 2014, 15 (02) :177-184
[2]  
AZAR M. G., 2011, ADV NEURAL INFORM PR, P2411
[3]  
Bellemare MG, 2017, PR MACH LEARN RES, V70
[4]   Sepsis and Coronavirus Disease 2019: Common Features and Anti-Inflammatory Therapeutic Approaches [J].
Beltran-Garcia, Jesus ;
Osca-Verdegal, Rebeca ;
Pallardo, Federico V. ;
Ferreres, Jose ;
Rodriguez, Maria ;
Mulet, Sandra ;
Ferrando-Sanchez, Carolina ;
Carbonell, Nieves ;
Garcia-Gimenez, Jose Luis .
CRITICAL CARE MEDICINE, 2020, 48 (12) :1841-1844
[5]   Julia: A Fresh Approach to Numerical Computing [J].
Bezanson, Jeff ;
Edelman, Alan ;
Karpinski, Stefan ;
Shah, Viral B. .
SIAM REVIEW, 2017, 59 (01) :65-98
[6]  
Bezdek JC., 1981, PATTERN RECOGN, P65
[7]  
Bock M, 2020, SIAM J MATH DATA SCI
[8]  
Bradford RJ., 2019, ARXIV
[9]  
Brockman Greg, 2016, arXiv
[10]   A SINGULAR VALUE THRESHOLDING ALGORITHM FOR MATRIX COMPLETION [J].
Cai, Jian-Feng ;
Candes, Emmanuel J. ;
Shen, Zuowei .
SIAM JOURNAL ON OPTIMIZATION, 2010, 20 (04) :1956-1982