Decision-dependent distributionally robust Markov decision process method in dynamic epidemic control

被引:4
作者
Song, Jun [1 ]
Yang, William [1 ]
Zhao, Chaoyue [1 ]
机构
[1] Univ Washington, Dept Ind & Syst Engn, Seattle, WA 98195 USA
关键词
Distributionally robust Markov decision process; distributionally robust optimization; SEIR model; real-time dynamic programming; epidemic control; OPTIMIZATION; QUARANTINE;
D O I
10.1080/24725854.2023.2219281
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
In this article, we present a Distributionally Robust Markov Decision Process (DRMDP) approach for addressing the dynamic epidemic control problem. The Susceptible-Exposed-Infectious-Recovered (SEIR) model is widely used to represent the stochastic spread of infectious diseases, such as COVID-19. Although the Markov Decision Process (MDP) offers a mathematical framework for identifying optimal actions, such as vaccination and transmission-reducing intervention, to combat disease spread calculated using the SEIR model. However, uncertainties in these scenarios demand a more robust approach that is less reliant on error-prone assumptions. The primary objective of our study is to introduce a new DRMDP framework that allows for an ambiguous distribution of transition dynamics. Specifically, we consider the worst-case distribution of these transition probabilities within a decision-dependent ambiguity set. To overcome the computational complexities associated with policy determination, we propose an efficient Real-Time Dynamic Programming (RTDP) algorithm that is capable of computing optimal policies based on the reformulated DRMDP model in an accurate, timely, and scalable manner. Comparative analysis against the classic MDP model demonstrates that the DRMDP achieves a lower proportion of infections and susceptibilities at a reduced cost.
引用
收藏
页码:458 / 470
页数:13
相关论文
共 45 条
[1]  
[Anonymous], 2021, World Economic Outlook
[2]  
[Anonymous], 1992, Moore
[3]   LEARNING TO ACT USING REAL-TIME DYNAMIC-PROGRAMMING [J].
BARTO, AG ;
BRADTKE, SJ ;
SINGH, SP .
ARTIFICIAL INTELLIGENCE, 1995, 72 (1-2) :81-138
[4]   Distributionally robust facility location problem under decision-dependent stochastic demand [J].
Basciftci, Beste ;
Ahmed, Shabbir ;
Shen, Siqian .
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH, 2021, 292 (02) :548-561
[5]  
Bhardwaj A., 2020, AAAI FALL S
[6]   Distributionally robust optimization for sequential decision-making [J].
Chen, Zhi ;
Yu, Pengqian ;
Haskell, William B. .
OPTIMIZATION, 2019, 68 (12) :2393-2422
[7]  
Davies S, 1997, ADV NEUR IN, V9, P1005
[8]   OPTIMAL CONTROL APPLIED TO VACCINATION AND TREATMENT STRATEGIES FOR VARIOUS EPIDEMIOLOGICAL MODELS [J].
Gaff, Holly ;
Schaefer, Elsa .
MATHEMATICAL BIOSCIENCES AND ENGINEERING, 2009, 6 (03) :469-492
[9]   Modelling the COVID-19 epidemic and implementation of population-wide interventions in Italy [J].
Giordano, Giulia ;
Blanchini, Franco ;
Bruno, Raffaele ;
Colaneri, Patrizio ;
Di Filippo, Alessandro ;
Di Matteo, Angela ;
Colaneri, Marta .
NATURE MEDICINE, 2020, 26 (06) :855-+
[10]   Extensions of the SEIR model for the analysis of tailored social distancing and tracing approaches to cope with COVID-19 [J].
Grimm, Veronika ;
Mengel, Friederike ;
Schmidt, Martin .
SCIENTIFIC REPORTS, 2021, 11 (01)