Reinforcement Learning Approach to Feedback Stabilization Problem of Probabilitic Boolcan Control Networks

被引:58
作者
Acernese, Antonio [1 ]
Yerudkar, Amol [1 ]
Glielmo, Luigi [1 ]
Del Vecchio, Carmen Del [1 ]
机构
[1] Univ Sannio, Dept Engn, I-82100 Benevento, Italy
来源
IEEE CONTROL SYSTEMS LETTERS | 2021年 / 5卷 / 01期
关键词
Probabilistic Boolean control networks; Q-learning; feedback stabilization; systems biology; BOOLEAN NETWORKS;
D O I
10.1109/LCSYS.2020.3001993
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In this letter, we study the control of probabilistic Boolean control networks (PBCNs) by leveraging a model-free reinforcement learning (RL) technique. In particular, we propose a Q-learning (QL) based approach to address the feedback stabilization problem of PBCNs, and we design optimal state feedback controllers such that the PBCN is stabilized at a given equilibrium point. The optimal controllers are designed for both finite-time stability and asymptotic stability of PBCNs. In order to verify the convergence of the proposed QL algorithm, the obtained optimal policy is compared with the optimal solutions of model-based techniques, namely value iteration (VI) and semi-tensor product (STP) methods. Finally, some PBCN models of gene regulatory networks (GRNs) are considered to verify the obtained results.
引用
收藏
页码:337 / 342
页数:6
相关论文
共 29 条
[11]   ON THE CONVERGENCE OF STOCHASTIC ITERATIVE DYNAMIC-PROGRAMMING ALGORITHMS [J].
JAAKKOLA, T ;
JORDAN, MI ;
SINGH, SP .
NEURAL COMPUTATION, 1994, 6 (06) :1185-1201
[12]   METABOLIC STABILITY AND EPIGENESIS IN RANDOMLY CONSTRUCTED GENETIC NETS [J].
KAUFFMAN, SA .
JOURNAL OF THEORETICAL BIOLOGY, 1969, 22 (03) :437-&
[13]   Reinforcement Learning and Feedback Control USING NATURAL DECISION METHODS TO DESIGN OPTIMAL ADAPTIVE CONTROLLERS [J].
Lewis, Frank L. ;
Vrabie, Draguna ;
Vamvoudakis, Kyriakos G. .
IEEE CONTROL SYSTEMS MAGAZINE, 2012, 32 (06) :76-105
[14]   State feedback based output tracking control of probabilistic Boolean networks [J].
Li, Haitao ;
Wang, Yuzhen ;
Guo, Peilian .
INFORMATION SCIENCES, 2016, 349 :1-11
[15]  
Papagiannis G., 2019, DEEP REINFORCEMENT L
[16]  
Robeva R., 2013, Mathematical concepts and methods in modern biology: using modern discrete models
[17]   Probabilistic Boolean networks: a rule-based uncertainty model for gene regulatory networks [J].
Shmulevich, I ;
Dougherty, ER ;
Kim, S ;
Zhang, W .
BIOINFORMATICS, 2002, 18 (02) :261-274
[18]  
Sootla A., 2015, TOGGLING GENETIC SWI
[19]  
Sootla A, 2013, IEEE DECIS CONTR P, P4086, DOI 10.1109/CDC.2013.6760515
[20]  
Sutton RS, 2018, ADAPT COMPUT MACH LE, P1