Control of a Water Tank System with Value Function Approximation

被引：0

作者：

Lalvani, Shamal ^{[1
]}

Katsaggelos, Aggelos ^{[1
]}

机构：

[1] Northwestern Univ, Evanston, IL 60208 USA

来源：

ARTIFICIAL INTELLIGENCE APPLICATIONS AND INNOVATIONS, AIAI 2023, PT I | 2023年 / 675卷

关键词：

Reinforcement Learning; Optimal Control; Water Tank System;

D O I：

10.1007/978-3-031-34111-3_4

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We consider a system of two identical rectangular shaped water tanks. A source of constant water inflow is available, which may only be directed to one tank at a time. The objective is to find a control policy to maximize the final sum of the water levels at some terminal time T, subject to minimum water level constraints on each tank. Water exits each tank corresponding to Toricelli's law (i.e., the velocity depends on the current water level). We derive a closed form dynamic programming solution in discrete time to this problem without the water-level threshold constraints. Subsequently, we implement the value iteration algorithm on a set of support points to find a control policy with the threshold constraints, where a random forest regressor is iteratively used to update the value function. Our results show consistency between the dynamic programming solution and the value iteration solution.

引用

页码：36 / 44

页数：9

共 50 条

[21] A tutorial on value function approximation for stochastic and dynamic transportation [J].

Arne Heinold .

4OR, 2024, 22 :145-173

[22] Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation [J].

Foster, Dylan J. ;

Krishnamurthy, Akshay ;

Simchi-Levi, David ;

Xu, Yunzong .

CONFERENCE ON LEARNING THEORY, VOL 178, 2022, 178

[23] A grey approximation approach to state value function in reinforcement learning [J].

Hwang, Kao-Shing ;

Chen, Yu-Jen ;

Lee, Guar-Yuan .

2007 IEEE INTERNATIONAL CONFERENCE ON INTEGRATION TECHNOLOGY, PROCEEDINGS, 2007, :379-+

[24] Controller design and value function approximation for nonlinear dynamical systems [J].

Korda, Milan ;

Henrion, Didier ;

Jones, Colin N. .

AUTOMATICA, 2016, 67 :54-66

[25] Distributed Value Function Approximation for Collaborative Multiagent Reinforcement Learning [J].

Stankovic, Milos S. ;

Beko, Marko ;

Stankovic, Srdjan S. .

IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, 2021, 8 (03) :1270-1280

[26] Modified value-function-approximation for synchronous policy iteration with single-critic configuration for nonlinear optimal control [J].

Tang, Difan ;

Chen, Lei ;

Tian, Zhao Feng ;

Hu, Eric .

INTERNATIONAL JOURNAL OF CONTROL, 2021, 94 (05) :1321-1333

[27] ON CONVERGENCE RATE OF ADAPTIVE MULTISCALE VALUE FUNCTION APPROXIMATION FOR REINFORCEMENT LEARNING [J].

Li, Tao ;

Zhu, Quanyan .

2019 IEEE 29TH INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING (MLSP), 2019,

[28] Improving Gaussian Process Value Function Approximation in Policy Gradient Algorithms [J].

Jakab, Hunor ;

Csato, Lehel .

ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2011, PT II, 2011, 6792 :221-+

[29] The Divergence of Reinforcement Learning Algorithms with Value-Iteration and Function Approximation [J].

Fairbank, Michael ;

Alonso, Eduardo .

2012 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2012,

[30] Approximation of the infinite-horizon value function of the switched LQR problem [J].

Hou, Tan ;

Li, Yuanlong ;

Lin, Zongli .

AUTOMATICA, 2024, 159

← 1 2 3 4 5 →