Hierarchical optimal control for input-affine nonlinear systems through the formulation of Stackelberg game

Substantial efforts have been undertaken to explore nonzero-sum differential games. Most of these studies are devoted to devising algorithms to pursue Nash equilibrium, where all players with the same access to information will take policies synchronously. However, when it comes to hierarchical optimization and asymmetric information, Nash equilibrium is ineffective. The Stackelberg game provides us with an idea of leader-follower strategy to cope with this conundrum. The paper investigates hierarchical optimal control for continuous-time two-player input-affine systems characterized by nonlinear dynamics and quadratic cost functions. By introducing new costates, this optimization problem is formulated as a Stackelberg game in conjunction with a parametric optimization problem. Besides, the closed-loop information is available for both players. An adaptive learning algorithm is thus developed to approximately obtain the open-loop Stackelberg equilibrium while ensuring the uniform ultimate bounded stability of this closed-loop system, and two approximators structured by neural networks put this purpose into practice. Finally, two numerical examples illustrate that the proposed methodology can accurately obtain optimal solutions, and a comparative example illustrates its characteristics. (C) 2020 Elsevier Inc. All rights reserved.