THE RELATIONSHIP BETWEEN THE MAXIMUM PRINCIPLE AND DYNAMIC-PROGRAMMING

Let V(t,x) be the infimum cost of an optimal control problem, viewed as a function of the initial time and state (t,x). Dynamic programming is concerned with the properties of V and in particular with its characterization as a solution to the Hamilton-Jacobi-Bellman equation. Heuristic arguments have long been advanced relating the maximum principle to dynamic programming according to p(t) equals minus V//x(t,x//0(t)). Here x//0 is the minimizing state function under consideration and p is the costate function of the maximum principle. In this paper we examine the validity of such claims and find that this relationship, interpreted as a differential inclusion involving the generalized gradient, is indeed true, almost everywhere and at the endpoints, for a very large class of nonsmooth optimal control problems.