Solving Hamilton-Jacobi-Bellman equations by a modified method of characteristics

The numerical solution of Hamilton-Jacobi-Bellman (HJB) equations is derived by viscosity approximation. The first-order hyperbolic HJB equation is perturbed as a convection-diffusion equation by adding a diffusion term with a small diffusion coefficient/viscosity and is solved by a modified method of characteristics (MMOC) in time and a finite difference in state space. An algorithm is designed to decouple the value and control functions. Numerical results show that MMOC is efficient for solving non-trivial HJB equations and that the transition layer parameters a and b are dimension-independent. Thus, the method is promising for solving real-world optimal control problems via HJB equations.