A MONTE-CARLO METHOD FOR SENSITIVITY ANALYSIS AND PARAMETRIC OPTIMIZATION OF NONLINEAR STOCHASTIC-SYSTEMS - THE ERGODIC CASE

For high-dimensional or nonlinear problems there are serious limitations on the power of available computational methods for the optimization or parametric optimization of stochastic systems of diffusion type. The paper develops an effective Monte Carlo method for obtaining good estimators of systems sensitivities with respect to system parameters, when the system is of interest over a long period of time. The value of the method is borne out by numerical experiments, and the computational requirements are favorable with respect to competing methods when the dimension is high or the nonlinearities "severe." The method is a type of "derivative of likelihood ratio" method. For a wide class of problems, the cost function or dynamics need not be smooth in the state variables; for example, where the cost is the probability of an event or "sign" functions appear in the dynamics. Under appropriate conditions, it is shown that the invariant measures are differentiable with respect to the parameters. Since the basic diffusion (or other) model cannot be simulated exactly, simulatable approximations are discussed in detail, and estimators of the derivatives of the cost functions for these approximations are obtained and analyzed. It is shown that these estimators and their expectations converge to those for the original problem. Thus, we prove a robustness result for the sensitivity estimators, namely that the derivatives of the ergodic cost functions (and their estimators) for the simulatable approximations converge to those for the approximated process. Such results are essential if a simulation based method is to be used with confidence.