Extremum Seeking With Stochastic Perturbations

Extremum seeking (ES) using deterministic periodic perturbations has been an effective method for non-model based real time optimization when only limited plant knowledge is available. However, periodicity can naturally lead to predictability which is undesirable in some tracking applications and unrepresentative of biological optimization processes such as bacterial chemotaxis. With this in mind, it is useful to investigate employing stochastic perturbations in the context of a typical ES architecture, and to compare the approach with existing stochastic optimization techniques. In this work, we show that convergence towards the extremum of a static map can be guaranteed with a stochastic ES algorithm, and quantify the behavior of a system with Gaussian-distributed perturbations at the extremum in terms of the ES constants and map parameters. We then examine the closed loop system when actuator dynamics are included, as the separation of time scales between the perturbation signal and plant dynamics recommended in periodic ES schemes cannot be guaranteed with stochastic perturbations. Consequently, we investigate how actuator dynamics influence the allowable range of ES parameters and necessitate changes in the closed loop structure. Finally simulation results are presented to demonstrate convergence and to validate predicted behavior about the extremum. For the sake of analogy with the classical methods of stochastic approximation, stochastic ES in this technical note is pursued in discrete time.