Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes

Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes are analyzed in this paper. The analysis is carried out for the algorithms with additive noise, as well as for the algorithms with non-additive noise. The algorithms with additive noise are considered for the case where the noise is state-dependent and admits the decomposition as a sum of a martingale difference sequence and a telescoping sequence. The algorithms with non-additive noise are analyzed for the case where the noise satisfies uniform or strong mixing conditions, as well as for the case where the noise is a Markov chain controlled by the algorithm states.