Convergence of Markovian stochastic approximation for Markov random fields with hidden variables

This paper studies the convergence of the stochastic algorithm of the modified Robbins- Monro form for a Markov random field (MRF), in which some of the nodes are clamped to be observed variables while the others are hidden ones. Based on the theory of stochastic approximation, we propose proper assumptions to guarantee the Holder regularity of both the update function and the solution of the Poisson equation. Under these assumptions, it is proved that the control parameter sequence is almost surely bounded and accordingly the algorithm converges to the stable point of the log-likelihood function with probability 1.