Fitting a reversible Markov chain by maximum likelihood: Converting an awkwardly constrained optimization problem to an unconstrained one

We consider here the problem of fitting, by maximum likelihood, a discrete-time, finite-state-space Markov chain that is required to be reversible in time. The technique we use is to impose the detailed balance and other constraints by transforming the transition probabilities to a set of unconstrained 'working parameters', after which any general-purpose routine for unconstrained numerical optimization can be used to maximize the likelihood. The main advantages of this procedure are its simplicity and its use of standard, well-tested optimizers; very little computational expertise is needed, and all computations can easily be carried out in R on a standard machine. We provide several examples of applications, to simulated and other data, and give an example of the computation of standard errors and confidence intervals for parameters. We discuss also the variation of the problem in which it is required that the transition probabilities satisfy detailed balance with respect to a pre-specified stationary distribution, demonstrate that this case can be handled by a modification of our methods, but suggest two altogether different routes that may be preferable. A feature of our methods is the availability of excellent starting-values for the optimizations. (c) 2020 Elsevier B.V. All rights reserved.