Markov Chain Monte Carlo Solution of the Implicit Nonlinear Inverse Problem with Application to Curve Fitting and Filter Estimation

We adapt the Metropolis-Hastings (MH) algorithm to facilitate construction of the ensemble solution of the nonlinear implicit inverse problem. The solution variable is the aggregation of the parameters of interest (model parameters) and the data. The prior probability density function (pdf) is the possibly-non-Normal joint pdf of the prior model parameters and the noisy data, and is defined in a high-dimensional space. The posterior pdf of the solution (estimated model parameter and predicted data) is the prior pdf evaluated on the lower-dimensional manifold defined by the theory. We adapt the MH algorithm to ensure that successors always satisfy the theory (that is, are on the manifold) and provide a rule for computing the probability of a given successor. Key parts of this adaption are the use of singular value decomposition to identify subspaces tangent to the manifold, and orthogonal projection, to move a preliminary estimate of a successor onto the manifold. We apply the adapted methodology to three exemplary problems: fitting a straight line to (x,y) data, when both x and y have measurement noise; fitting a circle to noisy (x,y) data, and finding a filter that takes one noisy time series into another. In these cases, the scatter of the ensemble solution about the linearized maximum likelihood solution is roughly consistent with the linearized posterior covariance, but with some non-Normal behavior. We demonstrate the usefulness of the ensemble solutions by computing empirical pdfs of several informative statistical parameters, the calculation of which would be difficult by traditional means.