The GIGG-EnKF: ensemble Kalman filtering for highly skewed non-negative uncertainty distributions

Observations and predictions of near-zero non-negative variables such as aerosol, water vapour, cloud, precipitation and plankton concentrations have uncertainty distributions that are skewed and better approximated by gamma and inverse-gamma probability distribution functions (pdfs) than Gaussian pdfs. Current Ensemble Kalman Filters (EnKFs) yield suboptimal state estimates for these variables. Here, we introduce a variation on the EnKF that accurately solves Bayes' theorem in univariate cases where the prior forecasts and error-prone observations given truth come in (gamma, inverse-gamma) or (inverse-gamma, gamma) or (Gaussian, Gaussian) distribution pairs. Since its multivariate extension is similar to an EnKF, we refer to it as the GIGG-EnKF or GIGG where GIGG stands for Gamma, Inverse-Gamma and Gaussian. The GIGG-EnKF enables near-zero semi-positive-definite variables with highly skewed uncertainty distributions to be assimilated without the need for observation bias inducing log-normal or Gaussian anamorphosis nonlinear transformations. In the special case that all observations are treated as Gaussian, the GIGG-EnKF gives identical results to the original EnKF. A multi-grid-point and multi-variable idealized system was used to compare and contrast the data assimilation performance of the GIGG with that of both the perturbed observation and deterministic forms of the EnKF. This test system featured variables and observation types whose uncertainty distributions approximate Gaussian, gamma and inverse-gamma distributions. The normalized analysis error variance of the GIGG ensemble mean was found to be significantly smaller than that of the EnKFs. The higher moments of the analysed ensemble distributions were tested by subjecting the ensemble members to nonlinear forecast' mappings. The normalized mean square error of the mean of the corresponding GIGG forecast ensemble was found to be less than a 3rd of that obtained from either form of the original EnKF.