The GIGG-EnKF: Ensemble Kalman Filtering for highly skewed non-negative uncertainty distributions

Observations and predictions of near-zero non-negative variables such as aerosols, water vapor, cloud, precipitation and plankton concentrations have uncertainty distributions that are skewed and are often better approximated by gamma and inverse-gamma probability distribution functions (pdfs) than Gaussian pdfs. These non-Gaussian variables play a critical role in weather and climate. Current Ensemble Kalman Filters (EnKFs) and variational data assimilation schemes yield poor probabilistic state estimates in the presence of these non-Gaussian, highly skewed uncertainty distributions. Here, we introduce a variation on the EnKF that solves Bayes' theorem with a high degree of accuracy in univariate cases where the prior forecasts and error prone observations given truth come in (gamma, inverse-gamma) or (inverse-gamma, gamma) or (Gaussian, Gaussian) distribution pairs. Since the overall structure of its multivariate form is identical to the perturbed observations EnKF, we call this variation the GIGG-EnKF or GIGG where GIGG stands for Gamma, Inverse-Gamma and Gaussian. In the special case that all observations are treated as Gaussian, the GIGG-EnKF gives identical results to a perturbed observations EnKF. A multi-grid-point and multi-variable idealized system was used to compare and contrast the data assimilation performance of the GIGG with that of both the perturbed observation and deterministic forms of the EnKF. This test system featured variables and observation types whose uncertainty distributions approximate Gaussian, gamma and inverse-gamma distributions. The normalized analysis error variance of the GIGG ensemble mean was found to be significantly smaller than those of the ETKFs. The higher moments of the analyzed ensemble distributions were tested by subjecting the ensemble members to non-linear “forecast” mappings. The normalized mean square error of the mean of the corresponding GIGG forecast ensemble was found to be less than a 3rd of that obtained from either form of the EnKF.