Square Root and Perturbed Observation Ensemble Generation for Kalman and Quadratic Ensemble Filters

Ensemble-based Kalman Filter (EnKF) algorithms have proven highly effective in a wide-variety of applications in the geosciences. However, the application of the EnKF to the highly nonlinear physical processes found in geoscience applications may lead in certain situations to non-Gaussian prior distributions and therefore suboptimal or even degenerative behavior. A new extension of the EnKF will be described here that allows for the explicit effects of skewness in the prior distribution to be accounted for in the data assimilation algorithm. The algorithm operates as a global-solve (all observations are considered at once) using a minimization technique and Schur/Hadamard (element-wise) localization. The central feature of this technique is the squaring of the innovation and the ensemble perturbations so as to create an extended state-space that accounts for the second, third and fourth moments of the prior distribution. Both square-root and perturbed observation ensemble generation techniques will be shown to be implementable within the new framework and will be compared against the corresponding technique within the traditional EnKF. These new techniques will be illustrated on the Lorenz (1963) attractor as well as in a Boussinesq model of O(10000) variables configured to simulate nonlinear waves in shear flow. It is shown that ensemble sizes of at least 100 members are needed to adequately resolve the third and fourth moments required for the algorithm. For ensembles of this size it is shown that these new techniques are superior to the EnKF in situations with significant skewness, otherwise the new algorithms reduce to the performance of the EnKF.