Improving Particle Filter Performance by Smoothing Observations in a Multiscale Lorenz-96 Model

The ensemble Kalman filter and its many variants are leading methods for meteorological data assimilation. It operates on the basis of an assumption of Gaussianity, which results in errors when the actual forecast or observation distributions are non-Gaussian. Particle filters are a class of methods that make no assumption of Gaussianity, but are typically ineffective at practically-affordable ensemble sizes. A variety of methods exist for mitigating the non-Gaussian errors of the ensemble Kalman filters, and for reducing the required ensemble size in particle filters. We here investigate a new method for improving particle filter performance in a multiscale Lorenz-96 model. The method is based on smoothing observations before assimilating them with a particle filter under the assumption that the errors in the smoothed observations are independent. This has the effect of inflating observation errors at small scales, and limiting the effective dimension of the problem by focusing on large scales. The method is not, by itself, sufficient to render the particle filter practical, so it is hybridized with an ensemble Kalman filter. The connection to the true Bayesian posterior distribution is maintained as follows. First, observation errors are Gaussian and have much less uncertainty than the forecast, so that the posterior (the analysis) is approximately Gaussian. Hybridization is achieved by splitting the likelihood into a product. The particle filter acts first, producing an intermediate distribution that is much closer to Gaussian than the forecast; next, the ensemble Kalman filter is applied. The assumption of Gaussianity is much more accurate after the particle filter step, which makes the subsequent application of the ensemble Kalman filter more accurate. The dynamics are not stochastic, so a mean preserving random rotation is applied to the posterior (analysis) ensemble to break particle degeneracy. Results are compared for the pure ensemble Kalman filter and the hybrid.