Why Perturbing Observations in Ensemble Kalman Filters Is Inconsistent

The initial Ensemble Kalman filter (EnKF) by Evensen in 1994 was not sampling from the full posterior distribution as a term was missing in the posterior covariance. This was corrected later, but, unfortunately, the correction was called 'perturbing observations', leading to the confusing name 'Perturbed Observation EnKF'. The scheme uses the standard Kalman Filter update equation on each ensemble member, but each of them uses a different set of observations that are perturbed by a random vector drawn from the observation error distribution.

However, perturbing observations is fundamentally incorrect. It is a case of getting the correct answer for the wrong reason, and this contribution shows that one should perturb the modelled observation to obtain a statistically consistent scheme. We show that perturbing observations is inconsistent with Bayes Theorem, which treats observations as given, not as random variables. Furthermore, based on an argument by Snyder (2015), we also also show that the Best Linear Unbiased Estimator derivation of the Kalman Filter forces us to perturb the modelled observations, not the observations themselves. Finally, a new and consistent derivation and interpretation of the Stochastic EnKF equations is given that does not perturb observations and is of relevance for both filter and variational algorithms that perturb observations.