A Comparative Study of Innovation-based Methods for Estimating Observation Uncertainty During Data Assimilation

Most data assimilation schemes currently used for research or operational prediction require properly specified error statistics for observations in order to perform optimally. When this assumption is not well met, observations will be assigned too much or too little weight during data assimilation, resulting in a degraded analysis. To address this issue, we examine three methods for estimating the distribution of observation errors, focusing specifically on the case of modeling observation error as Gaussian (suitable for a number of data assimilation applications) with zero mean and unknown variance. Two of the three methods generate point estimates of the variance, while the last provides a Bayesian estimate of the variance that inherently comes with a degree of quantified uncertainty. All of the methods leverage statistics generated during data assimilation to create a running estimate of the variance that is meant to converge to the true value over time. These estimates can then be immediately used during later assimilation. Using low-dimensional models, we compare the performance of each method for a range of applications corresponding to possible real-world data assimilation challenges. Our experiments include initial over/underspecification of the variance, regime-dependent observation errors, estimation in the presence of model error, and non-Gaussianity of the true observation error distribution. This research aims to inform the future development of flexible data-driven observation uncertainty quantification methods that can cope with state-dependent non-Gaussian observation errors.