Innovation-based Methods for Estimating Observation Uncertainty During Data Assimilation

Most data assimilation schemes currently used for research or operational prediction rely on an accurate characterization of observation error statistics. When this assumption is not well met, observations will be assigned too much or too little weight during data assimilation, resulting in a degraded analysis. For some data assimilation methods, including variants of the Kalman Filter, imperfect characterization of observation error statistics also leads to errors in estimating the analysis uncertainty itself. To address this issue, we examine three methods for estimating the second moment (i.e. the variance) of the distribution of observation errors based on innovation statistics. Two of the three methods, including the widely used Desroziers “diagnostic check,” generate point estimates of the variance, but differ in what information they use to separate the background variance from the total innovation variance. For the third method, we present an approximate Bayesian scheme with an inverse gamma prior and Gaussian likelihood function. Using low-dimensional models, we compare the performance of each method for a range of applications corresponding to possible real-world data assimilation challenges. Our experiments include initial over/underspecification of the variance, non-Gaussianity of the true observation error distribution, nonlinearity in the observation operator h, and bias in the observation error. Finally, we consider future directions toward the development of fully nonparametric observation uncertainty quantification methods that can cope with non-Gaussian and state-dependent observation errors.