Adaptive estimation of background and observation error statistics in variational data assimilation

Variational data assimilation systems aim at providing an accurate estimate of the current state of the atmosphere by minimizing a cost function that measures the distance to the background state and a set of observations. The background error covariance matrix B and observational error covariance matrix R, which must be specified, play a very important role, however neither are well known. Generally, samples of background error can be obtained by using the so-called NMC method or a Monte Carlo approach such as the Ensemble Kalman Filter. However, both approaches suffer from some deficiencies. The matrix R is generally assumed to be diagonal with the variances estimated in an ad hoc way from innovation variances. Consequently a new tuning of both error covariance matrices may lead to improvements in the analysis system. In this study, a method proposed by Desroziers and Ivanov (2001) is used to tune the background and observational error statistics and several experiments were carried out to evaluate the impact of the tuning using various diagnostics. The idea of the method is based on an optimality criterion given by Talagrand (1999) concerning the statistical properties of the cost function at the minimum. The theoretical expected value of various components of the cost function can be computed and compared against the computed values. Discrepancies between the theoretical and computed values of a given cost function component are attributed to the misspecification of the corresponding error variances. These error variances are then tuned to reduce the discrepancy. Advantages of this method include: (1) allows separate tuning coefficients to be computed simultaneously for the background and observational error statistics; (2) for the observation component, the tuning coefficients can be divided into subsets (per instrument type and pressure level) due to the assumption of uncorrelated observation error; and (3) the tuning procedure uses the existing analysis system with only minor modifications. Several experiments were carried out by using the 3D-Var data assimilation system at the Canadian Meteorological Centre.