Observation Informed Generalized Hybrid Error Covariance Models

Because of imperfections in ensemble data assimilation schemes, one cannot assume that the ensemble derived covariance matrix is equal to the true error covariance matrix. Here, we describe a simple and intuitively compelling method to fit calibration functions of the ensemble sample variance to the mean of the distribution of true error variances given an ensemble sample variance. Once the calibration function has been fitted, it can be combined with ensemble-based and climatologically based error correlation information to obtain a generalized hybrid error covariance model. When the calibration function is chosen to be a linear function of the ensemble variance, the generalized hybrid error covariance model is the widely used linear hybrid consisting of a weighted linear sum of a climatological and an ensemble-based forecast error covariance matrix. However, when the calibration function is chosen to be, say, a cubic function of the ensemble sample variance, the generalized hybrid error covariance model is a non-linear function of the ensemble sample estimate. Consistent with earlier work, it is shown that the linear hybrid is optimal in the case where the climatological distribution of true forecast error variances is an inverse-gamma probability density function (pdf) and the distribution of ensemble sample variances is a gamma pdf. However, when these conditions are not met, the mean of the distribution of true error variances given an ensemble sample variance will, in general, be a non-linear function of the ensemble sample variance. To aid understanding, a hierarchy of cases in which the generalized hybrid outperforms the linear hybrid are given. It is shown that in the case of the Lorenz ’96 model, data assimilation performance is improved considerably by using the generalized hybrid instead of the linear hybrid.