Hidden Error Variance Theory: Application to Ensemble Post-Processing

The square of a single error poorly approximates error variance. If one could find numerous historical forecasts like today's forecast, the variance of the associated forecast errors would approximate the forecast error variance given today's forecast. However, chaos makes it difficult to obtain relevant similar forecasts. In such situations the error variance is not directly observable. It is hidden. Ensemble forecasts provide imperfect estimates of these hidden error variances. Recent developments in hidden error variance theory allow estimates of (a) the climatological distribution of error variances (b) the relative error variance of the ensemble variance estimate of the error variance in terms of an “effective ensemble size”, and (c) the distribution of error variances given an imperfect ensemble variance to be directly estimated from time series of (ensemble variance, error) data pairs. “Straw man” experiments are used to assess the value of this knowledge to ensemble post-processing. Rank Frequency Histograms (RFH) and the effective daily interest rates associated with a hypothetical game of weather roulette are used to quantify the value of accurate information about the distribution of error variances given the ensemble variance. It is found that ensemble post-processing schemes that utilize the full distribution of error variances given the ensemble sample variance outperform those that do not. Specifically, when the ensemble accounts for the varying error variance, we obtain a flat RFH and an optimal effective daily interest rate. It is also found that probabilistic scores differ markedly in their sensitivity to the ability of an ensemble to accurately track flow dependent changes in error variance. For example, it is found that the Brier score is insensitive to such inaccuracies whereas “effective daily interest rate” and “effective ensemble size” are very sensitive to this type of inaccuracy.