Accounting for Varying Variances in Ensemble Post-Processing

Ensemble variances provide a prediction of the flow dependent error variance of the ensemble mean or, possibly, a high resolution forecast. However, small ensemble size, unaccounted for model error, and imperfections in ensemble generation schemes cause the predictions of error variance to be imperfect. In previous work, the authors developed an analytic approximation to the posterior distribution of error variances, given an imperfect ensemble prediction, based on parameters recovered from long archives of innovation and ensemble variance pairs. Here, we introduce a method by which information from the estimated posterior distribution of error variances is used to post-process ensemble forecasts. Specifically, this method of post-processing allows climatological information to be incorporated in the forecast in a way that takes advantage of the best available estimate of the distribution of true error variances given an ensemble variance. A hierarchy of post-processing methods are described, each graded on the amount of information about the posterior distribution of error variances used in the post-processing. These “Straw man” ensembles are used to assess the value of knowledge of the mean and variance of the posterior distribution of error variances to ensemble post-processing and explore sensitivity to various parameter regimes. Testing was performed using both synthetic and operational ensemble forecasts. Rank Frequency Histograms (RFH) and the effective daily interest rates associated with a hypothetical game of weather roulette are used to quantify the value of accurate information about the distribution of error variances given the ensemble variance. It is found that ensemble post-processing schemes that utilize the full distribution of error variances given the ensemble sample variance outperform those that do not. Specifically, when the ensemble accounts for varying error variance, we obtain a flat RFH and an optimal effective daily interest rate. It was found that the value of accounting for the full distribution of variances increases as the correlation between raw ensemble variance and error variance decreases and as the relative variance of the climatological distribution of error variances increases.