Diagnosis and Optimization of Ensemble Forecasts

In the current study we present a technique which unifies evaluation of the forecast uncertainties produced either by initial conditions or different model versions, or both. The technique consists of first diagnosing the performance of the forecast ensemble which is based on explicit use of weather analysis uncertainties, and then optimizing the ensemble forecast using results of the diagnosis. The technique includes explicit evaluation of probabilities which are associated with the Gaussian stochastic representation of the analysis and forecast. It combines the technique for evaluating the analysis error covariance that was first presented in the Ensemble Transform Data Assimilation method and the standard Monte Carlo approach for computing samples from the known Gaussian distribution. We applied this technique to two relatively simple forecast systems to evaluate ensemble characteristics including ensemble size, various observation strategies, and configurations including model versions and initial conditions and to test improvements in the ensemble forecasts by optimal weighting of the ensemble members which was based on the time varying prior probabilistic skill measure. For the observation scenarios the results show differences in the characteristics of the analysis pdf that narrowed for dense observation spacing and broadened for sparse observations. The finding here was, as observations become more dense there is need for larger ensembles and/or individual members must be more accurate for the ensemble forecast to exhibit prediction skill. The main findings relative to the different physics configurations were that almost all physics members typically exhibited some skill at some point in the model run, suggesting that all should be retained for the best ensemble forecast and that the normalized probability metric can be used to determine what sets of weights or physics configurations are performing best. Comparing ensemble forecasts with a simple ensemble mean with a mean developed from variably weighting ensemble members based on prior performance by the probabilistic measure was shown to substantially reduce mean absolute error. The study indicates that a weighting scheme that utilized more prior cycles showed additional reduction in forecast error.