4.1
Calibrated probabilistic mesoscale forecasting with ensembles via Bayesian model averaging
Adrian E. Raftery, University of Washington, Seattle, WA; and F. Balabdaoui and T. Gneiting
We consider the problem of calibrated probabilistic mesoscale forecasting of a single future meteorological quantity. We define this as the specification of a probability distribution of the quantity of interest which is both calibrated and sharp. By calibrated, we mean that if we define a probability interval, such as a 90% probability interval, then on average in the long run, 90% of such intervals contain the true value. By sharp, we mean that the distribution is more concentrated than forecast distributions from climatology alone.
Mass and colleagues have been developing an ensemble mesoscale forecasting system based on runs of MM5 initialized using different global models. They have established a clear spread-skill relationship, but the resulting forecast intervals are not calibrated; they are too narrow.
We apply Bayesian Model Averaging (BMA; Hoeting et al, 1999, Statistical Science), to develop calibrated probability forecasts using the same ensemble that underlies the Mass system. Bayesian Model Averaging is a formal statistical framework for combining probability forecasts from competing models in a calibrated way, taking account of the models' past forecasting performance. The theory of Bayesian Model Averaging explains and predicts both of the main empirical findings from the Mass group: the spread-skill relationship, and the fact that the intervals from the Mass ensemble are too narrow on average.
We calculate BMA forecasts of temperature 48 hours ahead in winter in the Puget Sound region. The resulting forecasts are well calibrated and are also substantially sharper than calibrated forecasts from climatology. These preliminary results suggest that BMA is successful at providing probabilistic forecasts that are both calibrated and sharp. We will mention plans to optimize various aspects of the method's implementation, and to incorporate it in the UW online ensemble system.
Session 4, Bayesian Probability Forecasting (Room 602/603)
Wednesday, 14 January 2004, 8:30 AM-9:30 AM, Room 602/603
Next paper