Bayesian Verification Measures for Forecasts of Continuous Predictands

The Bayesian verification theory (BVT) treats the problem of verification of forecasts from the viewpoint of a rational decision maker. From that viewpoint, a forecast provides information for making an optimal decision insofar as it reduces the prior uncertainty about a predictand and has a positive economic value. Thus before using a forecast produced by a particular system, model, or forecaster, a rational decision maker asks two questions: (i) “Can the forecast be taken at its face value?” (ii) “Does the forecast reduce the uncertainty? By how much?” The BVT provides answers by evaluating two attributes of the forecast system: calibration and informativeness.

The calibration is an attribute necessary for consistent interpretability of forecasts; it is attainable through a transformation (or recalibration) of the original forecast. The informativeness is an attribute necessary for positive economic value, regardless of the decision maker's prior distribution and loss function; it is intrinsic to the forecast system. This talk will highlight the principles of the BVT and will present Bayesian verification measures for deterministic forecast (which provides a point estimate) and probabilistic forecast (which provides a distribution function) of a continuous predictand (e.g., precipitation amount conditional on precipitation occurrence, temperature). For each type of forecast, there are two measures: a measure of calibration and a measure of informativeness. These measures are separate, independent, and sufficient. The Bayesian verification measures will be contrasted with some ad hoc measures that have been traditionally used to verify meteorological forecasts; the pitfalls of the traditional measures will be highlighted.