Extended Abstract (36K)1.3
Additional Measures of Skill for Probabilistic Forecasts
F. Wesley Wilson, NCAR, Boulder, CO
The Brier Score is the established measure of skill for probabilistic forecasts. While its meaning and capabilities are well established through both tradition and some statistical viewpoints, there are other viewpoints, which reveal some limitations to the utility of the Brier Score. For example, as probabilities become nearly certain (values of 0 or 1), the Brier Score reduces to the complement of the Hit Rate, a widely disparaged measure of skill for deterministic forecasts. When applied as the Objective Function for an optimization procedure, the Brier Score often has large shallow basins, and is restricted in its ability to distinguish the optimal solution.
Through an appropriate generalization, every skill measure for deterministic forecasts can be given equivalent interpretation in the context of probabilistic forecasts. These extended measures have the pleasing mathematical property that they converge to the traditional deterministic measures as the probabilistic forecasts become more certain. In addition, they can be combined with the Brier Score, to select optimal forecasts within a Brier basin, when the Brier Score is unable to distinguish between a continuum of forecasts.
We present the methodology and provide some examples to illustrate the benefit of this approach.
Session 1, forecast evaluation
Monday, 14 January 2002, 9:30 AM-3:30 PM
Previous paper Next paper