8.4
Another look at proper scoring rules for probability forecasts
PAPER WITHDRAWN
Laurence J. Wilson, MSC, Dorval, QC, Canada
A scoring rule is said to be proper if the forecaster achieves his best score by predicting according to his true belief about the future weather. A score is strictly proper if this is the only way to achieve a best score. Most scores in common use for probability forecasts, such as the Brier and rank probability are proper, as are the continuous rank probability score and the ignorance score for probability distribution forecasts. Linear scores such as the mean absolute probability error of probability forecasts and linear probability scores for probability distributions are not proper, even though these scores are appealing in their simplicity in practice. Skill scores, though widely used, are only asymptotically proper; one must calculate the standard score over a large sample to approach proper behaviour.
Using examples of several different scores, the question of the importance of properness in the choice of verification measures is discussed. The presentation will also include some comparative tests of proper and improper score characteristics using ensemble forecast data where the forecasts have been systematically altered to try to improve the score.
Session 8, Forecast Evaluation I
Wednesday, 23 January 2008, 1:30 PM-2:30 PM, 219
Previous paper