Tuesday, 13 January 2004: 1:45 PM
Examining the sensitivity of various performance measures
Room 3A
Performance measures, such as the critical success index
(CSI or threat score), are widely used as summary measures
of forecast quality. However, the sensitivity of these
measures to aspects of forecast quality is not well
understood. In this paper, we will examine the sensitivity
of several measures of forecast quality, such as the
CSI, equitable threat score, and true skill statistic,
to changes in bias error for a hypothetical forecasting
situation. The forecast situation is modelled through
circular areas of observed and forecast events, such as
precipitation greater than a threshold. The circles are
separated by a given distance, and the radius of the
observed circle is held fixed. The bias error is modelled
by varying the radius of the forecast circle, and the
aforementioned performance measures are computed. The
sensitivity of these measures to bias error will be
presented at the conference along with a discussion on
the implications for interpreting verification information
using these measures of performance.
Supplementary URL: