The relative operating characteristic (ROC) curve measures the ability of a probabilistic or categorical forecasting system to discriminate between situations preceding the occurrence and the non-occurrence of an event of interest. It has been applied frequently, for example, to the assessment of the ability of ensemble forecast systems to discriminate between the occurrence and non-occurrence of precipitation accumulations over specific thresholds. The ROC can be applied to any set of probabilistic forecasts of a dichotomous variable, from any source.

We have been experimenting with the ROC for verification of statistical forecasts of lightning occurrence at high spatial resolution. In this application, both the forecasts and the verifying observations are available with a spatial resolution of 25 km or better, and both the forecasts and the observations tend to show considerable spatial variability on these scales. By calculating the ROC for individual cases, the spatial discriminant ability of the forecasts has been characterized. The assessment was done for different scales by recalculating the score over successively larger grid boxes within the domain of the forecast. It is possible also to match forecast and observed patterns within boxes by moving forecast and observed boxes with respect to each other, and calculating the squared distance over all points within the box (the Brier score) and determining the translation which minimizes the score. Such analysis should determine whether there is a spatial bias in the forecasts at any selected scale.

The paper will present verification results for ranges of 0 to 48-h, for single cases and averaged over larger samples of cases.