A critical challenge in evaluating the quality of these forecasts is determining how to appropriately match the forecasts to the observations so that statistical results are representative of the forecast characteristics, the forecast spatial and temporal scales, and portray the forecast’s operational relevance. An extensive evaluation of several approaches for defining the observation field and assessing the quality of the convective forecasts reveals important differences among these approaches.
The first approach to be tested is a standard deterministic method where thresholds are applied to the forecasts and observations to indicate a Yes or No convective event. The forecasts are matched with the observations and the result is logged on the standard 2x2 contingency table. The results from this method are used to baseline the other approaches. The second method is based on developing an observed field of convective coverage that relies on the coverage of convective activity occurring within a predefined area of influence. The coverage map is then compared to the forecasted coverage to obtain the matched pairs. A third approach involves smoothing and filtering the observations to extract the forecast scale from the observed field, then matching the filtered field to the forecast field to generate the forecast/observation pairs.
Statistical results from application of each of the three approaches are described and the strengths and weaknesses of each approach are documented. Special consideration is given to applications of these techniques to probabilistic forecasts.
Supplementary URL: