New verification approaches for convective weather forecasts

Over the last several years, new methods to evaluate the skill of convective and precipitation forecasts have begun to emerge. In fact, development of alternative verification approaches for these forecasts has been an important focus of research in the verification community for the last several years, at least in part because there is a desire to obtain information that is more meaningful in an operational context than the information that can be obtained from traditional grid-based verification approaches. The new approaches will make it possible to diagnose the specific sources of errors in forecasts.

This paper reviews the status of new verification approaches for convective forecasts – including operational forecasts such as the convective SIGMETs (C-SIGMETs) and the Collaborative Convective Forecast Product (CCFP), as well as automated forecasts such as the National Convective Weather Forecast (NCWF) and the new version of the NCWF (NCWF-2), which provides probabilistic forecasts of convective weather. The techniques also can be used to verify precipitation and convective forecasts produced by numerical weather prediction models, such as the Weather Research and Forecasting (WRF) model and the Rapid Update Cycle (RUC) modeling system.

A specific approach applied to C-SIGMETs and CCFP forecasts is demonstrated. This approach optimizes the location and orientation of the forecasts, relative to the observed field, and compares the optimized forecasts to the original forecasts. A more complex approach, called “object-oriented” verification, is applied to the NCWF-2 and to precipitation fields. Two approaches are described for matching forecast and observed objects, one based on the application of fuzzy logic, and the second based on a binary image comparison approach. The additional complication of verifying probabilistic forecasts is also considered. An issue that is fundamental to all verification problems – the definition of the verifying observation field – is considered briefly, but is treated more completely in a paper by Mahoney et al. Development and interpretation of operationally meaningful verification measures is demonstrated using data from summers 2003 and 2004.