Convective intercomparison exercise: Baseline statistical results

Each summer, convective weather is responsible for disrupting the flow of air traffic. In an attempt to mitigate this disruption, experimental forecasts, such as the Collaborative Convective Forecast Product (CCFP) and the National Convective Weather Forecast Product (NCWFP) were developed. These forecasts and those produced by the Aviation Weather Center (convective Significant Meteorological Advisories; SIGMETs) were verified in real-time from 1 June through 31 August 1999 to determine a verification baseline for convective weather, strengths and weaknesses for each of the forecasts, and to test and develop appropriate verification methods.

The various forecasts were collected in realtime and evaluated as Yes/No forecasts of convection. They were verified using individual lightning and radar observations, as well as a convective field (known as the National Convective Weather Detection Product; NCWDP) in which both lightning and radar observations were combined. For some of the analyses, these data were filtered in an attempt to screen out isolated short-lived convections. The statistical results were stratified by forecast domain, forecast length, and observation type.

This study covers a near real-time evaluation of the forecasts generated by the Real-Time Verification System (RTVS) developed at the Forecast Systems Laboratory. A web-based interface (http://www-ad.fsl.noaa.gov/afra/rtvs/RTVS-project_des.html) (including contingency tables of statistical results, time series and scatterplots, and graphical displays) was developed to provide an efficient and easy way for users to access the results in near real time.

Results of the exercise suggest that forecasting convection is difficult. However, forecasts improve when convection is associated with long-lived convective cells. The CCFP discriminates well between convective and nonconvective areas. However, the false alarms for the convective areas are large. In comparison with the convective SIGMET Outlooks, the CCFP convective area is smaller, at least partly due to the shorter valid period. Areas and detections for the NCWFP are smaller than for the other forecasts.

Plans are underway to continue this convective exercise through the summer of 2000. The exercise will again intercompare the various convective forecasts. RTVS will generate statistical displays and provide output on the Web. The verification methods will be enhanced to allow a more thorough evaluation of coverage and probability forecasts.

Verification results from this exercise will be presented. The verification exercise was funded by the Federal Aviation Administration (FAA) Aviation Weather Research Program (AWRP).