A Verification Approach Suitable for Assessing the Quality of Model-Based Precipitation Forecasts during Extreme Precipitation Events

Accurate forecasts of precipitation are exceedingly important to the public. Emergency management teams rely on these forecasts to plan when and where disasters might strike, and the general public is concerned about whether precipitation will impact their many outdoor activities, particularly in situations when heavy precipitation is possible. In either case, accurate precipitation forecasts are largely dependent upon numerical model predictions. Therefore, a verification approach which assesses a model's ability to accurately predict precipitation at specific locations, and over relatively short (~3h) time periods, has been developed. Although numerical models are not prepared to directly address the problem of point-specific precipitation forecasting, this stringent verification approach will serve to track the progress of models over time as they improve to meet this high expectation of the public.

The Real-Time Verification System (RTVS), developed at the Forecast Systems Laboratory and funded by the Federal Aviation Administration's Aviation Weather Research Program, has been enhanced to include verification of precipitation forecasts. Currently within RTVS, precipitation forecasts from the National Centers for Environmental Prediction's Rapid Update Cycle (RUC-2) and Eta models are verified using hourly precipitation gauge data (subject to quality control screening at River Forecast Centers and at FSL) from the National Weather Service Hydrometeorological Automated Data System (HADS). In near real-time, the model forecasts at specific precipitation thresholds and accumulation periods are interpolated to the quality controlled precipitation observation points. A record is logged into a two-by-two contingency table used to compute standard statistical measures such as Equitable Skill Score (ESS) and Bias. Users can access these statistical results via the RTVS web-based interface (http://www-ad.fsl.noaa.gov/afra/rtvs/RTVS-project_des.html.

Preliminary results from 1 January 2000 - present show that the quality of precipitation forecasts produced by Eta (measured by ESS) are generally better than those produced by RUC-2 at all model initialization and lead times; however, Eta consistently overforecasts precipitation at smaller accumulation thresholds, while RUC-2 consistently underforecasts at all thresholds. Unexpectedly, RUC-2 improves drastically, relative to Eta, at forecasting all accumulation amounts when initialized at 0600 UTC. This result is likely due to the inclusion of initialization data not available to Eta, or possibly due to decreased convective activity for forecasts generated at this hour.

Statistical results from RTVS for RUC-2 and Eta will be presented. Highlights will include, variations in the models due to model initialization and lead times, seasonal dependencies, diurnal effects, and model-spin up problems.