Evaluating 11 Years of Quantitative Precipitation Forecast Performance for Extreme Events

Extreme precipitation events (i.e., events associated with the tail end of the precipitation probability distribution) are high impact events that can cause loss of life and significant disruption to local, regional, and even national economies. There are many communities (e.g., water resources management, agriculture, transportation, emergency management), which require accurate forecasts of extreme events for decision-making, preparation, and management; however, accurately forecasting such events remains one of meteorology's most difficult challenges. Since verification provides both a way to measure improvement in quantitative precipitation forecasts (QPF) and a method by which forecast errors can be identified, the Hydrometeorology Testbed (HMT) has identified QPF verification as an integral component to improving extreme QPFs.

This study examines national QPF performance for extreme events over an 11 year period (January 2001 through December 2011) using regionally defined extreme precipitation thresholds. Data for this analysis include 32-km gridded QPFs from the National Centers for Environmental Prediction's (NCEP) Hydrometeorological Prediction Center (HPC) and 4-km gridded Stage IV data from the National Weather Service (NWS) River Forecast Centers (RFC). Regional extreme precipitation thresholds were quantitatively defined as the 99th and 99.9th percentile precipitation values of all “wet-site” days (i.e., ≥ 0.01 in 24 h^-1 at each grid point) for each RFC region. Five verification metrics [probability of detection (POD), false alarm ratio (FAR), threat score, mean absolute error (MAE) and bias] were calculated by aggregating all regional extreme wet-site days. The results of these metrics were compared to the current NOAA Government Performance and Results Act (GPRA) precipitation threshold (≥ 1.0 in 24 h^-1) to determine a baseline performance.

Results from this study indicate that national 32-km extreme QPFs have improved over the last 11 years, although the yearly threat scores of the baselined extreme precipitation are approximately half of the GPRA threat scores. In addition, extreme QPF threat scores appear to be improving slightly faster (~10-15%) than the GPRA threat scores (~9%) between 2001 and 2011. Further examination has also shown that extreme precipitation amounts tend to be consistently under predicted. Seasonally, national extreme QPFs show highest skill during the winter months (i.e., December, January, February) and lower skill during the summer months (i.e., June, July, August) although a significant increase in QPF skill is observed during the month of September, most likely due to landfalling hurricanes and tropical cyclones.

A key challenge of this verification work is the smaller sample size of the extreme events, which tend to occur less frequently and over smaller areas. The results of this study provide feedback to operations at NCEP/HPC regarding extreme QPF performance for the last 11 years. Finally, the method and framework applied in this study to define and verify extreme events can be applied to any gridded dataset, and extreme QPF baseline performance can be established for that dataset.