A Report and Feature-based Verification Study of the CAPS 2008 Storm-Scale Ensemble Forecasts for Severe Convective Weather

During spring 2008, the Center for Analysis and Prediction of Storms (CAPS) ran realtime 10-member storm-scale ensemble forecasts (SSEF) at 4-km grid length for a nearly-CONUS domain as part of the NOAA Hazardous Weather Testbed (HWT) Spring Experiment. Because of a high-level of uncertainty with numerical forecasts of localized, severe, high-impact weather, probabilistic forecast guidance that can be derived from ensemble forecasts is considered very valuable. The ensemble systems allow the quantification of the forecast uncertainty, and for discrete events provide a range of timing, location, and magnitude of convective weather.

Traditional field-based verification techniques have many problems with localized, discrete features. Feature-based verification can be much more informative on the quality of forecasts. Feature-based verification techniques have so far been mostly applied to deterministic forecast verification. In this study, we develop and apply a set of verification procedures that utilize several diagnostic parameters derived from the ensemble and local severe storm reports. Hail, wind, and tornado reports are used as the verification data whose probabilistic relations with model-derived features, including the updraft helicity, maximum column vertical velocity, and 4 km AGL simulated reflectivity, are sought. Probabilistic forecasts are extracted from the ensemble using three methodologies: traditional ensemble-frequency-based probability, fractional coverage probability and a combination of the two. The skill, reliability, and bias scores of probabilistic forecasts of the features are compared for each of these techniques. Various thresholds of the forecast parameters and neighborhood radii are examined to determine those giving the most skillful and reliable forecasts of the severe weather events.