Evaluation of Experimental Forecasts from the 2009 NOAA Hazardous Weather Testbed Spring Experiment Using Both Traditional and Spatial Methods

NOAA Storm Prediction Center (SPC) and National Severe Storms Laboratory (NSSL) have a collaborative testbed facility called the Hazardous Weather Testbed. The NOAA Hazardous Weather Testbed (HWT) has conducted Spring Experiments since 2000. The 2009 Spring Experiment took place over a 5 week period from 4 May – 5 June. A main focus of recent Spring Experiments is to gain an understanding of how to better use the output of near-cloud resolving configurations of numerical models to predict convective storms. HWT Spring Experiment participants have found through subjective evaluation that high resolution convective storm predictions are at times difficult for operational forecasters to reconcile, in part because many solutions appear to be plausible for a given mesoscale environment

The Model Evaluation Tools (MET), developed by the Development Testbed Center (DTC), was used in 2009 to help evaluate performance of three models during the Spring Experiment. Four important goals of these evaluations have been (i) to provide objective evaluations of the experimental forecasts; (ii) to supplement and compare to subjective assessments of performance; (iii) to expose the forecasters and researchers to both new and traditional approaches for evaluating precipitation forecasts; and (iv) to evaluate the impact of radar data assimilation on forecast skill. Two tools from the MET system were used for the 2009 experiment, Grid-Stat and MODE. Grid-Stat provides traditional verification methods for gridded data sets, while MODE offers a spatial, or object-oriented, approach to verification. A select set of statistics were available to the forecasters on a daily basis via a web interface.

The goal of the 2009 objective evaluation was to assess the impact of radar assimilation on the forecasts of strong convection. Two forecasts came from the Center for Analysis and Prediction of Storms (CAPS) and are WRF-ARW members of the 4km Storm Scale Ensemble Forecasting. The third forecast was the NOAA/Earth System Research Lab (ESRL) High Resolution Rapid Refresh (HRRR) 3-km model, which is also based on the WRF-ARW core. Twelve hour forecasts of accumulated precipitation and reflectivity from both the 00 UTC and 12 UTC initializations were included in the evaluation.

Generally speaking, the objective evaluation provided by MET supported the subjective evaluation performed by the forecasters. Preliminary results for the two CAPS ensemble members have been aggregated over the 5 week experiment period. Technical difficulties limited the number of complete HRRR runs received and hence these data will only be used in a case study mode. In this paper, the results from both the traditional and spatial methods will be presented, compared, and contrasted.