Geospatial verification of experimental severe weather warnings

NWS severe thunderstorm and tornado warnings are issued as recently-adopted “storm-based” polygons that cover a two-dimensional area of expected threat of severe weather within a specific time period up to 60 minutes. The entire spatial and temporal extent of a warning polygon is verified when at least one ground truth report of severe weather occurs within the valid time period of the warning. The time difference between the time of the first severe weather report within the valid time and area of the warning and the issuance time of the warning is calculated as the lead time. Any warning polygons without a matching report of severe weather is considered a false alarm, and any severe weather report without a warning is deemed a missed event. A single point in space and time can be used to verify an areal polygon warning.

This paper will explore alternative methods for assessing the goodness of NWS severe thunderstorm and tornado warnings. One motivation behind this study was to evaluate the results of springtime experiments at the NOAA Hazardous Weather Testbed (HWT). During real-time severe weather events in the HWT, visiting NWS forecasters issued experimental warnings while evaluating multiple-radar and multiple sensor (MRMS) severe weather applications developed by the National Severe Storms Laboratory. To get a more robust picture of warning goodness, verification data sets were created on a 1-km grid by using a combination of actual reports (which are sparse) and MRMS radar proxies for hail and mesocyclone/tornado paths. Hail verification grids are bias-corrected with actual reports. Mesocyclone/tornado paths, determined from ground truth and MRMS data, are converted to verification grids with varying degrees of “splatting” to allow for a degree of spatial tolerance for tornado warnings. After converting the warning polygons to 1 km grids, we can calculate hits, misses, false alarms, and lead time for each grid point and each time interval. Gridded warning and verification data also allow for the calculation of aggregate false alarm areas and false alarm times, departure times, and times under valid warnings. The HWT experimental warnings are compared to the official NWS warnings for the same events to determine whether or not the experimental severe weather applications offered improvements in warning goodness.