A multi-tiered verification of SPC tornado watches

A top-down decision tree approach is used to characterize the degree to which Tornado Watches issued by the Storm Prediction Center between 2003 and 2008 verified. A total of 1901 tornado watches were examined during the 6-year period. Particular attention is paid to "null cases," or watches that do not fully verify. Watch performance is classified into nine categories of verification that assess for each watch a variety of phenomena occurring within the watch, including: the number and EF-scale rating of tornadoes, and the number of large hail and wind damage reports, NWS tornado and severe thunderstorm warnings, and occurrence CG lightning. Based on SPC in-house verification guidelines, these nine categories were further grouped into four “superclasses”: 1) verified tornado watch, 2) verified severe thunderstorm watch, 3) near-miss, and 4) full-miss, in order to more easily categorize the results. The data were further stratified to identify regional, seasonal, and diurnal characteristics of tornado watch performance.

Nearly half of all tornado watches verified fully, and another 31% verified as severe thunderstorm watches. Particularly Dangerous Situation (PDS) tornado watches verified at a substantially higher rate (~80% verified fully), with less than 4% of the PDS tornado watches failing to verify as either tornado watch or severe thunderstorm watch. Overall, less than 5% of all tornado the watches contained no severe storm reports nor warnings; these typically occurred in situations of extreme CAPE when very rapid storm development was possible but convective initiation was uncertain owing to the presence of a capping inversion.

The highest verification rates were generally associated with watches issued during the springtime months and during the daylight hours, and a slight but noticeable improvement was evident toward the end of each forecast shift. Higher false alarm rates were associated with tornado watches during the summer over the northeast US, and during the fall and winter over the southeast states. These results are consistent with the climatology of severe convective storm environments and tornado occurrence. For example, the lower verification rate over the southeast US in fall and winter reflect forecasting challenges associated with low CAPE-high shear environments that often characterize both landfalling tropical cyclones as well as cool season baroclinic systems over this region.