Compared to What? Establishing Environmental Baselines for Tornado Warning Skill

In recent years, the focus of tornado research has broadened from the traditional Great Plains supercell springtime paradigm to include more atypical scenarios, such as tornadoes that occur during the fall and winter, at night, in the southeastern U.S., or associated with non-supercell storm modes. With this breadth of focus, however, come deceptively complicated questions: should we be applying the same forecast evaluation criteria across the entirety of this vast range of tornado scenarios? What is the baseline against which we should be comparing warning performance? Can warning evaluation be somehow weighted to reflect the added forecasting challenges associated with tornadoes that occur in less “textbook” environments?

Using 15 years of tornado event and warning data for the continental U.S., we create warning performance summaries in environmental parameter spaces. These summaries provide a “moving target” baseline with which we build an Environmental Skill Score (ESS): this score measures the degree of improvement in tornado warnings as compared with the climatological skill we would expect for tornadoes occurring under similar environmental conditions. We then apply the ESS to the evaluation of tornado warning skill in a variety of categories: parent storm mode, time of day, time of year, etc.

Through ESS summaries, we identify and discuss scenarios in which additional work, research, and/or training is needed to bring warning skill up to speed with the skill level we would expect given the tornadic near-storm environment. We also highlight the scenarios in which warning performance exceeds expectations. The ESS described in this proof-of-concept work is not limited to tornado warning verification, and we encourage its application in forecasting any phenomenon for which the difficulty of that forecast varies according to the ambient environmental conditions.