Exploring Supportive Analytics in the Performance Evaluation of NWS Tornado Warnings

The National Weather Service (NWS) issues Tornado Warnings for the American public with the intent of providing advance notice of tornadic activity. For the past two and a half decades, the statistical measures of performance for these warnings has been defined by the Government Performance and Results Act (GPRA) of 1993. The metrics outlined by this Act account for the following three primary measured characteristics of warnings: the “accuracy” of the warning, through the Probability of Detection (POD); the False Alarm Ratio (FAR); and the average Lead Time (LT) in advance of a tornadic event. A supplementary fourth metric, the Critical Success Index (CSI), is intended to provide a more comprehensive score of performance for each warning, based on a statistical comparison of POD and FAR. However, these metrics neglect the Enhanced Fujita (EF) Scale rating, which is the official intensity rating of each tornado. Additionally, while there is a statistical relationship between POD and FAR, neither metric collectively or individually represent the full value of service from a warning for a tornadic event. The CSI metric therefore also does not provide any more apparent measure of service from a warning, relative to the scope of the tornadic event.

In recent years, the NWS Damage Assessment Toolkit (DAT) has allowed NWS damage surveyors to include substantially more information about the characteristics of a tornadic event than what was previously possible. Damage surveyors can include information about specific damage points, assigning individual wind speed estimation information and EF ratings for each damage point into the online DAT interface. This data can be utilized through more robust analytical and statistical methods to assess and evaluate the performance of each tornado warning, relative to a qualifying event. These analytical methods in this presentation integrate DAT damage point information to produce a scaled depiction of statistical performance of each warning, relative to the totality of the damage path from a tornado. These methods can also assess cumulative warning performance for any area, length of time, and other queryable elements. Evaluation of these prototype methods are expected to yield more reasonable and understandable results and may more appropriately assess the service provided through Tornado Warnings for the American public.