Reevaluating Performance in the Statistical Metrics for National Weather Service Tornado Warnings

The National Weather Service (NWS) issues Tornado Warnings for the American public with the intent of providing advance notice of tornadic activity. For the past two and a half decades, the statistical measures of performance for these warnings has been defined by the Government Performance and Results Act (GPRA) of 1993. The metrics outlined by this Act account for the following three primary measured characteristics of warnings: the “accuracy” of the warning, through the Probability of Detection (POD); the False Alarm Ratio (FAR); and the average Lead Time (LT) in advance of a tornadic event. A supplementary fourth metric, the Critical Success Index (CSI), is intended to provide a more comprehensive score of performance for each warning, based on a statistical comparison of POD and FAR. However, these metrics neglect the Enhanced Fujita (EF) Scale rating, which is the official intensity rating of each tornado. Additionally, while there is a statistical relationship between POD and FAR, neither metric collectively or individually represent the full value of service from a warning for a tornadic event. The CSI metric therefore also does not provide any more apparent measure of service from a warning, relative to the scope of the tornadic event. In recent years, the NWS Damage Assessment Toolkit (DAT) has allowed NWS damage surveyors to include much more information about characteristics of a tornadic event than what was previously possible. Damage surveyors can include information about specific damage points, assigning individual wind speed estimation information and EF ratings for each damage point into the online DAT interface. This data can then be utilized within more robust statistical methods to better assess and evaluate the performance of each tornado warning, relative the event that occurred. The methods employ the integration of DAT points and produce a proper scaling of the performance of each warning, relative to the totality of the damage path from a tornado. Evaluation of these prototype methods are expected to yield more reasonable and understandable results and may more appropriately assess the service provided through Tornado Warnings for the American public.