An evaluation methodology applied to the Damaging Downburst Prediction and Detection Algorithm

The warning guidance issued by WSR-88D algorithms has traditionally been evaluated using a technique known as "time-window" scoring. Although this method has been useful for evaluating algorithms that diagnose the presence of hail, mesocyclones, and tornadoes, it is inadequate for evaluating radar algorithms that predict or detect events of a shorter time scale, such as downburst events predicted by the National Severe Storms Laboratory's Damaging Downburst Prediction and Detection Algorithm (DDPDA). Algorithms that detect the presence of hail, mesocyclones, and tornadoes are typically diagnostic in nature, while the DDPDA is predictive in nature. The radar-detectable precursors to these downburst events are ephemeral in nature, and therefore the "time-window" scoring methods used to evaluate longer-lived events are inappropriate for use in evaluation of the DDPDA.

We propose a "cell-based" scoring method for downburst events that simulates how a forecaster might use the DDPDA output in the warning decision-making process. Since the precursors to downburst events are extremely short-lived, the DDPDA may only issue a downburst prediction for a short period of time (frequently one volume scan or 5 minutes), while other algorithms may issue warning output for 30 minutes or more. The "cell-based" scoring method allows each storm cell to be identified with a correct prediction, incorrect prediction, missed event, or correct non-prediction. In addition, a lead time can be calculated from the DDPDA-issued downburst prediction and the event time.

We extend this method to compare historical thunderstorm warnings issued by the National Weather Service and downburst predictions issued by the DDPDA. The data set used in this evaluation includes approximately 75 severe downburst events and 1000 non-severe cells from 40 separate days. Through this process, we plan to determine if the DDPDA would improve the warning verification scores of forecasters in various historical events.