Development of a comprehensive severe weather forecast verification system at the Storm Prediction Center

A truly valuable and informative verification system does not simply shine a light on the past through simple comparison between forecasts and verifying events. It should also provide prompt, valuable feedback to forecasters, including links to patterns and environments that are related to forecast performance, so that improvements can be made through a continuous learning process. Forecast verification also provides an avenue of investigation into the underlying predictability of the forecast phenomena. This manuscript will describe the current efforts and future plans of the Storm Prediction Center (SPC), which forecasts severe convection over the continental United States, to evaluate forecasts and advance the constructive feedback between the verification and the forecaster.

While forecast verification has long been a priority for the SPC, recent advances in computing speed and technology have allowed a more comprehensive verification database to be developed, greatly increasing the amount of relevant information available. The database can be queried to provide a number of verification measures (such as POD, FAR, reliability of probabilistic forecasts) for any combination of severe report type and magnitude, geographical area, time period, and forecast type. For example, performance during a particular severe weather episode can be examined in near real-time, or performance during during the tornado season in the Plains or the “pulse severe” season in the southeast U.S. can be compared to the overall performance of the unit. In addition, reports can be clustered in time and space, so that isolated reports that do not meet specific forecast criteria can be identified and considered in the verification.

Additional approaches to forecast verification are also being developed. A project is underway to use environmental data from soundings and grid-based analyses to develop a climatology of severe storm environments, in order to provide a greater context for the forecasts and reports. Since isolated severe weather reports and reports that occur in less favorable environments for severe convection are generally more difficult to forecast, these additional data will allow for an assessment of the predictability of a given event. This will provide a new and essential component to the verification data, offering a link between verification statistics and synoptic and mesoscale backgrounds that drive the forecast process. In the future, lightning and radar data will be added to the database to provide even more context for the forecast verification.

As the verification database continues to evolve, forecasters will be able to slice through the data described above to determine the specific types of environments and cases where not only they do well, but also cases where they do not do as well. Through the development of a comprehensive integrated database containing severe weather forecasts and reports, remote sensing information about convective storm occurrence, and environmental information associated with each report, forecasters will gain new tools to better understand attributes of their decision-making processes, providing them with unique capabilities to improve severe weather forecasts.