Testing of Scorecards for Convection Allowing Models during HWT 2018

The use of scorecards to summarize forecast skill has been suggested as a best practice for designing ensemble prediction systems (Sandgathe et al., 2011, Sandgathe et al., 2013). Scorecards have been used in the development of the National Center for Environmental Prediction Environmental Modeling Center (EMC) Global Forecast System (GFS – Yang, 2010-2017 and Zhou et al., 2016) and a data assimilation framework (Kuhl et al., 2013). In addition, the National Center for Atmospheric Research (NCAR), through the Developmental Testbed Center (DTC), has been using scorecards for synthesizing the results of extensive testing and evaluation for many years. Statistics and significance for these scorecards are computed by the Model Evaluation Tools (MET) verification package, which is supported out to the community by the DTC. Flexible scorecarding capability was added to the METViewer database and display package in 2017 and is now being used by the DTC, EMC and others.

The Hazardous Weather Testbed (HWT) Spring Forecast Experiment is a collaborative project between the National Severe Storms Laboratory (NSSL), Storm Prediction Center, and the research community that investigates the predication of hazardous convective weather. Specifically, a variety of convection allowing models (CAMs) are evaluated daily and used as guidance for experimental forecasts. During HWT 2018, NCAR’s Research Applications Laboratory collaborated with NSSL to develop a verification system based on the Model Evaluation Tools (METplus) system that included the use of scorecards to assess improvements in these CAMs. The focus of the system was on storm attributes such as composite reflectivity, accumulated precipitation, and updraft helicity forecast out to 36 hours. This presentation will introduce scorecarding and examine scorecards developed from the HWT 2018 Spring Experiment.