Testing of Scorecards for Convection Allowing Models during HWT 2018

The use of scorecards to summarize forecast skill has been suggested as a best practice for designing ensemble prediction systems (Sandgathe et al., 2011, Sandgathe et al., 2013). Scorecards have been used in the development of the National Center for Environmental Prediction Environmental Modeling Center (EMC) Global Forecast System (Yang, 2010-2017 and Zhou et al., 2016) and a data assimilation framework (Kuhl et al., 2013). In addition, the National Center for Atmospheric Research (NCAR), through the Developmental Testbed Center (DTC), has been using scorecards for synthesizing the results of extensive testing and evaluation for many years. Statistics and significance for these scorecards are computed by the Model Evaluation Tools (MET) verification package, which is supported out to the community by the DTC. Flexible scorecarding capability was added to the METViewer database and display package in 2017 and is now being used by the DTC, EMC and others.

An ongoing collaborative project between NCAR and the National Severe Storms Laboratory (NSSL) is focused on determining which metrics and fields should be included in comprehensive scorecards. Specifically, the Hazardous Weather Testbed (HWT) Spring Forecasting Experiment is contributing to the development of a scorecard focused on storm attributes including composite reflectivity, updraft helicity, and accumulated precipitation. These scorecards are used to assess improvements to multiple Convection Allowing Models out to 36 hours. This presentation will introduce scorecarding and examine scorecards developed from the most recent HWT Spring Experiment.