Verification for the High Impact Weather Prediction Project (HIWPP) and other verification work to support model development within ESRL/GSD

Researchers within the Environmental Modeling Branch (EMB) of NOAA ERSL/GSD have been working in conjunction with scientists at the NCEP Environmental Modeling Center (EMC) to build a common verification system for evaluation of HIWPP global model runs. The initial verification system includes components from both the NCEP/EMC and ESRL/GSD/EMB verification systems, that have been independently developed, but share a number of common attributes. The Naval Research Laboratory (NRL) also has expertise in model forecast verification and is participating in a collaboration to define the most important features in these currently available tools and combine them into a new unified verification platform.

The new system, engineered with best practices, includes a database for storage of a variety of verification data and an interactive web-based interface. A variety of verification capabilities already exists, from current EMB and EMC systems. These span a range of measures from traditional large-scale metrics (global anomaly correlation coefficient, global upper-air errors relative to rawinsonde and aircraft observations) to metrics reflecting smaller-scale atmospheric processes (global surface errors relative to METAR temperature, dewpoint, and wind observations; as well as CONUS gridded reflectivity and precipitation verification). These EMC and EMB tools have been ported and adapted for use as a comparison baseline for the evolving HIWPP system. From this baseline, the HIWPP verification team has established a common framework, including standards for database storage and web display. The web interface is a key component of the system, featuring a wide range of display modes (time-series, lead-time, diurnal cycle, vertical profile, and threshold) and selectable stratification parameters (threshold, date range, vertical level, horizontal region and scale, time averaging, and event matching). These features are extremely useful, enabling on-the-fly decision making for interrogation of the verification data. As part of the project, this interface is being adapted from a java-based version to an html5-based version.

Additional capabilities are also being added to the system, including SYNOP-station-based and CMORPH-analysis-based global precipitation verification, ensemble verification, multi-dimensional “scorecard” verification, and graphical display of spatial error patterns. Although the system is being developed for HIWPP needs, a design goal is to provide a platform that can serve broader NOAA model development needs, including verification of storm-scale models and ensembles. Toward that goal a variety of additional capabilities that are included in the EMB verification system, are candidates for migration to the HIWPP system. At the conference, the verification system will be described and a variety of results will be shown, both to highlight the verification system and illustrate results for various model formulations.