An Overview of the Environmental Modeling Center Verification System (EVS)

The first version of the Environmental Modeling Center Verification System (EVS) was implemented into real-time operations on NOAA supercomputers in March, 2024. The EVS is designed to provide verification statistics and graphics for NCEP operational forecast systems, and to unify EMC’s verification strategy under one maintainable and supportable software system. Using the Model Evaluation Toolkit (METplus) suite of verification software, the EVS uses both real-time forecast output and environmental observations to create information on the performance on all real-time NCEP environmental models and products. The hundreds of thousands of graphics created per day by the EVS are displayed on a public web site, to produce a comprehensive and up-to-date overview of the performance of NCEP forecast systems for the general public, model developers, researchers, and decision makers. Ultimately, the EVS was developed to be a single, unified verification software system for real-time verification analysis, evaluation of new and upgraded systems proposed for NWS operations, and as a capability to determine systematic errors and biases that illustrate areas for potential model improvements. The next version of the EVS, currently in development, will add enhancements and additions to the software which include new metrics and graphics, and to fully incorporate the majority of performance metrics that were vetted and peer reviewed by the Unified Forecast System (UFS) community via the Developmental Testbed Center’s 2021 UFS Metrics workshop. In addition, EVS has been adapted to run on the release candidates for some new NCEP models, allowing for a comparison between new and previous systems, and legacy systems that are slated for eventual retirement.