Transitioning the GFS Verification to Using METplus

Verification for the Global Forecast System (GFS) is currently done using the National Centers For Environmental Prediction (NCEP) Environmental Modeling Center (EMC) Global NWP Model Verification Package, which is based in the shell and Fortran scripts that make up the Verification Statistics DataBase (VSDB). This package is used across EMC in a variety of ways. First, it is run daily producing statistics and graphics for the GFS and other centers’ operational global models. Second, it runs as a part of the GFS workflow producing statistics for experiments run by model developers, and, upon a completion of an experiment, is run to produce graphics which are sent to a webpage for display. This package has numerous capabilities: grid-to-grid verification (comparing model forecasts to model analysis), grid-to-observations verification (comparing model forecasts to surface and rawinsonde observations), precipitation verification, scorecards (shows the statistical significance difference between two model for various statistics, regions, forecast leads, variables), fits-to-observations (comparing the model forecast and data assimilation files to conventional ground and air observations), spatial map graphics, tropical cyclone verification, and data assimilation analysis increment graphics.

Under the Next Generation Global Predictions System (NGGPS) program, the decision was made to unify numerical weather prediction model verification around the Model Evaluation Tools (MET) that is developed by Developmental Testbed Center (DTC). This decision was expanded to include the METplus authoritative umbrella repository, which is comprised of MET, METviewer and METexpress (two database and display systems), and a group of python wrappers around all these tools. Many of the tools within MET and the use cases within METplus have been designed to mimic the verification capabilities of VSDB.

A new global model verification package is being developed at EMC that is based in METplus. The new package is a collection of shell and python scripts that set up a user’s environment, gets data, and calls METplus to create verification statistics and graphics. This will have the same capabilities as the old EMC VSDB-based global model verification package, and eventually expand upon them. This package will be incorporated into the GFS workflow as well as run as for daily operational verification. This package will be housed on National Oceanic and Atmospheric Administration’s (NOAA’s) Virtual Lab (VLAB) allowing it to be checked out and run by users across the NOAA community.