An Overview of Grid-to-Grid Verification at Environmental Modeling Center (EMC) of NCEP

The EMC of NCEP has two unified state-of-the-art verification systems, grid-to-obs (g2o) and grid-to-grid (g2g). The g2o is for verifying gridded forecasts against station observations such as METAR data while the g2g is for verifying gridded forecasts, either single model or ensemble, against gridded analysis data. In this paper, general verification methods and four major procedures in the g2g are briefly described, including gridded data preparation, verification user control definition, partial sum ( verification statistic database record , or VSDB) generation, statistical score generation and plotting. In the gridded data preparation, the locations or paths of forecast and analysis data files, in either GRIB1 or GRIB2 format, are identified and re-arranged according to the verification requirements. In this procedure, the forecast grid and analysis grid, if different, will be matched by using NCEP's copygb utility to convert forecast grid to analysis grid, or vice versa. The verification user control is to define requirements of a specific verification, such as cycles, forecast times and validation times, grid definition, observation type, verification regions or sub-regions, observation type, statistical type, variables, levels, thresholds, etc. The partial sum or VSDB record generation is the core part of the g2g verification system. In this procedure, a unified FORTRAN program is applied to generate SL1L2, or/and FHO partial sum for single model forecast or EFS partial sum for ensemble forecast. After the VSDB files are accumulated for a certain period of time, the final scores and their plots can be generated and displayed. This procedure can be done through various tools. The first tool is NCEP's legacy Forecast Verification System (FVS). In this system, accumulated VSDB files are read into the system, and then traditional scores such as non-category scores (RMSE, bias, correlation coefficient, etc) or/and category scores (hit rate, FAR, ETS, CSI, etc) can be generated and plotted. Another tool is NCEP's MYSQL based verification system. This system was adopted from GSD's RTVS and tailored for NCEP's current single model and ensemble forecast verification. In this system, the VSDB records are first stored in MYSQL database tables. Through an online database querying web-site, scores and plots can be obtained. Besides these two tools, we are collaborating with NCAR's MET-View development team to make it work on our VSDB format files as well. Currently the g2g has been implemented for routine forecasts at NCEP, including global (GFS) and regional (NAM, RAP) forecasts for total cloud, radar reflectivity, visibility, ceiling, echo-top, smoke and aerosols. In the next upgrade, more fields and model models, including ensemble verifications will be included.