An Overview of Verification in Support of GSD Model Development and the Evolution of Process-Oriented Analysis Tools

Verification has been and continues to be essential to the development and operational transition of Global Science Division (GSD) models, particularly the Rapid Refresh (RAP) and the High Resolution Rapid Refresh (HRRR). To that end, the GSD Assimilation Development Branch (ADB) has developed and maintained a verification system over the past decade. This system was designed around a MySQL database instance with three major requirements: responsiveness, flexibility, and consistency. Various verification types are included, with slightly different methodologies, but in general statistics are produced from partial sums derived from model-obs pairs stored within the database from either real-time or retrospective sources. This schema allows for the ADB verification web application, the Model Analysis Tool Suite (MATS), to quickly derive statistical scores in an interactive manner. While this approach has been successful in the past, the evolution of model development is facilitating the need for more in-depth analysis than pre-generated partial sums can currently provide. User prescribed regimes covering geographical domains, atmospheric conditions, and temporal options are becoming crucial to finer-scale model development. In support of the second Wind Forecasting Improvement Project (WFIP2), ADB is developing a prototype process-oriented verification system to meet these needs and we are working to incorporate this system into our larger verification workflow.

In this presentation, we will highlight the development of the ADB verification system over the past decade, including requirements, methodology, and infrastructure, and the ongoing development of a next-generation process-oriented verification system within our group.