Dynamic Interaction: Metadata in the ADB Verification System

Verification is a crucial component of model development at the Global Systems Division (GSD), particularly of the Rapid Refresh (RAP) and the High Resolution Rapid Refresh (HRRR). Automated real-time and retrospective verification is essential to fulfilling the developers’ requirements that verification be responsive, flexible, and consistent. To that end, the GSD Assimilation Development Branch (ADB) developed a MySQL database schema that allows for metadata to act as the foundation for the entire ADB Verification System, storing a diverse set of dataset, temporal, geographical, and managerial information. This metadata schema was originally designed to assist in the processing and ingest of model and observation data, along with the eventual generation of statistical output. Over the past few years, this role has expanded with the advent of the Model Analysis Tool Suite (MATS), which allows for dynamically updated parameter selectors in an effort to ensure the GUI provides valid output to the end user. This expansion lead to the metadata being the core of the entire verification full-stack system, with various aspects being more important at certain points in the overall workflow. While this MySQL database schema has been successful up to this point, ADB is currently undergoing a transition to the NoSQL document-oriented database, Couchbase. With that transition came additional design challenges and a chance to apply the lessons learned from the previous system, culminating in the new ADB data model that allows greater utilization, flexibility, and data comprehension for both verification and model developers.

In this presentation, we will highlight the importance of metadata to the current ADB Verification System, from data processing to visualization. Additionally, we will discuss the challenges faced when dealing with growing requirements and designing a data model for the database transition.