Enhanced MATS: Process-Oriented Verification in the Cloud

Verification is essential to the development and operational transition of Global Systems Laboratory (GSL) numerical weather prediction modeling systems, historically with the Rapid Refresh (RAP) and the High Resolution Rapid Refresh (HRRR), and presently with the experimental Rapid Refresh Forecast System (RRFS). To support model development efforts, GSL has been developing a novel process-oriented verification system to complement the legacy Model Analysis Tool Suite (MATS), which will allow metrics to be calculated on user-prescribed regimes covering geographical domains, atmospheric conditions, and observation networks. These process-oriented capabilities are becoming crucial to finer-scale model development, as causes of systematic error become harder to isolate when limited to large-scale predetermined statistical aggregation. However, data size and processing times pose a challenge to the GSL model verification team’s requirements of a verification system and web interface with responsiveness, scope flexibility, and consistency. To address these obstacles, the model verification team has been investigating the Couchbase NoQSL document database as a flexible alternative to the rigid relational MySQL schema used in the legacy system. Initial results are promising, with a new internal prototype application running in real time with an on-prem Couchbase database cluster as the backend, the first application in the new Enhanced MATS web interface. While GSL is pleased with the initial prototype’s performance, it has become clear that data availability, performance scalability, and licensing are limiting factors in relying solely on the on-prem cluster. After a successful evaluation, the GSL has made the decision to move to the managed Cloud Couchbase service, Capella, as the backend of the Enhanced MATS model verification system, culminating in the full-stack system being fully transitioned into the Amazon Web Services (AWS) Cloud.

In this presentation, we will highlight the ongoing development of Enhanced MATS at GSL, specifically the evaluation of the Cloud-based Capella database, transitioning the full stack Enhanced MATS systems to an AWS instance, and the advantages in functionality that the new system provides. We will discuss the motivation, requirements, and methodologies utilized and future plans and uses for the system within numerical model development at GSL.