Enhancing Community Collaborations through NWP Software Containers

A major hurdle when attempting to run an atmospheric modeling system is properly setting up and compiling all of the necessary code components, including a number of external libraries. In addition to running a forecast model, users often need pre- and post-processing software, as well as a means to visualize and verify output from their model runs. To ease the burden of establishing an end-to-end workflow from the ground up, the concept of software "containers" has been gaining momentum in the modeling community. Containers allow for technical advantages such as easily bundling all necessary components of end-to-end software systems and allowing for highly portable and shareable code as well as scientific applications such as reproducible science.

NCAR colleagues have established containers to run a subset of an end-to-end NWP system, including the Weather Research and Forecasting (WRF) model, the WRF Pre-Processing System (WPS), and the NCAR Command Language (NCL). To provide additional capabilities and round out the end-to-end workflow, the Developmental Testbed Center (DTC) has developed containers for the Unified Post-Processor (UPP), the Model Evaluation Tools (MET), and the METViewer database and display software systems. In addition, development of a Gridpoint Statistical Interpolation (GSI) data assimilation system container is ongoing. Through this complementary work, a full end-to-end NWP system has been established for two case studies.

With models being run at higher resolution and as the use of ensembles become more prevalent, the necessary computational resources and increasing size of data volumes is making it natural to explore running containerized end-to-end NWP systems on cloud computing platforms. In addition, cloud computing offers an excellent opportunity to provide a sandbox to the community to share output and datasets, allowing for cross-community interactions. DTC is currently transitioning their end-to-end NWP containers to be functional on cloud computing platforms, with future sights on testing HPC platforms.

DTC staff are presenting a short course on the use of NWP containers in conjunction with the annual AMS meeting. This presentation will outline the motivation behind containers and provide a brief review of the short course material.