Weather Forecast Application Portability Using Container Technology

Modern weather forecasting applications have proven to be useful for a variety of purposes, and their open-source availability has enabled them to be distributed globally. Often users of these applications encounter challenges with adapting them to their computing environments.

The development, deployment and configuration of these applications can be problematic and expose issues with their portability. Often the deployment of these applications is limited to specific hardware platforms or systems due to differences in system components or configurations. Specialty components such as Infiniband (IB) for high speed networking rely on software stacks, such as Message Passing Interface (MPI) that require tuning and customization based on the compute platform they are installed.

For example, a weather application that executes as designed in a development environment can stop working in a test environment, and similarly when moving from testing to staging, and from staging into production.

Containers are an option for enabling these applications to run reliably when moved from one computing environment to another. Containers are designed to package entire scientific workflows, software and libraries, and data. The base image is immutable within the Singularity container, ensuring that the core application stack won't change. This useful from a reproducibility standpoint.

For this presentation, containers will be developed using two weather codes (WRF, FV3GFS). The Singularity container technology will be utilized. Singularity is designed for use with High Performance Computing (HPC) applications. Both WRF and FV3GFS benefit from using HPC systems for scale, speed and accuracy. Each container will be transmitted and run on multiple platforms, including on-premise systems and commercial clouds.

The initial tests of portability will assess the requirements and steps involved for each container to run on other computational platforms.

If successful, each completed run will be measured for accuracy – did the run numerically replicate production model output? Did variations appear in runtimes or other elements such as data volume?

Results and observations from these test runs will be shared to enable a discussion to determine if container technology is a possibility for the transportability and consistent application of weather codes.