Challenges and Solutions of Numerical Weather Prediction on the Cloud

In recent years, cloud computing technology has enhanced the efficiency of data generation, storage, and dissemination across multiple platforms and applications, allowing for easier collaboration and data sharing throughout the scientific community. Cloud-based computing provides new capabilities for Numerical Weather Prediction (NWP) modeling, which naturally aligns with the concepts of High Performance Computing (HPC) that are used within the meteorological community.

Some known issues with public cloud environments for HPC applications include: 1) varying latency among compute instances of unknown proximity; 2) unknown competition of multiple users operating VMs on the same hardware, 3) variability in hardware assigned to each virtual machine, and 4) occasional outages of different components. Public Cloud providers are mostly addressing these concerns by improving network efficiency and hardware homogeneity, however, the following issues remain unaddressed for operational NWP modelling in cloud computing environments: 1) interconnection speed, 2) cost management, 3) security, 4) scalability, 5) data ingress/egress.

ClimaCell Inc. has recently launched an automated, on-demand, scalable, cloud computing service for operational NWP modeling named ClimaCell Bespoke Atmospheric Model (CBAM). CBAM provides custom short- and medium-range forecasts anywhere in the world and shares some of the core capabilities of the U.S. 3-km resolution High-Resolution Rapid Refresh (HRRR) model. CBAM is a suite of atmospheric models that require extensive computational resources to operationally solve the laws of motion for accurate prediction of weather. CBAM operates in a cloud-based HPC environment, thereby eliminating the need to purchase dedicated HPC resources and allowing seamless scaling to meet the demand of ever growing weather prediction needs. In the present work, ClimaCell’s NWP team will address the above-mentioned challenges and high-level solutions regarding the design, development and deployment of CBAM system on Cloud.