Powering AI-based Weather Prediction (Invited Presentation)

Over the past two years, AI-based weather prediction (AIWP) models have exploded onto the scene with performance competitive with or better than traditional numerical weather prediction (NWP) models. Once trained, AIWP models accomplish this feat while running four to five orders of magnitude faster than their NWP counterparts, opening up new forecasting applications such as large ensembles and rapid sensitivity analyses.

This represents a shifting landscape for the use of high performance computing (HPC) in weather forecasting. At present, AIWP model training is out of reach for most of the public sector due to the extremely high cost and scarcity of powerful GPUs. Furthermore, while AIWP model inference is much more accessible than running NWP models, it still requires an A100 or comparable GPU to run state-of-the-art AIWP models rather than the traditional supercomputers used by NWP. In other words, existing weather forecast infrastructure is likely not sufficient for this paradigm shift. These requirements are only going to increase rapidly in the coming months and years to fully leverage the potential of AIWP, including generating hundred or thousand member ensembles in parallel and increasing AIWP resolution towards the global storm-scale.

In this talk, we recount our experiences and struggles working with AIWP models through the lens of HPC resources. Specifically, we will detail the on-prem and cloud compute resources that we have leveraged to develop real-time and retrospective AIWP applications, how we are preparing to develop our own storm-scale AIWP model, and the resources we foresee as necessary to keep up with a rapidly evolving, exciting discipline.