Evaluation and Comparision of Machine Learning-Based NWP Model Emulators in Multiscale Ensemble-Based Data Assimilation

Ensemble-based data assimilation (DA) requires a set of short-term background ensemble forecasts (practically Ο (10-100)) to represent the background uncertainties. Operational global and regional DA systems commonly adopt self-cycled background ensembles to maintain the consistency between the background error covariances and the background forecast. However, the background ensemble forecasts are the most expensive component in a full DA cycle. Significant advancements in machine learning (ML) models for weather forecasting enable surrogating numerical models with the cost-effective data-driven models (e.g., Pathak et al. 2022; Keisler 2022; Bi et al. 2023; Chen et al. 2023; Lam et al. 2023; Price et al. 2024). So far studies primarily focused on applying and evaluating the surrogate models for the medium-range (1-15-days) forecasts. For example, GraphCast (Lam et al. 2023) suffers from generating blurry atmospheric states, especially for longer lead times. While diffusion model (DM)-based GenCast (Price et al. 2023) addresses the blurring issue, multiple function evaluations for sampling each forecast timestep lead to more computational costs than GraphCast. However, unknowns remain in the performance of these state of the art surrogate models in emulating short-term background forecasts for ensemble based DA.

This study is the first to compare and evaluate the short term background ensemble forecasts from GraphCast and DM. As an initial study, an idealized numerical Surface Quasi-Geostrophic (SQG) model is used. The full fields of each surrogate short term forecast is verified against the corresponding short term forecast from the SQG model. Background ensemble covariance derived from the GraphCast-based and DM-based surrogate short term ensembles are verified against the covariances from the SQG ensemble Finally, the cycling DA experiments using the Multiscale Local Gain Form Ensemble Transform Kalman Filter (MLGETKF) framework (Wang et al. 2021) are performed to examine both surrogate ensemble emulators in the context of multiscale ensemble based data assimilation.