Generating Cost-Saving Surrogate Background Ensemble with Diffusion Model for Convective-Scale Ensemble-Based Data Assimilation

Given the high degrees of freedom in state-of-the-art numerical weather prediction (NWP) models, the proper estimation of the background uncertainty in convective-scale ensemble-based data assimilation (EDA) requires a large size of ensemble members, ideally Ο (1000) (Necker et al. 2020; Evensen et al. 2022). The computational cost of generating the background ensemble with a large size is prohibitively enormous. The issue becomes more problematic for convection-allowing models with fine resolution grid sizes. The machine learning (ML) methods have shown great potential in emulating numerical models in a cost-effective manner (e.g., Arcomano et al. 2020). However, so far, most studies focus on exploring the surrogate models for coarse resolution medium range forecasts. Rather limited studies explore the ML-based surrogate short term background ensemble for ensemble-based data assimilation, especially for the convective scales.

As the capability of the diffusion model in restoring fine-scale details, this study will investigate a diffusion model (DM)-based background ensemble emulator to generate the background ensemble forecasts for convective scale data assimilation. The diffusion model emulator will learn from the historical convective-scale forecasts (e.g., CONUS404; Rasmussen et al. 2023). The covariances, a critical quantity for DA, derived from the surrogate background ensemble forecasts are first evaluated against the ensemble forecasts from the real NWP model, MPAS. Cycled DA experiments with a 40-member ensemble are also conducted to further demonstrate the surrogate background ensembles in the estimation of background error covariances.