Assimilating Real Observations with ML Models

There has been a recent surge in development of accurate global atmospheric model emulators, including full ML model emulators (FourCastNet, PanguWeather, GraphCast, GenCast) and hybrid models (NeuralGCM). Evaluation of these models has generally been focused on forecasts, mostly with 1-10 day lead times. However, forecasts rely on accurate initial conditions that, in the context of operational weather prediction, generally come from a cycled data assimilation (DA) system. While there have been a few DA studies that utilize emulators, these studies have focused on synthetic observation experiments. Here we present what is, to our knowledge, the first study of a DA cycling system using ML models to assimilate real observations. A reduced observation network consisting of only surface pressure observations is assimilated into ML models (including NeuralGCM, PanguWeather, and GraphCast) using an ensemble Kalman filter. This experiment design provides an informative test case since the performance of the DA system relies on accurately capturing ensemble covariances to update unobserved state variables, where the observed variables may or may not be part of the loss function used in the optimization process. Results will be compared to those from experiments that cycle with a coarse-resolution version of a fully physical model, NOAA’s operational Global Forecast System. The ML models suffer from stability issues, and even when stable do not perform well compared to the GFS, even when run at higher resolution. Strategies for modifying the training of ML models to improve their performance in cycled DA will be discussed.