Enabling Different Ensemble Data Assimilation Scenarios In JEDI Using The SkyLab Workflow

The Ensemble Data Assimilation (EDA) approach involves using multiple model simulations, each initialized with slightly different conditions, to account for uncertainty in the model and the observations. This method has been shown to be effective in improving the accuracy and uncertainty estimates of forecasts and has been widely used in weather forecasting applications.

The Joint Center for Satellite Data Assimilation (JCSDA) has developed a comprehensive workflow (SkyLab) to facilitate the execution of diverse EDA scenarios. These scenarios can involve different instruments, algorithm choices such as background error covariance or cost function, and even different weather models. The SkyLab workflow serves as the orchestrator, driving the underlying JEDI code (Joint Effort for Data assimilation Integration) to execute the different EDA experiments.

In this study, we present an overview of our workflow's architecture and a comparative analysis of three distinct methods employed to conduct EDA experiments.

The first method involves a distinct DA run for each member. The second approach leverages block-based methods: since many similar optimization problems are solved it is possible to use information from all the members to construct a better approximation of the eigen-structure of the matrix at the heart of the optimization problem and accelerate the convergence. The block Lanczos algorithm is such an example. Finally, we introduce a novel application developed at the UK MetOffice of an EDA solving for the full unperturbed control run and only the perturbations of the ensemble members.

During our presentation, we will illustrate how the SkyLab workflow developed at JCSDA helps us set up and compare these different experiments. We will highlight the advantages and trade-offs associated with each ensemble method, with results from experiments using real observational data and the NOAA United Forecast System (UFS) model. Furthermore, we will discuss practical considerations and implications for future operational implementation.