Scalability and Forecast Impact when Varying Spatial Resolution, Ensemble Size and Observation Density in an Operational LETKF-based Ensemble Prediction System

The ensemble data assimilation system used for operational global medium-range ensemble weather forecasting at Environment and Climate Change Canada will soon be upgraded. Among the upgrades, the horizontal grid spacing of the forecast model and local ensemble transform Kalman filter (LETKF) was decreased from about 39km to 25km. Other notable modifications include the use of scale-dependent ensemble covariance localization for the 4D ensemble-variational analysis (4D-EnVar) used to partially recenter the ensemble of LETKF analyses. The full set of modifications results in significant improvements to ensemble forecast scores at all lead times and regions (e.g. up to 10h gain in lead time for 7 day forecasts of global 500hPa geopotential height). The LETKF and 4D-EnVar are both implemented in the same software framework, namely the Modular and Integrated Data Assimilation System (MIDAS), and therefore share much of the same Fortran code. The decrease in grid spacing represents a ratio of about 2.4 in total number of grid points. By increasing the number of computer processors used (ratio of about 1.7) the ratio of the execution time is only about 1.4 relative to the operational configuration. This modest increase in execution time was made possible by both optimizing the reading of 4D ensemble background states and the use of a similar coarse resolution grid for the computation of the LETKF weights. However, a detailed comparison of the 39km and 25km configurations also shows poor scalability in some sections of the MIDAS code. In preparation for future upgrades, additional comparisons were performed to evaluate the scalability and impact on forecast scores from using an even lower grid spacing of 15km (as currently used for deterministic forecasts), in addition to varying both the ensemble size and spatial density of observations.