Local Diagnostics to Assess the Performance of Multi-Model Ensemble Prediction Systems

In this study, we assess the performance of multi-model ensemble prediction systems using local linear diagnostics applied to ensemble perturbations in a small local neighborhood of each model grid point. A local error covariance matrix is defined for each local region, and the diagnostics are applied to the linear space defined by the range of the ensemble-based estimate of the local error covariance matrix. The particular diagnostics are chosen to investigate how well the ensemble performs in capturing the space of forecast or analysis uncertainties, predicting the magnitude of forecast and analysis uncertainties, and accurately representing the relative importance of different state space directions. We compare CMC, NCEP, and NAVGEM ensembles with multi-model ensembles created by pooling together the single model ensembles, with all members having equal weight. We assess whether the multi-model ensemble provides a better representation of the space of the error in the ensemble mean forecast and investigate whether multi-model ensemble systems provide advantages beyond those imparted by the increase in ensemble size alone. While this study focuses on forecasts out to 5 day lead times, the diagnostic framework presented here could be extended to assess predictability on subseasonal to seasonal (S2S) timescales.