Convergence of Sampling Uncertainty in Forecast Models: When Do We Have Enough Ensemble Members?

Current ensemble weather forecasts are impacted by sampling uncertainty due to limited computational resources and it is unclear how many ensemble members are needed to reach an acceptable level of sampling uncertainty, and as a result the ensemble size is often chosen based on the qualitative shape of distributions. Using large (up to 100,000 member) idealised convective-scale ensembles based on modified shallow water equations, we look at how sampling uncertainty decreases as ensemble sizes go far beyond those of currently operational probabilistic forecasting systems. A bootstrapping technique is used on the distributions from the ensembles to create a convergence measure which quantifies how sampling uncertainty decreases with ensemble size. A universal power law in the sampling uncertainty is found across the model variables and all statistics tested, whereby when the ensemble is large enough, the sampling uncertainty decreases asymptotically, proportional to n^-1/2 where n is ensemble size. However, the magnitude of uncertainty differs depending on the synoptic situation, with a dependency on the shape of the ensemble distribution and the statistic of interest. Finally, it is discussed how to measure the convergence in a smaller ensemble of operational size, and to determine whether the ensemble is large enough for asymptotic convergence law to apply. With systematic asymptotic convergence, it is possible to determine the ensemble size needed to reach an acceptable level of sampling uncertainty for a given forecast statistic.