Probabilistic precipitation forecast skill as a function of ensemble size and spatial scale in a convection-allowing ensemble

Probabilistic quantitative precipitation forecasts (PQPFs) from the Storm-Scale Ensemble Forecast (SSEF) system run by the Center for Analysis and Prediction of Storms (CAPS) for the 2009 NOAA/Hazardous Weather Testbed Spring Experiment are evaluated using the area under the relative operating characteristic curve (ROC area). The ROC areas are examined as a function of ensemble size and spatial scale for 6-hour accumulation periods and various rainfall thresholds up to forecast lead times of 30 hours for 25 cases. A total of 17 SSEF system members were available and tests will examine PQPFs computed using 1 to 17 members. The spatial scale was varied by reassigning grid-point values based on the average forecast or observed precipitation from grid-points within specified radii that varied between 4- and 200-km.

As expected, PQPF skill generally increases with increasing ensemble size. However, the gain in skill from each additional ensemble member decreases as the number of members approaches the full ensemble. Significance tests reveal that the ensemble size at which differences in PQPF skill relative to the full 17 members are no longer significant varies mostly as a function of forecast lead time and spatial scale. Specifically, more members are required to reach statistically indistinguishable PQPF skill relative to the full ensemble as forecast lead time increases and spatial scale decreases. These results reflect the widening of the probability distribution function (PDF) of future atmospheric states associated with decreasing spatial scale and increasing forecast lead time (i.e. more members are required to effectively sample a wider PDF), and illustrate that to efficiently allocate computing resources to obtain skillful PQPFs from a convection-allowing ensemble, the spatial scale and forecast lead time at which forecasts are desired should be carefully considered.