Impact of model error and imperfect initial condition perturbations on ensemble-based probabilistic forecasts: UNPREDICTABLE SPOTS

The ultimate goal of ensemble forecasting is to reliably estimate the time-evolution of a probabilistic density function (PDF) of meteorological fields by quantifying forecast uncertainties both at the initial time and over the entire model integration. In this study, using the NCEP short range ensemble forecasting (SREF) system, two "perfect model" experiments were conducted to address the following three issues: (1) given a near-perfect Ensemble Prediction System (EPS), how well can PDF be predicted? (2) how can bad PDF forecast regions be identified (referring to "unpredictable spots")? and (3) what is the relative importance between model error and imperfect initial condition (IC) perturbations over the evolution of the PDF?

Although a good ensemble system could produce good mean, spread and probability forecasts at the majority of model grid points, it's almost certain that it also generates extremely bad and misleading forecasts at some locations which are defined as "unpredictable spots". As long as the model used is imperfect, "unpredictable spots" will never diminish even if the IC perturbations used in an EPS is perfect. Identifying the location of "unpredictable spots" is important for forecast calibration, but it's not an easy task because those spots are not well correlated with ensemble spread (predictability) in general, i.e., ensemble spread alone might not be a good indicator for identifying them. Our results further indicate that the correctness of the model physics might be more important than that of the IC perturbations in order to have a correct PDF forecast, at least in the big picture (but it is not conclusive at this point). Only if given a perfect model and very realistic IC perturbations could an EPS produce good (but still not perfect) forecasts over nearly the entire model domain. In a word, the task of correctly predicting the probability distribution or PDF using ensembles is extremely challenging if not impossible.