Quantifying the Predictive Skill of the Global Ensemble Forecast System for Precipitation based on 20 Years of Reforecast Data

Ensemble forecast systems are receiving increased attention as a tool for assessing forecast confidence, particularly in the medium range (3–8 days). For example, the National Weather Service's Western Region Headquarters is aggressively promoting the use of ensemble-based situational awareness tools, many of which focus on anticipating heavy precipitation events in the medium range. However, it is generally accepted that the medium-range quantitative precipitation forecast (QPF) from global ensembles has limited skill. Hence, recent studies focusing on the western U.S. have shown that more spatially coherent “proxy” variables, such as integrated water vapor (IWV; Ralph et al. 2004) and IWV transport (IVT; Rutz et al. 2014) are highly correlated with observed precipitation over complex terrain. Despite the increasing attention these variables are receiving, a fundamental question remains: are forecasts of proxy variables more skillful in predicting observed precipitation than the model QPF itself?

We address this question by examining 20 years (1993–2012) of cool-season (October–March), Global Ensemble Forecast System (GEFS) reforecast data (Hamill et al. 2013) to quantify the relationship between selected forecast variables (QPF, IWV, IVT, and observed precipitation. To assess the predictive skill of the GEFs, we compare the percentile rank of the ensemble-mean forecast for a given variable and lead time (relative to all forecasts at that lead time, i.e., the "model climate") to the percentile rank of the corresponding analysis (relative to all analyses). By focusing on percentile ranks, this approach normalizes the probability density functions of ensemble-mean forecasts and analyses for different variables and different lead times, allowing for more intuitive comparisons between these quantities. Results are presented regarding the skill of GEFS ensemble-mean forecasts for the selected variables and the usefulness of each in predicting observed precipitation. A key aspect of this analysis is to determine, for a variety of variables, lead times, and geographic areas, whether exceptional forecasts relative to the model climatology can be reliable indicators of exceptional precipitation events.

Finally, we investigate post-processing methods to determine the probability of exceptional precipitation events using the percentile rank of forecast fields relative to the model climate. These methods are evaluated using both an independent set of GEFS reforecast data and a set of operational GEFS ensemble forecasts covering multiple cool seasons.