Precipitation forecasts of the Canadian ensemble prediction system

The Canadian ensemble prediction system (eps) has been running operationally since 1998. As this forecast tool matures there is mounting interest in harnessing it to the early detection of potentially severe weather which could significantly impact the public. To that end the extreme forecast index (EFI) implemented at the European Centre for Medium-Range Weather Forecasting (ECMWF) has been adapted to the Canadian eps, with a view to testing its utility in an operational environment. The Canadian eps is comprised of 17 members (1 control and 16 perturbations) compared to 51 members in the current implementation of the ECMWF system. The size of the sample of forecasts produced by the ECMWF system lends itself to an adequate description by means of the empirical cumulative distribution function (ecdf), which is what is used in the computation of the European EFI. However, owing to the much smaller size of the forecast sample available from the Canadian system, we investigated other models for the distribution of the ensemble member forecasts produced from a particular model run.

Parametric models describing the climatological distributions of weather elements observed at a given location have been well known for quite some time. The probability density function (pdf) of the daily precipitation amount, for example, can usually be closely approximated by the gamma distribution, the shape and scale parameters of the gamma distribution chosen to give the best fit to the pertinent sample of observations or forecasts. However, the precipitation forecasts from the ensemble members, for a particular run of the model, are conditional upon the control forecast from which the perturbed members of the ensemble are constructed. There is no reason to expect these forecasts to follow a gamma distribution, and indeed no parametric model for the pdf of the ensemble member forecasts suggests itself a priori. We therefore fitted nonparametric models, specifically kernel density estimators with Gamma kernels, to the ensemble member forecasts, the choice of Gamma kernels being dictated by the constraint that precipitation amounts are confined to the positive real axis. While these models are devoid of parameters, per se, there does remain the choice of kernel bandwidth, which determines the amount of smoothing incorporated into the kernel density estimator. Increasing the bandwidth generates a smoother density estimate, generally at the expense of increased bias in this estimate which translates into less reliable forecasts. Increasing the interval over which the precipitation accumulates effects a temporal smoothing upon the forecast and observational signals. One would expect the forecasts to evince greater reliability for longer accumulation intervals as the eps forecasts become less sensitive to phasing errors in the ensemble members.

In order to explore these points precipitation amounts reported at a number of surface observing stations across Canada, along with the precipitation forecasts from the Canadian ensemble forecast system interpolated to the locations of these stations, were collected for the period from August, 2001 to November, 2004; the Canadian eps did not undergo any drastic changes during this period. Kernel density estimates were constructed, employing several different algorithms to determine the smoothing bandwidth, to model the distribution of the daily precipitation forecasts from the ensemble members. The member forecasts were also modelled with the ecdf, which can itself be considered a kernel density estimator in the limit of vanishing bandwidth.

The models were evaluated on the afore-mentioned sample by means of the usual measures, capturing the various facets of forecast performance: Brier and ROC scores, reliability tables, frequency distributions, etc. This evaluation was carried out for forecasts of precipitation amounts accumulating over a 24-hour period and over intervals of 2, 3, 5, 7 and 10 days. The relative performance of the different models is examined, as well as the overall quality of the Canadian eps precipitation forecasts. These properties are explored as a function of forecast projection and precipitation threshold separating events from non-events, and the impacts of seasonal and regional stratification are also investigated. Implications for the application of the Canadian ensemble system to the forecast of high-impact weather events are discussed as well as possible directions for future work in this area.