Long-range forecast verification

The Canadian Meteorological Centre first implemented a long-range forecast system in August 1995, based on outputs from two models: a General Circulation Model with spectral truncation at 32 waves and 10 levels in the vertical and a Numerical Weather Prediction Model with 63 waves and 23 levels. The approach is based on an ensemble of twelve members, composed of six cases per model, initialised from objective analyses lagged by twenty-four hours. The two-model approach provides two independent ensembles on which the forecast is based. The expected skill of this approach and questions of systematic error correction, spread/skill relationships, and the optimum combination of several forecasts have been investigated over a 26-year Historical Forecast period, from 1969 to 1994, using NCEP re-analysis data. The long-range forecast system is used operationally to provide deterministic seasonal surface temperature and precipitation anomaly forecasts, classified into three equiprobable categories, above, below and near normal.

The seasonal temperature anomaly forecasts are produced using simple linear regression equations relating the anomalies of the blended 1000-500 hPa thickness forecast by the two driving models to the surface temperature anomalies at an ensemble of 240 Canadian stations. A strategy to blend the thickness of the two models based on the Best Linear Unbiased Estimate has been developed and tested over the 26-year Historical Forecast period and was shown to be better than the simple averaging of the models. Evaluation of the temperature anomaly forecasts over the Historical Forecast period shows reasonable skill, particularly during Winter seasons.

The precipitation outlook is produced using a more direct approach: the two ensemble means of forecast precipitation are subtracted from their respective model climatology, and normalised by model standard deviations. These normalised outlooks are then added, divided by two and re-scaled to have a standard deviation of one. Precipitation anomaly forecasts were evaluated over Canada over the 26-year Historical Forecast period and show marginal skill.

Long-range forecast verification is carried at the ensemble of Canadian stations and also on a latitude/longitude grid. Three by three contingency tables are constructed, from which several scores are calculated, the main one being the percentage of the forecast correct. Other scores are also calculated from the three by three contingency tables: frequency bias, probability of detection, false alarm ratio, critical success index and Heide skill score. A skill score based on root mean square error involving persistence and climatology as standards is also calculated for the anomaly forecasts before categorisation into three equiprobable classes.

The long-range forecast anomalies being based on an Ensemble Prediction system, can be rendered into probabilistic terms, by simply counting the number of members in each equiprobable categories. The probabilistic anomaly forecasts are verified using the Relative Operating Characteristic and compared to the deterministic forecasts. A calibration of forecast probabilities for each equiprobable category involving the skill of the forecast system evaluated over the Historical Forecast period can be applied.

Skill of the long-range forecast system will be discussed and comparative verification results for deterministic and probabilistic seasonal forecasts will be presented.