Trends in skill of weather element forecasts in Canada

Forecasts of maximum and minimum temperature and probability of precipitation extracted from the early morning issue of the public forecast bulletins have been verified since 1984 at an ensemble of twenty-three stations, corresponding to the main cities across Canada. The verified forecasts were valid for Today, Tonight and Tomorrow. Statistical forecasts of maximum and minimum temperature and of probability of precipitation at the same set of stations, based on 00 UTC model run, have also been verified over the past six years. These latter forecasts are based on the Perfect Prog multiple linear regression approach. The same verification system has been used to verify the manually produced forecasts and the objective forecasts. Both verifications have been carried with a seasonal data stratification. It is assumed that the forecasters had access to the objective statistical forecasts as guidance.

The forecast data extraction system used to collect data and the verification system have evolved considerably over the period of the experiment. The forecast data extraction and the verification tasks were mostly carried manually in the early years, and went through an automation process in the early 1990’s. Considerable care was taken in the manual processes to quality control the data and to ensure a maximum of data available for verification. However, quality control of the data has been automated and missing data are not retrieved in the automated processes. The automated verification system is based on the following framework: all available surface observation, synoptic, hourly and supplementary aviation observations are used to create a truth file at a set of stations. The truth file is a matrix which includes all observed weather elements with a time resolution of one hour, taking into consideration the observations produced at non-standard times. The truth file is generated once a day at each station, for the past twenty-four hours. On the other hand a similar set of matrices are generated for the forecasts. The forecast matrices and the truth matrices can then be compared and the validity and skill of the forecasts assessed.

Different verification scores have been used to assess whether or not there is an improving trend in forecast skill since 1984. Results indicate that the maximum and minimum temperature forecasts have improved globally across Canada over the period of the study. However, verification prior to 1992 fails to detect any statistically significant trend in skill of temperature forecasts, but there is a definite improvement trend after 1992. There is also an improving trend in skill of probability of precipitation forecasts over the period of the study, but at a lower rate. There are indication that the manually produced probability of precipitation are leveling off over the past four years. The manually produced forecasts have also been compared to the objective forecasts. Although the skill of the objective statistical forecasts and that of the manually produced ones vary significantly from station to station, the results indicate that overall the manually produced forecasts improve upon the objective forecasts.