Project Phoenix - Optimizing the machine-person mix in high-impact weather forecasting

In the numerical model-dominated weather services of the world, do meteorologists still add value in weather forecasting? Project Phoenix was an innovative experiment designed to test this question. Conducted early in 2001 at the Prairie Storm Prediction Centre (PSPC) in Winnipeg, Manitoba, a team of three forecasters prepared noon and late afternoon public weather forecasts for the Canadian Provinces of Alberta, Saskatchewan and Manitoba. This Phoenix team began with the text forecasts produced by SCRIBE, Canada's automated model-based forecast production software. From this starting point, the meteorologists produced their weather forecasts using only their analysis, diagnosis and prognosis skills by using real data, such as radar, satellite imagery, surface observations and upper air soundings. The Phoenix forecasters had no access to any numerical weather prediction data beyond the original SCRIBE text forecasts. The goal of the two week experiment was to discover if short-term forecasting techniques applied by meteorologists could achieve a significant improvement over the automated SCRIBE product, particulalry in terms of high-impact weather (HIW). The performance of the SCRIBE, Phoenix, and the official forecasts were verified against a comprehensive evaluation system.

The Phoenix meteorologists improved the forecasts beyond expectations during the two-week test. In a curious twist, the Phoenix forecasts also were significantly better than the official versions prepared by the PSPC. The PSPC repeated the experiment a few months later. The results confirmed the first test and Project Phoenix became a formal training program for all PSPC meteorologists. Over a dozen more groups completed a one-week version, including one limited to senior personnel and another team comprised of recent graduates. Every Phoenix group posted major improvements over the automated forecast products. As well, all the Phoenix teams achieved at least a slight improvement over the PSPC official forecasts. This training system has now been adopted by the Meteorological Service of Canada to position forecasters for HIW forecasting roles in the years ahead. This presentation examines Project Phoenix, the verification system, and offers an explanation for the results.