Extended Abstract (404K)8.2
Verification of The Weather Channel Probability of Precipitation Forecasts
J. Eric Bickel, Texas A&M University, College Station, TX; and S. D. Kim
The Weather Channel (TWC) has established itself as the major provider of weather information to the public. The public uses this information to make decisions as mundane as whether or not to carry an umbrella to as significant as whether to seek shelter from an approaching storm. But, how accurate are these forecasts and are they free from bias? Should the public accept TWC forecasts at face value or do they need to be adjusted to arrive at a better forecast? In this paper, we analyze the calibration of probability of precipitation (PoP) forecasts provided by TWC, via their website, over an eight-month period, at 50 reporting stations around the United States. Specifically we analyze n-day-ahead forecasts where n ranges from 0 (same day) to 9. We collected this data on a daily basis from www.weather.com and then worked directly with TWC personal, to obtain the observed precipitation at these locations and interpret these results. The results are quite interesting. For example, the same day forecast is relatively well calibrated for PoPs of 0.3 and above, but does demonstrate a tendency to over forecast precipitation for PoPs below 0.3. A PoP of 0.3 is the dry/wet cutoff at TWC and this tendency to over forecast is driven by a desire of the forecasters to shade their forecasts towards wet. TWC believes that consumers prefer this built in "hedge." In addition, since any PoP below 0.3 is considered "dry", less care is exercised in deriving the actual PoP used by the TWC system. Calibration results for 1 to 4 day lead times are similar to the same day results. Calibration performance worsens significantly beyond 6 days; TWC over forecasts the PoPs above 0.2 and under forecasts below 0.2, which is close to the climatological average for the 50 locations we considered. For example, TWC forecasted a 60% chance of precipitation 9 days in advance over 1600 times during the study period, yet precipitation was only observed in approximately 30% of these cases. Likewise, a PoP of 0% was given 9 days in advance over 1500 times, but precipitation was observed in approximately 11% of these cases. Thus, while the actual frequency of precipitation is monotonically increasing in TWC's PoP, the forecast itself is not well calibrated. The dramatic change beyond six days coincides with the TWC's reliance on numerical models without human intervention. Human forecasters use models as an input to their 0 to 6 day forecasts. But, beyond this point the model output is not adjusted. The fact that it is more difficult to forecast precipitation this far in advance, does not explain these results. The difficulty of the forecasting task should be reflected in the probability assessment itself (e.g., by not giving 60% PoPs 9 days in advance), which should still be well calibrated. In addition to these calibration results, we will discuss the use of forecasts in decision making and argue that forecasts should not be adjusted to protect consumers. Rather, PoP providers should produce unadjusted forecasts and leave it to consumers (decision makers) to incorporate this information into their decision making process.
Session 8, Forecast Evaluation I
Wednesday, 23 January 2008, 1:30 PM-2:30 PM, 219
Previous paper Next paper
Recorded presentation