Is Change Good? Measuring the quality of updating forecasts

The question of consistency of updated forecasts through time often comes up in the context of weather events such as hurricanes or high wind days. For a single event, forecasts are made and updated as the time of the event nears. If these forecasts change much or often, a user may believe they are of low quality, possibly even random. This is particularly an issue for decision makers who create plans based on early forecasts, then must change their plans repeatedly as new forecasts arrive. Thus, for forecasts that get updated, the consistency in the updates is an important aspect of forecast quality. Unfortunately, though everyone knows forecast consistency or jumpiness when they see it, the use of objective measures in forecast verification is very limited.

A similar problem exists in economic forecasting, where consistent forecasts are referred to as rational. Several measures used by economists to determine rationality are tested on example weather forecasts. The utility and sensitivity of these tests for evaluating the quality of updated weather forecasts are discussed.