Revisiting Cost-Loss Decision Making with Deterministic and Ensemble Forecast Data.

A simple decision-making framework used with weather predictions is the cost-loss approach, discussed in many articles. The simple dichotomous cost-loss model assumes that the user may decide to employ protective actions in anticipation of adverse weather, with the cost of protection “C.” While proposed models vary in their assumptions, zero loss is often assumed under protective action, and a loss “L” is incurred if the adverse event occurs while no protective action is taken. No cost is incurred for an event neither forecast nor observed. One then sets up a contingency table of whether the event occurred or not, and whether forecast or not (and hence whether protective action is taken), and populates this based on samples of forecast and observed weather. It’s then possible to quantify, over many examples, the relative economic value of the weather forecast relative to reference forecasts such as climatology. This model has been used to demonstrate the appeal of probabilistic forecasts relative to the use of deterministic forecasts.

Using synthetic deterministic and ensemble forecast data constructed with perfect-model assumptions, this talk will expand upon previous studies in a few ways. First, the results to be presented will evaluate pseudo-probabilistic deterministic forecasts, using varying forecast values as decision thresholds in the same conceptual manner that different forecast probabilities from the ensemble are used as decision thresholds. Pseudo-probabilistic forecasts consistently provide more economic value than deterministic forecasts. Second, the results show that for some the cost-loss ratio of some users and some levels of innate forecast skill, both deterministic and pseudo-probabilistic forecasts may actually exhibit negative economic value, illustrating the reservations one should have in making decisions with deterministic forecasts and highlighting the appeal of probabilistic approaches.

Figure: an illustration of relative economic value for three prediction methods, using deterministic, pseudo-probabilistic, and probabilistic information in the decision making. Different panels provide results for different climatological event frequencies.