The Effects of the Use of Uncertainty Estimates on Weather-Related Decision-Making

Uncertainty is inherent in weather forecasts, so the inclusion of uncertainty estimates in forecasts might therefore be useful to end-users. Providing probabilistic uncertainty estimates could help the decision-making of different types of users with specific risk tolerances, and it might increase trust in forecasts, as forecasts might seem more correct if the observed weather events fall within a range predicted by the uncertainty estimates. However, from both a theoretical and practical standpoint, there is much debate about the degree to which non-experts can understand forecast uncertainty estimates and about the effect of uncertainty information on decision-making. Of particular interest are situations in which precautionary action is required at low probabilities, as is often the case in severe weather situations, such as hurricanes. There is a concern that under such circumstances, uncertainty may increase people's reluctance to take action. The studies reported here take a novel approach by directly comparing weather-related decisions made with and without uncertainty forecasts. In a series of three experiments, participants assumed the role of a manager of a road maintenance company in charge of treating a town's roads with salt during winter months to avoid icy conditions. Over the course of 120 trials, participants were presented with forecasts for overnight low temperatures that included either deterministic information alone, uncertainty estimates (expressed in several different ways), decision recommendations, or both uncertainty estimates and decision recommendations together. There was a cost to salt the roads and a relatively severe penalty for failing to salt when freezing temperatures were observed. Participants were given immediate feedback about the observed low temperatures and were paid commensurate with their performance on the task. Results suggested that although decision-making was suboptimal from a normative perspective, uncertainty information improved decision quality overall. Participants with uncertainty forecasts took appropriate precautionary action more often and withheld action more often when it was appropriate to do so than did participants using deterministic forecasts. When error in the forecast increased, participants were reluctant to act, an effect that was attenuated by uncertainty forecasts. Furthermore, acknowledging the uncertainty in the forecast increased trust in the prediction. Participants given decision recommendations performed no better than participants given only deterministic forecasts, and when the decision recommendations were combined with uncertainty estimates, participants' performance did not improve beyond having only the uncertainty estimates. As this study tests a low probability situation in which costs are involved and shows improved decisions, the results of the experiments carry important implications about efforts to increase compliance with severe weather warnings.