How Forecast “Prototypes” Help Optimize Decisions (When Probability Forecasts Can’t)

Machine learning has proven to be an effective way to create skillful models for post-processing numerical weather guidance and fusing data from disparate observations to predict a wide variety of meteorological phenomena, including severe weather. Probabilistic forecasts are particularly powerful due to their ability to assist decision makers by helping to quantify risk and to gauge the cost of taking a mitigating action against the expected loss that would result from inaction. However, in many situations, weather’s impact is not a function of only a single weather variable at a given location and time, and this poses the challenge of how best to provide probabilistic models of weather impacts dependent on multi-variate and spatio-temporal weather phenomena. For instance, the risk of electrical outages depends not only on wind gusts but also soil saturation and snow or ice accretion; dangerous winter driving conditions are a function of road temperature, snowfall rate and wind speed; and the likelihood of excessive electrical usage can depend on the temperature, dewpoint and cloud cover over the preceding hours or even days. The predictive weather variables and their timeseries are generally not independent, making univariate probabilistic models inadequate for identifying an optimal decision that limits the risk of an unacceptable outcome and provides the greatest expected net benefit (e.g., in the examples above: whether to pre-position utility workers, close a highway or spin up an expensive reserve generator). An alternative that can directly address this need is a large set of representative, equally likely forecast realizations, each of which contains a coherent timeseries of all relevant weather variables and locations. While a similar set of forecasts can be obtained from a multi-model NWP ensemble or multi-method data fusion collective, such ensembles generally exhibit biases, under- or over-dispersion and unequal skill among members that make them unreliable for decision optimization.

To better enable weather-influenced decision optimization, The Weather Company, an IBM Business has launched a service called Probability Forecasts on Demand (PFoD). PFoD provides probability distribution function (PDF) and related descriptions of forecast uncertainty for several weather variables at any global location via an API. In addition, it uses a combination of machine-learned systematic error correction, calibration, and an adaptation of the Ensemble Copula Coupling technique to create a user-specified number of calibrated ensemble “prototype” forecasts that provide the multivariate weather trajectories required to evaluate nonlinear weather impacts and spatiotemporal dependencies. Coupled with machine-learned, heuristic or physically-based weather impact models, PFoD prototype forecasts provide a basis for quantifying risks and estimating expected values for prospective actions or policies. This approach has proven promising for several applications. Nevertheless, many opportunities remain for improving this approach and extending it to additional weather variables, timeframes, and extreme weather phenomena. The artificial intelligence community is well placed to embrace both the challenge of creating better prototype forecasts and refining their use in decision optimization. Successfully meeting these challenges holds the promise of significantly increasing the value that our weather enterprise contributes to society.