Consensus Forecasting using Constrained, Regularized Regression

Combining information from multiple forecasts -- for instance, via weighted averages -- has long been known to produce deterministic consensus predictions that are generally more accurate than any of the individual inputs, and numerous approaches have been proposed for optimizing the combination weights. However, no standard method has yet emerged. For weather forecasting, the problem is complicated by non-stationarity, geographical variability, frequent changes in the numerical weather prediction (NWP) models used to produce the input forecasts, errors in the verifying observations, and computational constraints.

In recent years, The Weather Company's 1-15 day ensemble consensus forecasts have been based on the Dynamically Integrated ForeCast (DICast) approach developed at the National Center for Atmospheric Research in the late 1990s. DICast uses stochastic gradient descent to adjust biases and combination weights for a pre-defined set of input forecasts when a new verifying observation is obtained. DICast is computationally efficient and requires little storage, since past observations and input forecasts are no longer needed after the bias and weight adjustments, and it adapts to changes in the relative accuracy of the input forecasts due to changing weather regimes or changing NWP models. However, DICast also has several disadvantages: (1) adding or removing input forecasts is not straightforward; (2) a missing input forecast requires either using an older forecast or setting its weight to zero, neither of which is optimal; (3) enforcing weight bounds or “preferred” weights is not naturally included; and (4) the influence of erroneous observations cannot easily be corrected.

This paper presents a fundamentally different approach -- a constrained, regularized regression technique that addresses these limitations and provides a more flexible consensus forecasting system with improved performance. The new system maintains a history of past observations and input forecast values, allowing optimal weights to be generated for exactly those input forecasts available at a given consensus forecast generation time. An exponential decay factor is used to discount the influence of older performance data, and input forecast error covariance matrices from neighboring sites and times may be aggregated to reduce the effect of random “noise” in the input forecasts and observations. Weight bounds and a “preferred” weight solution may be specified. The input forecast error covariance matrix diagonal is inflated to provide regularized regression (“ridge regression”) that mitigates overfitting. The technique produces a quadratic program for each site whose solution is the set of optimal combination weights to be used in the consensus forecast. Between forecast generation cycles, algorithm parameters may be adjusted based on the most recent performance data, allowing the system to more quickly adapt to seasonal, synoptic, or NWP model changes. Multi-season temperature forecast performance results for over 1200 sites in the continental U.S. are presented and compared to DICast and other methods, and the applicability of this approach to other forecast problems is discussed.