Benchmarking the Accuracy of Seasonal Forecasting for Renewable Energy Resource Anomalies

Forecasting on horizons of multi-month ahead time scales is becoming an increasingly important element of business process in the renewable energy sector.  After the record low wind production observed over much of the continental U.S. in winter 2015 caught many by surprise and the recent strong ENSO-positive (El Niño) event in late 2015 and early 2016 caused deviations from normal different to what was anticipated by the community, industry interest in the predictability of wind and solar generation anomalies has been growing. Understanding the seasonal variability of resource anomalies and the climate patterns to which they are related is an important first step (see companion abstract by Stoelinga et al. on examining wind variability). Projecting future power generation at seasonal time-scales is a second step, which capitalizes on the underpinnings established in the first step. While significant progress in seasonal forecasting has been occurring at the major operational weather centers like ECMWF and NOAA, with the continued development of state-of-the-art physical models that employ coupled land-ocean-atmosphere reanalysis and forecast systems, it is yet to be shown how these advances might translate into better monthly-mean estimates of renewable energy generation.

We conducted a benchmarking experiment to establish predictability expectations for 1-12 month horizon wind and solar resource anomalies. In our study, we selected a set of ten representative sites from North America, Europe, and Asia with sufficient data records in order to test the methodologies and their regional performance variations. We first constructed 35-year (1980-2014) reference time series for the wind or solar power resource at each location using global reanalysis data from the ERA-Interim and NASA MERRA-2 data sets, dynamically downscaled with the WRF numerical weather prediction model. Our target variable was an index defined by the ratio of the monthly-mean resource to the 35-year mean for that month. As candidate predictors, we selected 15 observed teleconnection indices and their lags, 6 forecast teleconnection indices from the NOAA SST Linear Inverse Model, and ensemble-mean point forecasts of the wind or solar resource from the System 4 ECMWF Seasonal forecast.

We evaluated forecast performance both as a standard regression problem and as a 2-class problem with above and below normal categories. A range of statistical algorithms were tested, including linear and logistic regression, k-nearest neighbors, support vector machines, and tree-based models. The forecasts were expressed as probabilities for each class and performance was scored using standard contingency table metrics summarized by the area under the ROC curve (AUC). Higher skill was observed for solar sites compared to wind sites, and for 3-month rolling mean periods (JFM, FMA, … DJF) compared to single months. We segregated statistics by the time of year and noted when the forecasts have relatively more skill than for the whole. As a result of this exercise, Vaisala delivers seasonal forecasts for wind and solar projects along with context from the historical performance of each method, so that customers know when and where anomalous wind and solar resource conditions are likely.