Optimizing Training Windows for Wind and Solar Generation Forecasting

Machine learning (ML) algorithms like XGBoost offer significant improvements in accuracy over linear methods for postprocessing of numerical weather prediction model output for forecasting of renewable generation and electrical demand. They are computationally efficient, allowing models to be constructed using large numbers of predictors and tuned using long training samples (many years). In this presentation, we review results from our multi-model ensemble forecast system to explore how trade-offs between skill and computational requirements can best be managed, and to explore some questions of NWP policy.

On the practical level, we will review results from our system on how the point of diminishing returns for training sample length and number of predictors changes depending on model characteristics and on the forecast location. We will also discuss the trade-offs between training on shorter forecast lead times (which have higher skill and less noise) and training on later forecast lead times (closer to the lead time for which the forecast will be issued).

As ML algorithms become more prevalent in the world of applied meteorology, and practitioners increasingly aim to extract more skill from NWP models through training with long time series, interesting problems arise when NWP models are changed and improved. We will explore these questions through the lens of the US Global Forecast System upgrade to the Finite Volume 3 (FV3) dynamical core, which presented our organization with an interesting series of challenges. Since long training samples are required to extract maximum skill, we needed to decide how to allocate resources between potential pathways. On the one hand, we could build a hybrid system in which forecasts issued based on the new model were trained using a combination of output from the old model and the new one, or we could train forecasts exclusively on the new model. The relative merits of these choices depended on how long the overlap period between the old and the new model would last, something that could not be known with certainty in advance.

For NWP prediction centers, there is a similar question of how much effort should be put into generating "reforecasts", to guarantee long training periods for post-processing, and how much into frequent model updates to capitalize rapidly on recent discoveries. We will explore how effective tests of these approaches might be generated to guide policy.