Coupling Numerical Weather Predictions to Demand and Solar Energy Forecasting Models in an Operational Setting

Managing a reliable, affordable, and sustainable power grid is a complex task because the technologies connected to the network continue to evolve. In particular, the recent increasing penetration of distributed renewable generation is causing new phenomena like negative power prices and reverse power flows on distribution substations.

To help electricity network operators better manage their assets under these conditions, they need better visibility and enhanced forecasts of power flows on their networks, especially at lower aggregation levels, like substations and distribution feeders. Improved forecasts help network operators make better decisions, for example, when activating measures to avoid congestion such as demand response at times of peak load and low photovoltaic (PV) power production.

This paper presents results from a partnership between The Vermont Electric Company (VELCO) and IBM Research to develop a demand forecasting system. This system is currently operational and produces day-ahead forecasts of power demand, solar energy generation and residual (net) demand at over 100 nodes on the power grid of Vermont. Forecasts are updated twice daily as new inputs from high resolution weather models become available.

As inputs, our models use weather data from IBM Deep Thunder, a state-of-the-art high spatial- and temporal-resolution forecasting system, customizable to meet the needs of specific weather-sensitive business decisions, as well as measurements of temperature, dewpoint and irradiance from the Meteorological Assimilation Data Ingest System (MADIS) provided by the National Oceanic and Atmospheric Administration (NOAA). We use generalized additive models (GAM) to forecast energy demand and use a combination of physical models and GAM to forecast PV power production. Models are trained utilizing measurements of power consumption and generation from SCADA systems and smart meters spanning several years.

Several technical challenges are addressed including: 1) Data Ingestion from Numerical Weather Models: Training models for power demand requires several years of training data. One year of data from Deep Thunder for the variables of interest for demand forecasting is about 1 TB in size. We discuss how to manage these large volumes of data in model training.

2) Combining data from multiple sources: Weather data from numerical weather models is not always available for a continuous period of several years. We therefore use a combination of Deep Thunder and MADIS data for model training. However, there are systematic differences between weather data from different sources. We discuss how to combine data from multiple sources into a consistent training set.

3) Combining models for PV generation with models for demand: Measurements of power flow usually represent residual demand (sum of consumption and generation). We discuss how to combine physical models for PV generation with Generalized additive models for demand.

4) Forecasting uncertainty: Having a measure for the uncertainty in energy forecasts is essential. In addition to modelling the expected power flow through a network node, we also produce an uncertainty measure for our forecasts by modeling historical errors.

5) Automation: Building forecasting models for demand and generation at many network nodes requires automation. We describe how model training, anomaly detection, and model deployment are automated.

6) Changing PV Generation: PV generation capacity is not constant; by modeling ensembles of PV systems we can measure and predict trends in PV capacity and are able to make higher accuracy forecasts of PV generation as well as residual demand at high aggregation levels.

These forecasts are validated by comparing forecasts values to measurements from SCADA systems at over 100 substations in the state of Vermont as well as feedback from power system operators.