Evaluation of Intraday Distributed PV Forecasts with Customized Metrics based on Operational Grid Management Decision-Making Scenarios

Short-term generation forecasting is recognized as a valuable tool for managing the impact of solar and wind generation variability on electric grid systems. However, in addition to accuracy, the value of renewable generation forecasts in an operational environment depends on a number of other factors such as the shape and variability of the demand profile, the operating characteristics and cost of available non-renewable generation resources, and constraints imposed by the operating rules associated with individual generation assets. This causes the value of forecasts to the decision-making process to vary significantly within a day and between days. Thus, forecast performance metrics that measure the typical error, such as the Mean Absolute Error (MAE) or Root Mean Square Error (RMSE), often are not a good indicator of the value of the forecasts to the decision-making process and the reduction of integration costs.

In order to address this issue in a specific setting, a study was conducted on an island grid system to examine the types of decisions in which intra-day forecast information was used and the aspects of forecast performance that were critical to positive outcomes for those decisions. The venue for this analysis was the system operated by the Hawaii Electric Light Company (HELCO) on the Big Island of Hawaii. The system demand typically ranges from 90 MW to 180 MW. The system includes 31 MW of wind generation capacity, about 90 MW of distributed photovoltaic (PV) capacity and 16 MW of run-of-river hydro generation capability.

HELCO receives two types of wind and solar power production forecasts from a commercial provider : (1) 0-6 hour forecasts with a time resolution of 15 minutes that are updated on a 15 minute cycle and (2) 0-168 hour (7 days) forecasts with a time resolution of 1 hour that are updates hourly. The forecasts are provided in deterministic and probabilistic formats. The forecasts include generation from distributed (mostly residential and commercial roof-top) photovoltaic (PV) systems as well as utility-scale wind and solar generation facilities.

The forecasts are based on a look-ahead-time-dependent composite of predictions from a multi-method forecast ensemble. The ensemble includes statistically processed output from global and regional NWP systems from the US National Weather Service and Environment Canada and three customized local area NWP models run by the forecast provider as well as forecasts from satellite-image-based cloud tracking and advection algorithms and machine-learning-based time series models.

Three critical decision-making scenarios and time frames that are impacted by the variability of renewable generation were identified. These are (1) the pre-sunrise preparation for the morning net load peak and the subsequent PV-induced decrease of net load to the mid-day minimum, (2) the mid-morning positioning of the system to handle potential large ramps in the net load caused by ramps in the distributed PV production, and (3) the early afternoon preparation for the evening net load peak. The attributes of solar and wind forecast performance that are critical to each scenario are different.

A customized category-based, event-oriented forecast evaluation schemes was developed for each decision-making scenario. The objective was to provide more relevant forecast performance information to users and also incentive and guidance to forecast providers to focus on improving the most operationally relevant aspects of forecast performance.

The presentation will include: (1) an overview of the generation mix and load profiles on the HELCO system, (2) a summary of the components and data flow of the forecast system that produces the operational forecasts currently used by HELCO, (3) a description of the key daily operational decision-making scenarios and time frames with specific case examples of decisions, forecast inputs and outcomes, (4) an overview of the customized category-based intra-day forecast evaluation scheme for each scenario and (5) a comparison of intra-day forecast performance based on the customized evaluation scheme and a traditional set of performance metrics such as MAE and RMSE.

The key point is that one’s view of forecast performance and the value of forecasts is strongly dependent on the evaluation protocol and the metrics that are employed. Ultimately, an evaluation of the performance of forecasts for specific applications should employ metrics based on the application’s sensitivity to forecast error.