Open Source Evaluation Framework for Solar Forecasting

We describe an open source evaluation framework for solar forecasting to support the DOE Solar Forecasting 2 program and the broader solar forecast community. The framework will enable evaluations of solar irradiance, solar power, and net-load forecasts that are impartial, repeatable and auditable. First, we define the use cases of the framework. The use cases were informed by the project’s initial stakeholder engagement sessions and include comparisons to reference data sets, private forecast trials, evaluation of probabilistic forecast skill, and examinations of forecast errors during critical periods. Next, we discuss the framework’s data validation toolkit, reference data sources, and data privacy protocols. Then we describe the framework’s benchmark forecast capabilities for intrahour and day ahead forecast horizons. Finally, we discuss the reports and metrics that will communicate the relative merits of the test and benchmark forecasts. The reports will be created from standardized templates and include graphics for quantitatively evaluating deterministic and probabilistic forecasts and standard metrics for quantitatively evaluating forecasts.