The Solar Forecast Arbiter: An Open Source Evaluation Framework for Solar Forecasting

We describe an open source evaluation framework for solar forecasting to support the DOE Solar Forecasting 2 program and the broader solar forecast community. The framework enables evaluations of solar irradiance, solar power, and net-load forecasts that are impartial, repeatable and auditable. We demonstrate how the Solar Forecast Arbiter addresses stakeholder-informed use cases including evaluation of forecast skill, comparisons to reference data sets, private forecast trials, evaluation of probabilistic forecast skill, and examinations of forecast errors during critical periods. We discuss the framework’s data validation toolkit, reference data sources, and data privacy protocols. We describe the framework’s benchmark forecast capabilities for intra-hour and day ahead forecast horizons. Finally, we summarize the reports and metrics that communicate the relative merits of the test and benchmark forecasts. The reports are created from standardized templates and include graphics for qualitatively evaluating deterministic and probabilistic forecasts and standard metrics for quantitatively evaluating forecasts.