Reproducible Forecast Evaluation with the Solar Forecast Arbiter

We describe an open source framework for impartial, repeatable, and auditable evaluations of solar power and weather point forecasts. The `solarforecastarbiter` core package includes the data validation toolkit and forecast evaluation functionality, along with functions to generate reference irradiance and solar power forecasts from NCEP models. It depends on standard scientific Python packages (numpy, Pandas, xarray) and is designed to be extensible for other forecasting communities. We also describe how the core package fits into the larger Solar Forecast Arbiter framework which includes a web API (https://api.solarforecastarbiter.org) and web dashboard (https://dashboard.solarforecastarbiter.org), and how each component interacts. The Flask/Bokeh dashboard supports a number of use cases including evaluating forecasts against reference data sets, continuous forecast trials, probabilistic forecast evaluation, and report generation including interactive Bokeh figures. The framework will be tested and refined as users evaluate forecasts as part of the DOE Solar Forecasting 2 program.