Project Name: Open Source Evaluation Framework for Solar Forecasting
Funding Opportunity: Solar Forecasting 2
SETO Subprogram: Systems Integration
Location: Tucson, AZ
SETO Award Amount: $999,364
Awardee Cost Share:  $145,524
Principal Investigator: William Holmgren

This project will create a framework to help solar forecast users, providers, and researchers assess solar irradiance and the accuracy of solar power forecasting. The framework will comprise a set of protocols that describe how to assess forecasts and a web-based service that verifies the forecast. The framework will establish standards for evaluating the accuracy of a forecast, automate the forecast evaluation process with vetted software, and generate reports with error metrics and graphics tailored for each use case. The framework will support projects in the Solar Forecasting 2 program by evaluating validated reference data and comparing the data to benchmark forecasts.


The project team will use a collaborative, transparent approach built on stakeholder engagement and open-source software development. The team will work with stakeholders, including solar forecast users, providers, and researchers, to obtain data to be used for forecast validation. Stakeholders will then guide the development, selection, and testing of core use cases, data-sharing protocols, evaluation metrics, and benchmark forecasts.  The project team will implement the recommended ideas in a reproducible and tested open-source software platform to allow stakeholders to fully vet the implementations.


The impartial, repeatable, community-vetted evaluations provided by the framework will promote better forecast development and allow for those improvements in forecasts to be accurately quantified over time. The framework’s evaluation reports will help end-users better understand the strengths and weaknesses of each forecast. This gained knowledge will reduce the barriers to forecast adoption and improve the application of forecasts, thus reducing solar integration costs. Both the protocols for data and forecast evaluation and the open source software will be broadly available and extensible to analysis of load forecasting and wind power forecasting.