The areas within the Office of Energy Efficiency and Renewable Energy (EERE) use a variety of types of evaluation to quantify impacts, assess progress, and promote improvement. Several of these are listed below.
As noted in the Strategic Evaluation Planning section, the type of evaluations performed depend on the evidence needed and questions that need to be answered. This includes a consideration of questions that if answered are expected to help provide the organization or program with evidence it can use to improve how it does business.
Focusing on evidence needed, questions to be answered, and decisions to be made is an important first step in strategic planning of evaluations. The What and When to Evaluate section describes when these types of evaluations are expected to be performed.
Outcome Evaluations
Outcome evaluations measure the extent to which a program achieves its outcome-oriented objectives. It measures outputs and outcomes (including unintended effects) to judge program effectiveness but may also assess program process to understand how outcomes are produced. This type of evaluation determines change in outcomes for specified metrics, but without consideration of causality or the counterfactual.
Downloads of EERE outcome evaluation reports can be found in the EERE Evaluation Library.
Impact Evaluations
Impact evaluations quantify the extent to which outcomes, such as energy saved, resources/costs saved, changes in MW capacity, emissions reduced, and oil displacement, can be attributed to a program intervention. Benefit-cost assessments, a specialized type of impact evaluation, determine the "realized" economic benefits and costs of R&D and technology deployment programs using discounted cash flow analysis. Another type of impact evaluation is knowledge diffusion assessments which intellectual property formation and transfer, for example, as measured by patent citation rates.
Downloads of EERE impact evaluation reports can be found in the EERE Evaluation Library.
Learn more about the process for conducting an impact evaluation.
In-Progress Peer Reviews
Peer reviews are a form of process evaluation. In-progress peer reviews are reviews of program, subprogram, and project portfolios (project-level).
Peer reviews (also referred to as peer evaluation) seek to answer questions of how well an activity is being implemented and how to make improvements. For project-level reviews, EERE defines in-progress peer review as:
"A rigorous, formal, and documented evaluation process using objective criteria and qualified and independent reviewers to make a judgment of the technical/ scientific/business merit, the actual or anticipated results, and the productivity and management effectiveness of an Office's portfolio of projects." [EERE Peer Review Guide, 2016]
Examples of review criteria used in project-level peer reviews include approach, productivity and accomplishments; relevance to program mission and market needs; extent of necessary coordination and collaboration; and management.
Downloads of EERE peer review reports can be found in the EERE Evaluation Library.
Learn more about the peer review process for in-progress projects.
Stage Gate
Stage Gate reviews are a form of process evaluation that focus on technology and market readiness. They determine when a technology or activity is ready to move to its next stage of development. Stage gate reviews involve use of a series of "Gates" to review projects before moving forward, typically using external experts to assess the technical and business cases. Each stage of the process is designed to reduce levels of uncertainty and risk.
Similarly, "Down-Select" reviews winnow the R&D paths pursued and "Off-Ramp" reviews determine if the DOE job is done and the technology can be graduated or terminated. A "Go-No/Go" review is another similar type of review that examines projects or project portfolios at stages and decisions to continue or discontinue depending on if conditions set at a stage are met.
Downloads of EERE Stage Gate reports can be found in the EERE Evaluation Library.
Technical Program Reviews and Advisory Committee Reviews
Technical Program Reviews and Advisory Committee Reviews are conducted by senior management, technical teams, or advisory committees.
Market Needs and Effects Assessments
Market Needs and Effects Assessments measure aspects of supply chain and market infrastructure or structure that could affect the development and adoption of technologies. These include, for example, the following:
- Formation of R&D and business alliances,
- Changes in production capacity and effectiveness throughout the supply chain (e.g., numbers and profiles of manufacturers, installers, material suppliers, product distributors), and
- Changes in supplier or user behaviors.
These assessments provide information about the baseline of current practices and behaviors of market entities. They can also be used to identify market potential barriers, the most needy market segments, possible program partners, and also to quantify market effects (e.g., changes in the scale-up of supply and distribution chains, consumer awareness, and adoption decision practices).