What to evaluate and when to evaluate a program depend on several interrelated considerations, such as:

• Stage of the program in its life cycle (i.e., creation of program, program underway, or at or near program closeout phase),
• Decisions to be informed by evaluative information during critical planning, budgeting, implementation, benefits reporting and communication cycles, and
• Any need to meet evaluation requirements. 
• An overall program-wide strategic evaluation plan should be developed to map out planned evaluation activity over a multi-year time frame, so the information produced:
  • Is timely, without being so frequent that not enough time passes to act on recommendations from the evaluation
  • In sync with the program's management and decision processes
  • Complements other program data collection activities.

What to Evaluate

There is no hard and fast rule on what programs or program areas should be evaluated. However, here are some circumstances that may suggest your program is in need of an evaluation:

(1) For a new program that is underway or the creation of a new program:

• Desire to understand if and how a new program or pilot works because you want to expand or replicate it later if determined to be successful
• New initiative with lots of uncertainties that need to be better understood to achieve the initiative's full potential
• A new initiative is a major budgetary priority, or addresses an important problem

(2) For an existing program that is underway:

• Program has never been formally evaluated, or has persisted year after year without adequate evidence that it is producing results
• Program activities cover large portions of the program's budget or are highly visible to major stakeholders for other reasons
• A need to re-evaluate an existing program where significant changes were made
• A program (or subprogram activity) appears to be in trouble (e.g., progress or participation lower than expected), or is experiencing better than expected success
• A program that has high potential risks or a complex design
• A program (or subprogram activity) where there is an interest in adapting it elsewhere
• A program that is a major budgetary priority, or addresses an important problem
• A program that has advanced to a stage where consumer and markets benefits should be accruing and the amount of economic return on investment should be determined, such as:
  • One that has potential for large benefits, or appears to be cost-effective, and the return on investment is currently not known;
  • One that was previously evaluated in an earlier period but had shown no significant realized economic return on investment, in part because accrued value (e.g., in the market) had not been established for its commercialized technologies during the period of the earlier study;
  • One previously evaluated but requires an update (e.g., two to five years later);
  • One that had performed benefit-cost calculations using an alternative approach not consistent with an updated benefit-cost assessment methodology; or
  • Congress directs an economic return on investment study be done

(3) For a program at or near closeout:

• Program that is phasing out or has closed out, where lessons learned and best practices could be identified and potentially applied to the development of a future successor initiative

When to Evaluate

Programs have critical planning, budgeting, implementation, benefits reporting and communication cycles. These elements of a program life cycle can be viewed in three stages –

• Creating a program,
• Program is underway
• Closing out or end of program.

In general, different type of evaluations are carried out over different parts of a program's life cycle.

Accommodating EERE, DOE, and OMB requirements where they exist, the following table illustrates where in the program life cycle offices should plan or consider doing an evaluation and types of evaluations that are sensible for each life cycle phase.

Generally speaking, selected program activities or the entire program should have some evaluation performed over a time period ranging from annually to every two or three years.