EECLP Webinar #2: Quality Assurance and Evaluation Monitoring Verification -- Text Version

You are here

Below is the text version of the EECLP Webinar 2: Quality Assurance and Evaluation Monitoring Verification, presented in December 2014.

Odette Mucha:
First slide:

Hello, everyone, and welcome to the webinar. This is a six-part series on the Energy Efficiency and Conservation Loan Program. We're very excited to have you with us today. We have a great lineup of experts who will be walking us through quality assurance and evaluation, monitoring, and verification. A couple logistics before we begin. You all can post your questions in the chat box on the right side of your webinar. We'll be taking questions at the end.

Next slide:
Our first speaker today is Gerry Moore from the Rural Utilities Service, and he's going to give us an explanation of the Energy Efficiency and Conservation Loan Program, as well as the quality assurance requirements. Take it away, Gerry.

Gerry Moore:
Next slide:

OK, thanks, Odette. Hello, my name is Gerry Moore. I am with the U.S. Department of Agriculture. I work in the Rural Utilities Service, called RUS. Specifically, I am the acting deputy assistant administrator of the Electric Program. I will discuss with you today a new eligible activity for RUS loan funds. We will discuss the new Energy Efficiency and Conservation Loan Program, also called EECLP. Specifically, we will talk about the quality assurance plan associated with EECLP. I was heavily involved in the writing of the program regulation. The slides I will present today specifically address requirements associated with applying for an EECLP loan. Other presenters will discuss their personal experiences, and DOE will discuss some best practices of efficiency EM and V. Those practices are not necessarily required for the RUS loan. The best practices are, however, excellent materials to review and decide if they are right for your program. Next slide.

Next slide:
So to give some background here on EECLP. The 2008 Farm Bill made energy efficiency an eligible purpose for RUS loan fund. We drafted an associated regulation to implement this, and this regulation is now final, and loan funds have been available since February 2014. The regulation now allows Electric Program borrowers to relend our funds to their consumers to implement energy efficiency upgrades. These consumers may be residential, commercial, and/or industrial. These efficiency upgrades are to be located on the consumer side of the electric meter. These upgrades must also be undertaken in the borrower's service territory. Eligible efficiency upgrades include HVAC upgrades, lighting, ground source heat pumps, and load modifiers such as renewables, like solar installed on the consumer's side of the electric meter. But we don't dictate the type of efficiency upgrades our borrowers undertake. The regulation is very flexible. We seek to enable our borrowers' efficiency ideas, not dictate them. Next slide.

Next slide:
OK, a typical RUS borrower's energy efficiency program would have the utility relending funds to their consumers. The utility can charge an interest rate above the cost of money from RUS. That interest rate, on top of the cost of money, is presently limited to 1.5 percent. So total interest rate is the federal financing bank rate, plus one-eighth percent, plus 1.5 percent. That would be total. On-bill payment of the consumer loan is a very popular option. Loan terms are determined based on useful life of the efficiency upgrades. Terms can be up to 30 years. The loan requirements for energy efficiency are similar to our existing program, but we do ask for some additional requirements from our borrowers that seek these loan funds. That is, an energy efficiency business plan, and what we'll talk about today in a bit more depth, an energy efficiency quality assurance plan. Any utility that has an existing energy efficiency program should have documents similar to these on hand already, we believe. Many borrowers, though, are small utilities with limited resources. The larger the borrowing entity becomes, such as a generation and transmission co-op, the greater the economies of scale achieved when establishing or running a program. Next slide.

Next slide:
Let's talk about who is an eligible borrower. Let me make this clear. We have constraints on who we can lend to, based on statute and regulation. Eligible borrowers should be 1) an entity in the business of providing direct or indirect retail electric service to consumers in rural areas, 2) an entity in the business of providing wholesale electric service to distribution entities, 3) an entity in the business of providing transmission service to distribution or generation entities providing services to consumers in rural areas. Next slide, please.

Next slide:
OK, now let's talk about a requirement for an EECLP loan called the quality assurance plan. This plan contains any requirements associated with measurement and verification. This is a global perspective on a borrower's energy efficiency plan. You should address efficiency upgrades in an aggregate form. So you don't need to address each individual efficiency upgrade you plan on implementing. You can just address them by category, for instance. Like ground source heat pumps, HVAC systems, etcetera. We want you to know -- we want to know that you have thought out the quality associated with your program thoroughly, to ensure the end user gets the type of savings envisioned, and the systems were installed and are working properly. OK. What should a Q and A plan contain? Well, we want to see that qualified energy managers or professional engineers were used to evaluate the program activities. How many when looking at an overview of the whole program. We want to know one of those types of professionals has reviewed the program. Energy audits shall be performed for these EE investments that involve building envelopes, let's say. Energy audits are a part of a quality program. Energy audits must be performed by certified energy auditors. We want to see that you're using them. We want to see that you're using good quality auditors to perform these functions. Follow-up audits are something that we require. They should be performed within one year after installation on a sample of the investments made, just to confirm whether these efficiency improvements are really meeting expectations. We don't dictate the percentage of follow-up audits. We leave that up to you. There are some good numbers out there. Some do five percent follow-up, etcetera, but we're really leaving that up to you. And you can specify that in your quality plan. Where applicable, program evaluation activities should use DOE's uniform methods project protocols for determining energy savings. Next slide, please.

Next slide:
Let's talk about energy audits. What is an audit? So I'll read you some definitions from the reg. It is an inspection and analysis of energy flows into a building process or system with the goal of identifying opportunities to enhance energy efficiency. It should result in an objective, standard-based technical report containing recommendations for improving the energy efficiency. The report should include an analysis of estimated benefits and cost of pursuing the recommendations and in a simple payback period. There are different types of audits. There are audits for residential buildings, and commercial and industrial applications. Next slide.

Next slide:
Now, I want to clarify in the reg the definition of a qualified auditor. I'm going to read this to you here. An individual possessing a current commercial or industrial energy auditor certification from a nationally, industry- recognized organization. A licensed professional engineer in the state in which the audit is conducted with at least one year experience who has completed at least two similar types of audits. An individual with a four-year engineering degree or architectural degree with at least three years experience who has completed at least five similar types of energy audits. An individual with an energy auditor certification recognized by DOE's Better Buildings workforce guidelines. You'll hear more about that in the webinar series. But there's flexibility here. There's a number of choices for a qualified, and there's differences for commercial and industrial energy efficiency improvements. Next slide, please.

Next slide:
So let's talk qualified energy auditors in the EECLP for residential improvements. The individual should be one of the following -- and you can see those. Just one or the other. You can read those definitions there. I don't need to read those for you. But there's clarification in the regulation on these points. Next slide, please.

Next slide:
OK, a Q and A plan for programs with single system upgrades. If the EE measures involve upgrades to a single system, the new system should be designed and installed by certified and insured professionals that are acceptable to the utility. Industry or manufacturer standards performance tests indicating that the installed system is meeting its designed performance parameters. We want to make sure you're really doing a quality job here. Next slide, please.

Next slide:
So consider in the quality plan when the EE program involves independent contractors. The utility should monitor the work done by the contractors and confirm that the contractors are performing quality work. The utility should remove substandard contractors from their recommended list if the subcontractor fails to perform at a satisfactory level. And what we've seen with a lot of the co-op programs out there, is they'll provide a list to their consumers of qualified contractors, and we're really asking that you manage the list. That if you hear consumers' feedback, they don't like certain installers, then we want to make sure you're addressing that and adding and dropping contractors that way. Contractors not hired by the utility may not act as agents of the utility in performing work financed with the EECLP. I want to tell you about some uptake that we've recently had on the program. We've seen samples of business plans and quality assurance plans come in from cooperatives, and they've really been pretty good. So what we have seen is, we've had two cooperatives come in and get loans, standard routine cooperatives. They've come in and gotten loans, produced very good plans, and they have -- some of them have already had a robust EE program going on. So it wasn't hard for them to produce this document. But we've also seen, behind those first two, those first two leaders, we've seen like four to six other cooperatives come in with plans. And it is typical that they show us their draft plan, and we go back and forth and discuss progress and how to adjust the plan so that it will be accepted. So if you are intending to get an EECLP loan, please do that. Reach out to us, and then we can discuss what kind of plan you already have and how we might calibrate them so that they fit into the reg. Don't be intimidated by this. Having an energy efficiency program really serves your customers. It's something that helps them save money. It really may even help you retain your consumers. I know I've heard of cooperatives that were concerned that energy efficiency programs could eat into their profits, but in reality, even a robust energy efficiency program is targeting one to two percent penetration rate in a year, and we don't believe this really is going to cut into revenues to any great extent. As far as uptake, we also have nontraditional borrowers coming to see us. We have borrowers in some cases that are looking at getting together a number of cooperatives all under one energy efficiency program and coming in for a loan. The entity would then do the work for a business plan, quality assurance plan; they would do that as a larger entity, and then they would come in for a bigger loan, and then many co-ops under them. So a lot of that kind of activity is taking place, and we're negotiating with them and helping them along. So we do have some good uptake, and it is moving along. And we've had some good cooperative leaders step up. So I've just given you an overview of EECLP. There's so much more to discuss. If you have a chance, please take a look at our website for more information, and there's a lot out there. Or you can call me, Gerry Moore: 202-720-6285. And reach out to your general field representatives, if you're aware of who that is. Thank-you.

Odette Mucha:
Alright, thank-you so much, Gerry. Thanks, Gerry. This is Odette again, and, as I was mentioning before, we're trying to get more voices from electric cooperatives included in the webinar series. Up next we have Alan Shedd, who is the director of energy programs at Touchstone Energy Cooperatives. And we're also hearing from Pat Keegan, the CEO of Collaborative Efficiency. Take it away, Pat and Alan.

Pat Keegan:
Yea, thanks, Odette. Thanks to the Department of Energy and to RUS for letting me and a few of my colleagues help out in this important webinar series. I've been working on energy efficiency for a long time, for over 30 years now. So has Alan. But for me, it's just been the last four years or so that I've been focused on co-ops and also on financing.

Next slide:
I think the co-op world of energy efficiency is very exciting. There's tremendous room for innovation. There's great leaders out there. There's a wonderful focus on the energy users, on your members. So it allows us to do a lot of things you just can't do in other kinds of energy efficiency programs. I've been working with my colleagues at Collaborative Efficiency over this last year and a half or so writing about energy efficiency and talking to co-ops around the country for the publications that we do for the cooperative research network. So we spent most of our time talking to co-ops about efficiency or talking to reviewers from co-ops that are reviewing the publications that we're putting out, so we feel like we've really gotten ourselves immersed in this. And we've worked directly with co-ops in South Carolina and Kansas and North Carolina, and in Washington and Oregon state. So we're getting a good feel, I think, for how different co-ops are, among themselves, on energy efficiency. Some are very sophisticated. Many of them are really trying to just get started with something, so this overview that I'm going to do on quality assurance really is kind of an introductory level. Some of the co-ops are going to know all of this already, but I think everybody's going to find something useful, we hope, in the next 20 minutes or so. So the big difference we mentioned earlier, that EM and V -- evaluation, monitoring, and verification -- is a real common kind of approach to large best-run utility programs that have regulators and even state and local government programs. It's not something that most co-ops are concerned about. But today we broadened the topic somewhat to quality assurance, because the EM and V history among all the programs in the country provides some really good tools for quality assurance. So later on you'll be hearing a little bit more about kind of the traditional EM and V for these larger entities. Next slide.

Next slide:
So Gerry talked about what's required in the EECLP, and he knows that stuff inside-out. What I just wanted to make a comment on here is a little bit about what that is, and then we'll talk about some of the essential items in quality assurance, and a few of the elements of quality assurance. But the purpose here is just to gain an understanding of the importance of QA and some of the QA tools that you have at your disposal. So let's go to the next one.

Next slide:
So the QA plan, as Gerry said, is a required part of the EECLP application. It's kind of a high-level perspective, but he talked about the elements that would be in it. But I just completed a QA plan for Inland Power in Washington state, and submitted that -- that part of ourapplication has been approved. It's a little over two pages. So it's really not an onerous part of the application at all. I found that the people from RUS have been really reasonable about this and have a lot of flexibility. I think that the way you can use the QA plan is really to do what's right for your own members. It's a very flexible thing. There's some key provisions that Gerry mentioned, you know, about using qualified people and conducting audits, keeping up with some kind of industry standards. But it's not a -- it doesn't have to be a really complicated thing. A two-page part of the application I don't think is that at all. Next slide.

Next slide:
So Alan is going to talk about some of the other important things about quality assurance. Now, Alan, do you want to introduce yourself?

Alan Shedd:
Thanks, Pat. I'm Alan Shedd with Touchstone Energy. I manage and develop energy programs for Touchstone Energy's 750 member co-ops across the country. And what I want to do is talk a little bit about QA and why it's important, and at the risk of being a little repetitive, I want to share with you sort of my views from sort of the co-op side of the fence. And really, at its key, QA, as the name suggests, is really about assuring quality programs for your members. We obviously have that special relationship with our cooperative member consumers, and we want to guard that. So if a co-op is putting together a loan program, it's important to have high standards so that the program actually works. Frankly, all energy efficiency programs aren't created equal. Some of them produce good savings. Some of them are somewhat questionable. Some of them go on for years. The scope varies from project to project. So it's really important when you look at a quality assurance program to really think of it in the context of, well, what are the programs we're doing, and what is really appropriate for this use here? And just to back up a little bit. Gerry talked about the requirements of the QA plan, as far as the EECLP, and Pat sort of provided an overview of the EM and V versus QA, and what it means from a definition. I want to make sure that -- just to kind of go on record saying EM and V is a good thing. I would recommend it where it's appropriate, because if you're looking at a new program or a measure that you don't have a lot of experience with, there's not a lot of data to back it up, it behooves you to dig in a little deeper and sort of understand what those measures are. If you've got the budget or the staff available to do it, that's great. And it really ought to be part of this overall QA approach. Also, obviously if you've gotten more rigorous tracking requirements or savings or carbon requirements, EM and V gives you that hard data you need. On the other hand, QA, you know, is something that we need to do to make sure we provide a good quality product that delivers on the promise and on the need. To go back to the slide here, reasons for doing QA. Frankly, if you're going to loan the money, wouldn't you like to know that the savings that you promised are actually being realized? I mean, that seems pretty basic. If the loan program were looking forward to getting that money back, we want to make sure that the measures measure up. And it's also about member satisfaction. And this is satisfaction on several levels. Comfort. If you do weatherization or insulation, for example, is the member going to be more comfortable? What health and safety concerns are there, if we seal a house, if we do very rigorous air-sealing, we need to make sure that we're providing adequate ventilation, that we take into account, you know, proper ventilation rates and code requirements there. You've got the co-op reputation to think about. We co-ops have a valued position. Our members look up to us, and we need to maintain that. And then, of course, managing the contractors as Gerry said. We need to make sure that the contractors who are doing the work are doing a good job, are delivering, you know, what we promised. And are going to be there to answer for their situation. Let's go to the next slide.

Next slide:
So Pat, I think, let me turn this back over to you a minute.

Pat Keegan:
Thanks, Alan. You'll be coming back on later, here. So, Alan and I talked about what we thought were some of the important elements of quality assurance. And really, every part of a program should be covered by some kind of quality assurance, because they can all affect the outcome. But there's a number of ones that we thought were kind of most important to think about in this kind of quick overview. So we're going to walk -- I'm going to walk through a couple of these, and then Alan's going to take over. OK, next slide.

Next slide:
So the energy audits. OK. Did we skip one? There's one on program design decisions. Maybe that's after this. I'll talk about energy audits. So energy audits -- well, let me say something about program design decisions first, because that's really the overarching area of quality assurance. It's the one step in quality assurance that you take that will cover kind of everything in the program. And it really builds the framework for everything you'll do after that. It's -- a lot of decisions have to happen during that program design part of the step. And it's really good to have some people involved that have done something with programs, that have either designed programs or hopefully they've also operated programs, evaluated them. And some co-ops have some qualified staff to do this all themselves. But what I found is even with the large and best-run utilities that have dozens of staff, they often hire outside help for program design. I think the most important thing is you make it just a really conscious step that you're taking. It's a really small investment, when you think about it. Even a small co-op over a period of years can be getting millions of dollars worth of measures done that will result in millions of dollars worth of savings. So for a pretty small investment, you can do the program design and to make sure you're kind of heading the direction that you want. And you can decide kind of who to involve. You want to build on what you've done before, of course. You might also want to build on something that's happening in a neighboring utility, or something you can do with other co-ops. And there's lots of experience to draw on, as program design is put together. And you can address quality assurance in each one of these.

So one of the things you'll do in the program that's in the design is work on energy audits and figuring out what you're going to do there. And the range of what constitutes an energy audit is pretty big. There's, you know, half-hour walk-through audits in homes. There's -- some commercial audits cost over $10,000 and take weeks to do. But what it comes down to is really, what are your different goals that you have? And there's lots of different possibilities here. A lot of the co-ops we've talked to -- most of them, I think -- provide some kind of audit. For example, just to resolve high-bill complaints. Other ones that have programs, the audit is simply a manner to identify some prescriptive energy efficiency measures. Or maybe they want to comply with a larger program that's being run by their GNT or by a federal marketing agency. One important goal for most loan programs, I think, is to have an audit that will actually estimate what energy savings will be. Because you're going to be concerned that those energy savings are going to help pay for the loan repayments for your program participants. But audits can also identify comfort problems, health and safety issues. And the audit is more than just the on-site work. You know, there's an energy analysis component of this that you can do before you even show up on-site. And the modeling of the building with simulation software. It's really common now. There's some good software packages out there. It's a critical tool to estimating energy savings, and figuring out how you can get the data in and come up with a sound estimate of what that building is going to save, I think, is just an important step for loan programs. And of course, most audits produce some kind of audit report after the on-site visit. So there's a lot of things happening besides just the on-site work. And then the tools, besides the building simulation software, you know, there's also blower doors and duct blasters that you use to really test the home for air leakage or for duct leakage. Doing an audit right means you need to have auditors with the right qualifications. There's way more auditors now with the right qualifications than there was just a few years ago. You'll want to consider, OK, what is that audit going to cost? As a co-op, are we going to provide some incentive or support for it? Are we going to allow that audit cost to be wrapped into the loan? So there's a bunch of things you can consider on energy audits.

Next slide:
Alan's going to talk about the next slide, and a few of the next ones.

Alan Shedd:
Good. Thanks, Pat. Now obviously, one key thing you've got to decide through this project is, where are you going to get the help? Where are the contractors coming from? How are you going to manage them? And really, this of course is a no-brainer, but the reason we bring it up in this context is, while you need to think through that, it also needs to be part of your quality assurance plan. You need to sort of outline what your thinking is here, and what you plan to do to make sure, you know, that these contractors are delivering on your promise. So the type of contractor you need obviously depends on a lot of factors. Certainly the nature of the project, the scope of it. If you're simply going out and replacing lightbulbs with LEDs, you know, that's not very rigorous. If you're installing ground source heat pumps, installing PV or doing whole-house retrofits, you know, you need a higher level of competency there. Some other considerations are who's actually running and managing the project. Is it done at the distribution co-op level, or is there a bigger group, you know, a statewide, or a G and T who's sort of heading things up. Because they may have more resources or be able to tap into additional resources. The smaller distribution co-op might not. So, of course, most co-ops have good relationships with local contractor networks and trade allies. That's just part and parcel to what we do. So by all means, you're going to rely on those. Many co-ops have preferred contractor lists. But, in looking at that contractor list, the list needs to be appropriate for the job at hand. Just because you've worked with, you know, Jim Smith installing air conditioners for 20 years, doesn't mean that he necessarily is well-qualified to do this other part of the measure. So you need to do your due diligence there. Frankly, you know, the other big question we get a lot is, well, where do I find these people? And I want to remind you that we are cooperatives, and going back to cooperative principle 6, you know, cooperation among cooperatives. There's a tremendous resource available to us through the network of cooperatives. You're not going this alone. There are other people who've done it. There are other people who've done projects like this. They can be of tremendous assistance to you. Of course, also, be sure to look at the rules and regulations under the EECLP, you know, from RUS. Let's go to the next slide.

Next slide:
So regardless of the scope and the people we're using, training and education is key. I can't emphasize this enough. Obviously, the level of training is going to depend on what you're doing. Again, if you're doing a simple change-out kind of thing, maybe you're just doing a HVAC tuneup that you've already done before, maybe you're changing out filters, some basic weatherization, it may be sufficient to simply do a half-day program orientation that provides your contractor network with the basics about what you expect from them, how the program works, what are the forms they need to fill out. But then carrying that further, and these are enumerated in the QA requirements that Gerry shared with us early in the presentation, you may need to get someone from Building Performance Institute, somebody who's a building analyst. Maybe you're doing HERS ratings, so you need to have a certified rater on the commercial side, certified energy manager is one of the key here, or a professional, depending on the scope. HVAC certifications, obviously there are several there. Solar, same thing. There are a lot of different groups that do training and certification, some more rigorous than others. But you know, reach out to your friends and see what they're using. And then of course, follow up is really important here. It's not a once and done thing. Don't think that, well, we did training back at the start of the program, and that's sufficient. You need to monitor what's going on. You need to see if maybe they're not understanding something. Maybe there's some changes in the scope that you need to reiterate. So training is really built into this thing and goes through it, throughout. How about the next slide.

Next slide:
So Gerry mentioned post-project inspections, that it's a good idea to go back to the site and see if it was done right. We have program standards. We know what we expect to be done, but frankly, how do you know it's been done right unless you go back and look? Well, the obvious question is, are you checking every project? What are you looking for? How detailed is this? What's required? You know, the requirements under the plan are fairly loose in this. There's some guidelines, and I would encourage you to look at that. You don't have to go out and test everyone. But I would encourage you to, you know, practice some common sense here. If you're having a lot of problems, you may want to tune up your site inspection, to make sure you're getting your contractors on track. Things are running smoothly, maybe you can back off of that a little bit. So think about it as a dynamic thing. Also, it's not just about surveying the job from a technical perspective. I want to emphasize that it's also important to look at member consumer satisfaction. Are they happy with what they got out of the job? Because ultimately, again, that reflects on us as co-ops. So you know, yes, it's important to make sure that the heat pump was installed properly. That it's working properly, that it's saving them money. But is the guy who is paying that loan, is he happy with the outcome? Did he think the co-op did right by him? So that's an important thing I think to look at. Next slide, please.

Pat Keegan:
Next slide:

Yea, Alan, thanks. So, just quickly, there's different types of program evaluation. There's a real informal sort of program review. I did one last year for a co-op in the Midwest. They were just trying to figure out how to make their program better. And then there's more formalized evaluation structures that you can get all kinds of information on. Process evaluations or impact evaluations. DOE's got a lot of that information. Next week, in fact, you'll be hearing something about an impact evaluation in the South Carolina programs from Lindsay Smith, where they really looked hard at figuring out whether the projects were really saving money. So, more to come on that. And you'll hear more from Mike Li at the end of this presentation. Let's go to the very last one, and just so you know, last slide ...

Next slide:
So, just for more information, there's some contact information there. DOE is good about putting these things up; these webinars are recorded, and you can access them on their website. I'm sure that we'll tell you about that. And I just did want to mention that there's a group, the Environmental and Energy Study Institute, that has a team, and I'm on the team. You'll be hearing from Lindsay Smith, who's also on the team, next week. And they're available to help co-ops pursue on-bill financing. And there's some contact information there. So that's it. We can move on to our next speaker. Thanks, Odette.

Odette Mucha:
Great. Thank-you so much, Pat and Alan.

Next slide:
Up next we have Michael Li from the Department of Energy, going into a little bit more details on evaluation, monitoring, and verification. Take it away, Mike.

Michael Li:
Thanks, Odette. So I'm going to be brief, and this is really just going to be a very basic introduction to the topic of EM and V. And so most of what I'm going to present today comes from a resource that you can get online from our website, and the name of that document will be posted on the last slide. So this is going to be brief, and for more details, go ahead and look at that document. Next slide, Odette.

Next slide:
So we touched on this a little bit, but you know, why are we evaluating programs? And you know, I'll just go through the first two reasons. One is to just document what kind of impact are the programs having. Are they actually saving energy? What other sort of impacts are these programs having? The second reason to document, to evaluate programs, is really to sort of get a sense of, you know, if you're buying electricity, are you also implementing an energy efficiency program? Knowing how much your efficiency programs are saving will also help you plan for how much electricity you need to buy in the future. And so that's another good reason to try and really get a good sense of how much energy is being saved. Next slide.

Next slide:
So we just mentioned this briefly, so I won't belabor too much. But impact evaluation is really about trying to determine how much energy was saved. And then process evaluation, you can think of it as you know, are the programs being run well? Did people like the contractor interaction that they're getting? Are the contractors doing good work? Are there things that we should change in the way that we implement our programs so that people will either have a better experience or more people will participate in the project? And then, the last one is market evaluation, and that's really just, if you're a big co-op or work in a big market and you're basically doing a market evaluation to try and see if your programs are having an impact on the things that like stores or Home Depot would stock on the shelves. Just to see if that changes over time. And you're transforming that market to be more efficient, with or without programs that you're running. Next slide. You can go ahead one more.

Next slide:
So I'm just going to talk briefly about some of these terminologies that some of you might hear about or might be tackling in the future. And so I realize that not all of you are really going to be dealing with this, but for some of you, this might be a good introduction to how we use some of these terms. So, gross savings really just refers to, when you go from incandescent lightbulb to LED lightbulb, that change in energy consumption is generally considered sort of the gross savings. As you can see there, gross savings is the change in energy consumption that results directly from program-promoted actions taken by program participants. And the key piece is this last phrase, which is regardless of why they participated. So net savings really addresses the issue of why did they participate. And so let's say you have a rebate, and somebody was going to the store, and they were going to buy a new ENERGY STAR® refrigerator anyway, but they happen to see a rebate in the mail as they were leaving the house. And they went ahead and used that rebate to get a lower price for the refrigerator. So that would be an example of somebody who participated in the program, but they were going to buy the ENERGY STAR refrigerator regardless of whether or not there was a coupon there or not, or a rebate there or not. And so in some cases the savings from those kinds of participants in the program get backed out because the savings is not directly attributable to the program. So it's a little bit of a mouthful, but hopefully you sort of get the general gist. And then for some utilities and some programs, we had time to calculate sort of the non-energy benefits, so jobs created, things along those lines that you know, aren't necessarily energy savings but do benefit the community. Next slide.

Next slide:
So again, a little bit more on sort of the terminology. I'm going to skip the first one. But claimed savings, that's generally when you know, whoever your contractor is, or implementer of the program, they say they went out and installed new efficient lighting in a hundred different businesses, and they're going to tell -- they're going to estimate for you how much energy they saved. And so that's sort of what we refer to as the claimed savings. And then evaluated savings, that's really when an evaluator goes back and double-checks the contractor's work, and then they will tell you what their estimate of the savings is relative to what work was claimed. And so that's often referred to as evaluated savings or ex-post-evaluated savings, so "ex-post" meaning, after the measure was installed. Next slide.

Next slide:
So this comes up a lot as the term deemed savings. So deemed savings is potentially something where parties are agreeing to say that a certain energy conservation measure results in an X number of kilowatt hours of energy savings. So for example, it might be if you went from an incandescent bulb to an LED bulb, you know, you would just say that you get, for every one that you change, you get 100 kilowatt hours a year of savings. And so that's deemed savings value, you know. It's generally prescribed for a bunch of different energy conservation measures. And in most cases and hopefully in all cases, the way that people arrive at that deemed savings value is a result of some previous measurement and verification of what the actual energy savings are. And so this is an easy way, it's a very inexpensive way, of sort of trying to estimate what the savings are, from different programs. So, a related term is this term stipulated savings value. And so a stipulated value is generally something like, in order to determine energy savings for something like a lightbulb, you also have to know how many hours a day or a year that lightbulb is on, in order to be able to calculate what that energy savings is. So a stipulated value might be something like a lightbulb is on in a living room for eight hours a day, and so any time you have a lightbulb being replaced with an LED in a living room, that stipulated value for hours of use would be eight hours. So there's stipulated values for a wide array of different sort of parameters and variables that are necessary for determining what energy savings are. Next slide.

Next slide:
So I think this is my last slide, and this is really just to sort of give you an idea of the different kinds of EM and V activities that are out there. And so you'll have to sort of figure out which one is most appropriate, if any, for your utility. And so they all sort of get into varying levels of PTL, but in any case, like a quality assurance plan, you know, it's probably a good idea to have some type of EM and V plan, which sort of documents what are or aren't you going to do as it relates to EM and V.

Next slide:
So the last slide, which we'll switch to, really just has contact information and lists some of the different resources that we have, which is in the bottom left corner. So if you just Google those different things, you'll be able to come up with them pretty easily. So, with that, I'll wrap it up and we'll have questions.

Odette Mucha:
Thanks so much, Mike. Now, really quickly, back to Gerry Moore from USDA.

Gerry Moore:
Hi, thanks. I just wanted to stress a few points, and I wanted to just stress that, as Pat mentioned, a really good point, I think, that approved quality assurance plans have been two pages long. It's just making sure that you thought through quality and how it impacts the program. After all, as Alan mentioned, it does reflect on the cooperative and the EE program itself, how successful it could be.

Next slide ("Questions"):
If you have a preliminary quality assurance plan, we invite you to share it with us, and we can talk it through, so don't be intimidated by it. Also, we want to hear your feedback about content of these webinars, to make sure we can calibrate it to your needs. That's all I had, Odette.

Odette Mucha:
Alright. Thanks, Gerry. So now we have a few minutes for questions. And I have a few that have come in so far. So to everyone on the line, if you have a question, feel free to type it in to the box on your dashboard, and we'll cover as many as we can. And any questions that we don't have time to answer, we can follow up over email. Alright, so here's the first question. And here we go. Many homes needing energy efficiency upgrades require repairs to ensure energy efficiency upgrades work. For example, a leaky roof or an attic insulation. So do these loans assist with getting the home ready to be weatherized?

Gerry Moore:
This is Gerry. So yes, that's a possibility. I think we refer to it in the reg as pre-upgrades. Other activities and investments as approved by RUS as a part of the EE program, but not limited to. I think we call them pre- retrofit. So we know that in older stock homes, and we've seen many, you can't insulate a wall that's not sound. So you might have to rebuild the wall. So the answer is yes. Put it in the business plan. Let us talk it through.

Odette Mucha:
Great. Is there a limit on the size of a small scale renewable system?

Gerry Moore:
OK, if memory serves me, originally we had limits there. We didn't want a lot of generators out there, because that's not the purpose of this. We didn't want a lot of generators, people generating from their homes. I don't think the utilities like that, either. But we had placed restrictions on it. But some in the comment period came back and said, don't put limits on these. So as it stands right now, we don't have limits on that. I mean, it's not for renewable generation and you're building utility scale stuff, obviously. It's behind the consumer's meter. But no, it got taken out.

Odette Mucha:
Thanks. Another question that came in is, can an energy assessment be conducted in lieu of an audit?

Gerry Moore:
I don't know. I'm not sure what an assessment is. If you follow what's in that regulation as far as the requirements, I say, we try to build it flexibly. But if you follow those points that are mentioned, it should work. And it depends if it's residential. I think that may be doable in a residential sense. But you still have to follow what's in that reg. And we can talk.

Odette Mucha:
Great. Those are all the questions that I've gotten so far. So, why don't we wrap it up. I wanted to thank all of our wonderful presenters. You've done a great job. And I'll echo what Gerry was just mentioning to our audience. If you have any feedback on the types of information or questions that you have about the EECLP that you'd like us to cover in the rest of the webinar series, feel free to get in touch with me. And you can reach me at the following email: se, which stands for Stakeholder Engagement, se@ee.doe.gov. And we're really open to hearing your feedback and answering any additional questions.

Pat Keegan:
Can I add one thing, Odette?

Odette Mucha:
Absolutely.

Pat Keegan:
We're also still kind of scouting around for some good speakers from co-ops, especially for webinars 4, 5, and 6. So please be sure to let Odette know if you have somebody that really would like to get their story out on this, and there's lots of good possibilities out there. I know Odette's looking for that, and I'm trying to help her with it.

Odette Mucha:
And the presentation from today will be sent out in an email sometime tomorrow. So you'll be able to reference this recording in the future.

Next slide:
So just to highlight the remainder of the webinar series. Like I mentioned, next week and the following week, we'll be talking about residential energy efficiency programs, and then coming up in January, we'll be talking about on- bill financing and the solar program overview. So you'll have to register for each one individually, but we hope you do. And we look forward to having a continued conversation. And again, the email address is se@ee.doe.gov. So please send us your questions. And we'll talk to you soon. Thanks again, everyone. Bye-bye.