PA Report or Proposal - National Grid · program and implementation staff. The draft final report was delivered to National Grid on March 23, 2012. Final comments on the draft report

National Grid Upstate New York EnergyWise Program Process Evaluation (Final)

October 9, 2012

ii

National Grid – Upstate New York EnergyWise Program Process Evaluation (Final). October 9, 2012

National Grid Upstate New York EnergyWise Program Process Evaluation (Final)

October 9, 2012

Copyright © 2012 Tetra Tech, Inc. All Rights Reserved.

Prepared for: National Grid

Tetra Tech 6410 Enterprise Lane, Suite 300 | Madison, WI 53719 Tel 608.316.3700 | Fax 608.661.5181 www.tetratech.com

iii


TABLE OF CONTENTS

TABLE OF CONTENTS ........................................................................................................ iii

APPENDICES ........................................................................................................................ v

TABLE OF TABLES ............................................................................................................... v

1. Executive Summary .................................................................................................... 1-1

1.1 Program and Evaluation Overview............................................................ 1-1

1.1.1 Original Program Design .......................................................................... 1-1

1.1.2 Program Design Changes ........................................................................ 1-2

1.1.3 Evaluation Methodology ........................................................................... 1-2

1.2 Summary of Key Findings ......................................................................... 1-4

1.2.1 Areas that are Working Well ..................................................................... 1-4

1.3 Conclusion and Recommendations .......................................................... 1-7

2. Introduction ................................................................................................................. 2-1

2.1 Program Background ................................................................................ 2-1

2.1.1 Original Program Design .......................................................................... 2-1

2.1.2 Program Design Changes ........................................................................ 2-2

2.1.3 Program Staffing and Operations ............................................................. 2-3

2.1.4 Program Objectives and Achievements To-date ....................................... 2-3

2.1.5 Program Logic Model................................................................................ 2-4

2.2 Evaluation Methodology ........................................................................... 2-7

2.2.1 Key Researchable Issues ......................................................................... 2-7

2.2.2 Data Collection ......................................................................................... 2-8

2.3 Report Organization ............................................................................... 2-10

3. Detailed Process Evaluation Findings ...................................................................... 3-1

3.1 Summary of Key Findings ......................................................................... 3-1

3.1.1 Program Successes ................................................................................. 3-1

iv


3.1.2 Challenges to Program Success ............................................................... 3-2

3.1.3 Program Recommendations ..................................................................... 3-4

3.2 Program Administration and Processes .................................................... 3-6

3.2.1 Program Team Structure and Collaboration .............................................. 3-6

3.2.2 Program Tracking and Information............................................................ 3-6

3.2.3 Quality Control .......................................................................................... 3-7

3.3 Customer Awareness and Marketing ........................................................ 3-7

3.3.1 Program Awareness ................................................................................. 3-7

3.3.2 Program Understanding ............................................................................ 3-9

3.4 Ease of Participation ............................................................................... 3-10

3.4.1 Barriers to Participation .......................................................................... 3-11

3.5 Decision-making Processes.................................................................... 3-13

3.5.1 Equipment Installation ............................................................................ 3-13

3.5.2 Program Participation ............................................................................. 3-14

3.6 Program Satisfaction .............................................................................. 3-15

3.6.1 Overall Satisfaction ................................................................................. 3-15

3.6.2 Satisfaction with Specific Program Aspects ............................................ 3-15

3.6.3 Suggested Program Changes ................................................................. 3-17

3.7 Customer Characteristics ....................................................................... 3-18

4. Recommendations ...................................................................................................... 4-1

v


APPENDICES

Appendix A: Program/Implementation Staff Interview Guide ............................................... A-1

Appendix B: Building Owner Guide ..................................................................................... B-1

Appendix C: Tenant Survey Instrument .............................................................................. C-1

Appendix D: Tenant Survey Response Rate ...................................................................... D-1

Appendix E: Completion Certificate ..................................................................................... E-1

TABLE OF TABLES

Table 2-1. EnergyWise Program Rebates ............................................................................ 2-2

Table 2-2. EnergyWise Program Progress ........................................................................... 2-4

Table 2-3. Key Researchable Issues ................................................................................... 2-8

Table 3-1. Tenant’s Reasons for Participation .................................................................... 3-11

Table 3-2. Benefits of Participation .................................................................................... 3-11

Table 3-3. Installation Persistence ..................................................................................... 3-13

Table 3-3. Tenant Satisfaction with Program Aspects ........................................................ 3-16

Table 3-4. Tenant Characteristics ...................................................................................... 3-18

1-1


1. EXECUTIVE SUMMARY

This report presents the findings and recommendations resulting from the 2011 process evaluation of National Grid’s EnergyWise Program in upstate New York, and includes feedback from participants and program activity between June 2010 and June 2011.This report is one of a series of energy efficiency portfolio standard (EEPS) process evaluation reports of National Grid’s energy efficiency programs in New York. Preliminary results, from the initial in-depth interviews with program and implementation staff (including trade allies), were provided to National Grid in memo format in April 2011.1 The memo also included preliminary recommendations based on the early evaluation activities.

1.1 PROGRAM AND EVALUATION OVERVIEW

The EnergyWise program was initially approved in July 2009, with the implementation plan resubmitted with new fixtures approved in April 2010. Final approval was received in May 2010 and the program officially launched in June 2010.

1.1.1 Original Program Design

The EnergyWise program in upstate New York targets individually metered electric and/or gas multifamily buildings with five to 50 dwelling units. Eligible customers include National Grid multifamily customers of record that are building owners and property managers or individual dwelling customers on residential rates. Single family home owners, residing in dwellings of one to four units, are not eligible. In the original program design, multifamily buildings that are master metered for both heating and water heating were not eligible for the program.

The parties involved in the design and delivery of the EnergyWise program include the program manager at National Grid, managers at RISE Engineering (RISE) and ICF International (ICF), auditors at both RISE and ICF, and equipment installers at RISE or who are managed by RISE. The program is being promoted and delivered by implementation contractors, RISE Engineering and ICF. Promotion also occurs through National Grid’s website and other marketing activities. RISE works with architects, builders, engineers, remodeling contractors, technicians, retailers, equipment suppliers, and related channel intermediaries.

Originally the program was designed to provide participants with a free energy use assessment and low-cost measures at no direct cost. These low-cost efficiency measures include CFLs, low flow showerheads, aerators, domestic hot water pipe wrap, and domestic hot water tank wrap. In addition, the program was to provide rebates of 25 percent for the installation of a number of electric and gas building envelope measures such as insulation, air sealing, attic ventilation, ductwork and air infiltration testing. Lighting fixtures in common areas were offered to building owners for $20 per fixture. Lastly, the program offered $300 toward the cost of each new early replacement refrigerator and up to $75 for a normal replacement refrigerator.

1 A preliminary results memo was delivered to National Grid in April 2011 based on feedback from program and implementation staff. The draft final report was delivered to National Grid on March 23, 2012. Final comments on the draft report were received by the evaluation team in May 2012 from National Grid and August 2012 from DPS.

1-2


1.1.2 Program Design Changes

Since the program launch in June 2010, there have been a number of adjustments made to program design. While these adjustments were not included in the evaluation because they had not yet been made, these do warrant mention.

Initially, only multifamily buildings that are individually metered for electric and/or gas service were eligible for the program. While National Grid was progressing well against their electric savings targets, they found that it would be nearly impossible to meet the original gas savings goals given the stock of individually metered gas units in Upstate New York. National Grid petitioned the PSC regarding the original forecasted goals and the likely potential of meeting those goals. In June 2011, the PSC approved the eligibility of master metered gas buildings for the program.

Once the program was launched and the energy assessments were proceeding, National Grid determined that the insulation and air sealing efforts were generally not cost effective and decided to focus more attention on the direct install measures (CFLs, showerheads, aerators, pipe wrap, and tank wrap). It has also been the case that virtually none of the refrigerators examined during the energy assessments were found to meet the guidelines for the refrigerator replacement rebate.

In researching what other measures could be offered to replace the measures that were discontinued, National Grid found verbiage in the filing that allowed for measure additions if a similar program exists that is also rebating that measure. National Grid found another program rebating programmable thermostats which were cost effective, as well as boiler reset controls. Both of these measures were approved for the program in July 2011.

As a result of the preliminary findings from the staff interviews and drafting of the logic model conducted as part of this process evaluation, the program staff have implemented quality assurance and quality control (QA/QC) inspections that were originally intended as part of the program but had not occurred early on. ICF is in charge of targeting ten percent of the participating properties to review what has been installed and to conduct a short survey with property managers. Program implementation staff are now trying to determine the most appropriate way to track the QA/QC information using the InDemand system.

The most recently added benefit to building owners and managers from the program is the availability of EnergyWise Choice logo window decals that property managers can use for advertising participation in the program and upgrades to individual units.

1.1.3 Evaluation Methodology

Process evaluation activities for the EnergyWise Program consisted of qualitative interviews with program and implementation staff, qualitative interviews with participating and dropout building owners and managers, and structured quantitative phone surveys with tenants. The process evaluation covered audit and equipment installations through the program between June 2010 and June 2011.

1-3


a. Program and Implementation Staff Interviews

The first evaluation activity for the EnergyWise program consisted of in-depth interviews with program and implementation staff involved with the design and/or delivery of National Grid’s EnergyWise Program in upstate New York State.

Tetra Tech conducted one interview with the upstate EnergyWise program manager and six interviews with implementation staff. National Grid provided a list of implementation staff with key responsibilities with the program in a variety of different roles. The six implementation staff interviewed were from RISE Engineering and ICF and included managers as well as auditors.

Interviews covered a variety of topics depending on each interviewee’s role within the program. Topics covered included2:

Roles and responsibilities

Program design and marketing

Program operations

Internal and external communication

Customer and trade ally outreach and interaction

Quality assurance

Program database and tracking

Barriers to program uptake and meeting goals and objectives

Satisfaction with program

Areas of interest to investigate during subsequent data collection.

A copy of the in-depth interview guide can be found in Appendix A of this report.

b. Participating and Dropout Building Owner/Manager Interviews

Overall, 89 unique records at the local level and 25 unique records at the decision-maker level were provided by National Grid for the purpose of interviewing participating and dropout building owners and managers. We completed a total of 18 interviews, 14 with property managers at the local level, and four interviews with corporate-level decision-makers. Interviews were scheduled with six more local property managers, but they missed their appointments. Rescheduling efforts were unsuccessful and were discontinued to focus on reaching more corporate decision-makers.

Initially, we attempted to reach the contact listed in the sample file at the local level, which turned out in most cases to be the local property manager for the property. Since the majority of

2 Tetra Tech conducted interviews between March 8, 2011 and March 25, 2011. Interviews ranged in duration from thirty to sixty minutes, depending on the interviewee’s role. We do not identify interviewees by name, nor do we identify them by role when discussing specific findings as a means of protecting interviewees’ confidentiality.

1-4


local property managers were not involved in the decision to participate in the program, we attempted to obtain information about the decision-making process by identifying and following up with the corporate-level contact for each property. After attempting to reach the corporate-level contacts by phone and having little success (only two completed phone interviews), we tried to obtain email addresses in order to send out a shortened questionnaire that contained key decision-making questions. This effort resulted in two additional completes.

Calls to building owners and managers were attempted from late July to the end of August. At least two attempts were made to determine working numbers, leave voicemail messages, and identify the best way to reach the most knowledgeable contact. However, many cases were attempted four or five times in order to find a convenient time to schedule the interview. The interview guide can be found in Appendix B.

c. Participating tenant phone surveys

The last phase of evaluation activities consisted of CATI (computer assisted telephone interview) surveys with tenants served by the program. Files containing participating building owners and their tenants were provided by National Grid. Due to staffing changes at National Grid, the tenant survey was approved in November 2011 and calling occurred in December.

A goal of 70 completes was met for the survey, which explored tenant characteristics, program awareness and satisfaction, measure persistence, and benefits of participation. The tenant phone survey can be found in Appendix C, and the detailed response rate can be found in Appendix D.

1.2 SUMMARY OF KEY FINDINGS

The following summarizes what we have identified as the most important and recurring themes based on all of the evaluation activities for the EnergyWise program.

1.2.1 Areas that are Working Well

There is a good team working relationship. The staff report that the working relationship

among National Grid, RISE engineering, and ICF International is very supportive and collegial. Meetings are held regularly along with regular reporting of current progress and barriers. This leads to quick resolution of concerns and questions fielded by the staff in a timely manner. Initial indications are that the program team is united in addressing the challenge of identifying and engaging eligible gas customers to achieve the targeted gas savings for the program.

Customer satisfaction is high. Several staff mentioned that program participants are satisfied with the program process. Feedback they have received from the completion certificates indicates that participants find the audit process easy and 90 to 95 percent are estimated to be satisfied with the program overall. This perception of high satisfaction is supported through interviews with property owners and managers, as well as with tenants receiving the measures.

The program is meeting goals for electricity savings. The interviewees reported that the electricity savings side of the EnergyWise program is performing well, and has actually exceeded the current electricity savings goal. Staff also indicated that CFL replacement made up the bulk of electricity measures installed, and that the rebate levels for these measures are competitive in the current service area.

1-5


Competition from other programs for customers has been minor to date. Whereas other commercial program customers may be eligible for incentives from multiple sources, there is a mandated split in the multifamily market by size. National Grid can approach customers with five to 50 units per building while NYSERDA can approach those with more than 50 units. In addition, National Grid has been working collaboratively with all of the administrators delivering EEPS programs. The only concern will be possible confusion when the National Grid team approaches management firms that manage buildings in both categories, but that was not an issue with the management we spoke with. In fact, one large building manager feels the utilities are much better at serving customers with energy efficiency programs than NYSERDA and that there is little action from NYSERDA.

The Program Manager is receiving necessary data. The team meets weekly by conference call and the implementers provide frequent reports to the program manager. The program database kept by the implementer is separate from the InDemand tracking and is more current.

The program is appropriately staffed. The implementer closely monitors the level of activity and has adequate staff available for audits, installation, and data entry. RISE has also added a marketing person to support the marketing effort for EnergyWise, as it has proven to be more work than originally anticipated.

Original issues with QA/QC checks have been resolved quickly. QA/QC was not occurring

as intended when the program was initially rolled out. At first, RISE was checking a portion of their own installations, but ICF was responsible for QA/QC as a second set of eyes. Program staff reported there is no QA/QC requirement in the implementation plan and no associated budget for it. Interviewees also report that the focus on meeting the gas savings goal was distracting from the QA/QC part of the program process. National Grid has already clarified the QA/QC process for projects since the delivery of the preliminary staff interview results by confirming ICF’s role as the QA/QC lead for the program and ICF has begun the process of selecting and verifying installations.

Dropouts are not has high as expected. Dropouts are defined as those properties that had an

energy assessment but had not yet installed any energy efficient equipment. Most projects listed as dropouts are not actually dropping out of the program, but are rather still in the process of determining what they will install at locations not already served by the program. Therefore, these cases would be more accurately categorized as “in-progress” rather than dropouts.

Program satisfaction is high. Participants report a high level of satisfaction with the program

experience and National Grid as a result of participation. Over half of the tenants (54 percent) reported that their experience with the EnergyWise Program increased their satisfaction with National Grid as their energy provider.

The program is promoting energy efficiency upgrades. The group of building owners and managers we spoke with typically only implement upgrades when equipment fails or if there is turnover in a unit. The program is motivating them to replace equipment earlier than planned.

a. Challenges to Program Success

Because of the newness of this program, there were several challenges identified through the evaluation activities for the first year of the program. The challenges mentioned in the interviews are highlighted below, with suggestions for addressing them in the Recommendation Section.

1-6


The gas savings side of the Energy Wise program struggled early on. Staff indicated that the initial individual metering requirements drastically limited the pool of eligible gas customers. Several staff also indicated that the current gas incentive levels do not seem to be high enough to influence participation of the property owners, which has compounded the problem of the small eligible customer pool. Some staff have heard property owners requiring at least 50 percent of the project cost covered before they can justify the expenditure.

Refrigerator cost-effectiveness requirements are unrealistic for the current service area. Staff report customers’ interest in the refrigerator rebate has driven overall interest in the program. However, several staff stated that it is nearly impossible for customers to qualify for a refrigerator rebate if the unit is less than 17 years old (early replacement). Implementation staff reported that almost none of the tested refrigerators ended up cost-effective based on the current calculation. Property managers confirm that this has led to frustration and confusion on their part as many were originally interested in the program for the refrigerator rebates.

There has been difficulty in selling the program to property owners. The interviewees report that the current economic situation is causing property owners to overlook measures that benefit tenants, but do not directly benefit themselves. In addition, insulation and weatherization improvements are high-cost measures that require advance budgeting. Interviewees have found that the 25 percent program contribution is not enough to motivate customers to undertake this advance budgeting. They also indicate that gaining contact information and access to out-of-state property owners has been a challenge, which evaluators confirmed to be the case when they attempted to contact these owners to complete an interview.

Current marketing efforts have not been sufficient. Numerous members of the staff mentioned that more marketing materials are needed. They also indicate that the lack of marketing has been a factor in the current slow generation of energy audit leads. Mass marketing efforts have not reached the targeted customers and have created little activity on the program website.

Cost-effectiveness screening at the measure level has impacted customer participation.

Several staff mentioned that it is not uncommon for projects that achieve an overall positive cost ratio on the project level to be passed over due to the requirement that each measure be cost-effective. They have also reported that measure level screening is causing confusion with customers, as well as burdening property owners with tracking measures installed on a building-by-building basis. Staff feels that this leads to only a fraction of the measures a facility could receive being installed. Some property owners and managers confirmed that they did not understand why they were not eligible for the refrigerator or insulation rebates and why not all units are treated the same.

The InDemand database has little support for report generation. Staff mentioned that the lack of a reporting feature in the InDemand database does not allow for an up-to-date picture of the current progress of the program. This has contributed to redundancy as other databases with reporting features are being used to track the same information.

Program tracking data from InDemand is not 100 percent accurate. During the evaluation,

we found that there were inconsistencies concerning how measures were recorded or pulled for the participating tenant interviews. For example, interviews with property managers revealed that they often had water saving measures installed while the database records received showed only lighting measures. Review of some of the applications from RISE revealed that the

1-7


full range of measures installed did not match what we received. It is not clear if the issues arise from data entry errors or how the queries pull data from the InDemand system.

There is a lack of understanding regarding who offers services and eligibility criteria.

Building owners and managers find it very difficult to determine who provides programs to assist the different types of multifamily properties with energy efficiency upgrades. There are varying eligibility criteria depending on the location of the property, whether buildings are individually or master metered, the type of equipment they are interested in replacing, and the number of units per building. Currently, many of the building owners and managers said that they are not completing energy efficiency projects because they are not able to identify utility or other programs to assist them.

Multiple visits to properties lead to building owner and manager frustration. Some building owners and managers have gone through multiple visits from the program to serve their properties for electric and then gas measures. Others received one visit for tenant units, then another for common areas. These respondents did not understand why the efforts could not be coordinated and that it was a waste of time to go through the properties twice.

The inability to serve the entire property removes some of the benefits to building owners and managers. In particular, the restriction that National Grid can serve the five to 50 unit properties diminishes the ability of building owners and managers with multi-unit properties to use the program and energy efficient equipment for marketing the units, especially if buildings in the complex do not fall within the five to 50 range.

1.3 CONCLUSION AND RECOMMENDATIONS

Overall, feedback from program staff, implementers, building owners and managers, and tenants involved with the EnergyWise Program indicate that after a late launch and some mid-stream program adjustments, the program is on the right track and achieving estimated savings for gas. Participants are highly satisfied with their program experience and participation has improved their perception of National Grid.

At the same time, the program faces several internal challenges, most specifically those associated with effective outreach and cost effectiveness testing. Current marketing efforts are not reaching multifamily customers efficiently and multifamily customers are uncertain of where to turn for energy efficiency upgrades. In addition, the inability for any refrigerators to pass the cost effectiveness testing, the requirement for individual measure cost effectiveness testing, and multiple property visits are testing the patience of some building owners and managers.

Below are recommendations to consider that address some of the opportunities identified.

Pursue the opportunity with regulators to expand to 5-100 units and continue to target mid-sized property management firms that handle multiple properties. The implementation team feels they have been more successful in targeting some of the mid-sized property management firms and working with them to serve multiple facilities than targeting individual properties. The program should continue to make an effort to identify and target those firms that handle multiple properties. Another approach to consider is utilizing regional property manager or landlord associations.

Increase the coordination between National Grid marketing and the program manager and implementers. Program implementers have been cold-calling multifamily customers to

1-8


create awareness and interest in the EnergyWise Program. This practice is very time intensive. Consider selecting customer records based on metering so the program can target appropriate customers with detailed marketing information.

Clearly define program benefits for customers. In order to address both economic barriers and the potential competition from other programs, National Grid will need to fully understand what other programs are offered to customers within their territory and position National Grid programs to offer the best product to customers. Benefits may be a simple application process, payment for their portion of the project costs only (eliminating the wait for a rebate check), knowledgeable contractors offering technical assistance, and more responsive program staff. As suggested by the new RISE marketing staff, a Frequently Asked Question (FAQ) document for customers would also be useful.

Review program incentive levels for gas and weatherization measures. All program staff and implementation interviewees pointed to difficulties gaining interest from building owners and managers given what they feel are low incentive levels. The EnergyWise program has had no trouble meeting electric savings goals. That portion of the program is designed as a direct install service. However, the program pays 25 percent of gas and weatherization measure costs, which the program team feels is too low to overcome the long lead time for decision making and promote interest from a group historically unwilling to spend money given split incentives with tenants. Implementer experience with other multifamily programs, and comments implementers have received from customers, suggest that offering 40 to 50 percent would be much more successful in encouraging customer participation in the program. These measures may now be more attractive with the eligibility of master metered gas customers. There is interest from them in decreasing their gas bills, and the weatherization measures could be marketed as a way to achieve savings.

Explore the possibility of offering bill payment or loan options for major measures similar to other programs. Program staff mentioned that other programs for commercial customers often provide the option of paying off the amount on their monthly bills. Some customers have told staff the reason they are not participating has to do with the lower proportion paid by the utility for the projects. Including a financing option would provide implementation staff with another sales tool to help overcome the initial cost barrier mentioned by some customers for major measures such as insulation that are not fully covered by the program. One building manager suggested that option during our interviews and at least one-quarter of upstate Commercial and Industrial customers (participating and nonparticipating) interviewed through the upstate commercial and industrial process evaluation confirmed interest in financing offered by National Grid for energy efficiency upgrades. Another 25 percent were open to the option depending on factors such as management decisions, the size of the project, and the interest rate.

Investigate how to better serve an entire property in a more comprehensive manner. In conjunction with more targeted marketing, program staff may want to consider incentivizing RISE to work more closely with multifamily customers in order to get them directed to the appropriate programs. Awareness of other programs and energy efficiency offerings appears to be a major gap for many of the multifamily building owners and managers. RISE is in the position to assist multifamily customers with a variety of energy efficiency needs by routing them to Residential High–Efficiency Heating and Water Heating and Controls Program for heating and water heating measures, Enhanced Home Sealing for two to four unit properties, and even NYSERDA if needed for the more extensive renovations and retrofits that National Grid cannot cover. RISE and ICF should have very clear and distributable information on what other

1-9


programs are offered that multifamily customers can utilize. The ability to offer a more cross cutting service to multifamily managers would help alleviate some of the frustration and maximize savings potential.

Review the tracking database inputs for accuracy and completeness. There were mislabeled measures in the program database download we received that were identified as we conducted interviews with building owners and managers. In addition, only one-third of the records had a correct phone number attached to the record. It would be worthwhile for tracking purposes and evaluation purposes to check a percentage of the database records against paper documents to ensure program information is accurate and recorded as expected by all parties using the data. For multifamily programs it is also important to capture the differences between local property managers and corporate decision makers to facilitate future communication and evaluation.

Consider presenting a case to the PSC regarding the requirement that each measure pass the cost effectiveness test. Interviewees have found that it is very confusing to customers to go through the process of an audit, only to find that several of the recommended measures are not eligible for program incentives because they are not individually cost effective. This becomes even more of an issue when program staff deal with a property owner who has multiple buildings and a measure may be cost-effective at one building but not at others. Interviewed parties would like to see more of a whole-building or whole-project approach if the overall project level cost ratio is positive to increase participation and building-wide efficiency.

2-1


2. INTRODUCTION

This report presents the results of the 2011 process evaluation of National Grid’s EnergyWise Program in upstate New York, and includes feedback from participants and program activity between June 2010 and June 2011. This report is one of a series of EEPS process evaluation reports of National Grid’s energy efficiency programs in New York. Preliminary results, from the initial in-depth interviews with program and implementation staff (including trade allies), were provided to National Grid in memo format in April 2011.3 The memo also included preliminary recommendations based on the early evaluation activities.

2.1 PROGRAM BACKGROUND

The EnergyWise Program was initially approved in July 2009, with the implementation plan resubmitted with new fixtures approved in April 2010. Final approval was received in May 2010 and the program officially launched in June 2010. Original Program Design

The EnergyWise program in upstate New York targets individually metered electric and/or gas multifamily buildings with five to 50 dwelling units. Eligible customers include National Grid multifamily customers of record that are building owners and property managers or individual dwelling customers on residential rates. Single family home owners, residing in dwellings of one to four units, are not eligible. In the original program design, multifamily buildings that are master metered for both heating and water heating were not eligible for the program.

The parties involved in the design and delivery of the EnergyWise Program include the program manager at National Grid, managers at RISE Engineering (RISE) and ICF International (ICF), auditors at both RISE and ICF, and equipment installers from RISE or managed by RISE. The program is being promoted and delivered by implementation contractors, RISE Engineering, and ICF. Promotion also occurs through National Grid’s website and other marketing activities. RISE works with architects, builders, engineers, remodeling contractors, technicians, retailers, equipment suppliers, and related channel intermediaries.

Originally the program was designed to provide participants with a free energy use assessment and low-cost measures at no direct cost. These low-cost efficiency measures include CFLs, low flow showerheads, aerators, domestic hot water pipe wrap, and domestic hot water tank wrap. In addition, the program was to provide rebates of 25 percent for the installation of a number of electric and gas building envelope measures such as insulation, air sealing, attic ventilation, ductwork, and air infiltration testing (Table 2-1. Original EnergyWise Program RebatesTable 2-1). Lighting fixtures in common areas were offered to building owners for $20 per fixture. Lastly, the program offered $300 toward the cost of each new early replacement refrigerator and up to $75 for a normal replacement refrigerator.

3 A preliminary results memo was delivered to National Grid in April 2011 based on feedback from program and implementation staff. The draft final report was delivered to National Grid on March 23, 2012. Final comments on the draft report were received by the evaluation team in May 2012 from National Grid and August 2012 from DPS.

2-2


Table 2-1. Original EnergyWise Program Rebates

Measure Rebate

Insulation (Attic, wall, basement/crawl space, rim joist, duct, heating system pipes)

25% of measure cost

Air infiltration sealing for gas heated property where National Grid is the gas utility

25% of measure cost

Refrigerators $300 for each new early replacement and up to $75 for a normal replacement

Lighting fixtures Participant pays $20

Compact fluorescent bulbs 100% of measure cost

Low flow showerheads, Aerators 100% of measure cost

Minimal domestic hot water pipe wrap 100% of measure cost

Domestic hot water tank wrap 100% of measure cost

2.1.1 Program Design Changes

Since the program launch in June 2010, there have been a number of adjustments made to program design. While these adjustments were not included in the evaluation because they had not yet been made, these do warrant mention.

Initially, only multifamily buildings that are individually metered for electric and/or gas service were eligible for the program. While National Grid was progressing well against their electric savings targets, they found that it would be nearly impossible to meet the original gas savings goals given the stock of individually metered gas units in Upstate New York. National Grid petitioned the PSC regarding the original forecasted goals and the likely potential of meeting these goals. In June 2011, the PSC approved the eligibility of master metered gas buildings for the program.

Once the program was launched and the energy assessments were proceeding, National Grid determined that the insulation and air sealing efforts were generally not cost effective and decided to focus attention on the direct install measures (CFLs, showerheads, aerators, pipe wrap, and tank wrap). It has also been the case that virtually none of the refrigerators examined during the energy assessments were found to meet the guidelines for the refrigerator replacement rebate.

In researching what other measures could be offered to replace the measures that were discontinued, National Grid found verbiage in the filing that allowed for measure additions if a similar program exists that is also rebating that measure. National Grid found another program rebating programmable thermostats which were cost effective, as well as boiler reset controls. Both of these measures were approved for the program in July 2011.

As a result of the preliminary findings from the staff interviews and drafting of the logic model conducted as part of this process evaluation, program staff have also implemented quality assurance and quality control (QA/QC) inspections that were originally intended as part of the program but had not occurred early on. ICF is in charge of targeting 10 percent of the

2-3


participating properties to review what has been installed and to conduct a short survey with property managers. Program implementation staff are now trying to determine the most appropriate way to track the QA/QC information using the InDemand system.

The most recently added benefit to building owners and managers from the program is the availability of EnergyWise Choice logo window decals that property managers can use for advertising participation in the program and upgrades to individual units.

2.1.2 Program Staffing and Operations

Initial marketing for the program consisted of mass mailings to customers thought to be multifamily customers. The staff we interviewed felt the mass mailings were not effective in reaching the intended targets or customers who are eligible for the program. Therefore, National Grid has supplied lists of customers with meter type to the implementation contractors who have turned to one-on-one communications where they cold-call customers and talk with them about the program.

While RISE and ICF have focused on personal communication with condominium owners, RISE also hired a marketing staff person who is responsible for targeted marketing and outreach efforts to eligible gas customers.

Once customers are aware of the program, they can contact RISE for more information. RISE sends any leads to ICF who schedules and conducts the free energy audits. ICF forwards all resulting audit information to RISE to calculate the cost-effectiveness of each measure. Based on the data, RISE drafts a contract for the customer outlining what equipment is eligible for installation through the program. ICF then presents that contract to the customer, answers their questions, and obtains the customer’s consent to continue. If the customer accepts the contract, RISE or their subcontractor installs the equipment. ICF staff will then conduct quality assurance checks on a portion of the installations.

RISE requires that a certificate of completion be returned by the customer that reports what was installed, the location of installation, five survey questions, and signatures of the customer and program representative. RISE then invoices the customer for their portion of the project cost and National Grid for the remaining portion.

2.1.3 Program Objectives and Achievements To-date

After a late launch in 2010 and some recent program adjustments, the program has gained momentum in reaching gas goals and has far exceeded its electric goals.

The electric and gas savings goals for 2009-2011 were set when the program was approved. These goals are outlined in the EnergyWise Program Revised Implementation Plan4. At the end of 2010, the program achieved 92 percent of its 2009-2010 electric goals. On the gas side, the program achieved three percent of its 2009-2010 goals.

4 Niagara Mohawk Power Corporation d/b/a National Grid, Case 08-E-1133 and Case 09-G-0363, “EnergyWise Program Revised Implementation Plan, June 18, 2010.

2-4


While the achieved gas savings were very low, it was not for lack of conducting energy use assessments, but rather due to a smaller than anticipated potential market. Four hundred thirty gas assessments were performed, with less than one-half (205) of these projects being eligible for the program. The remaining 225 projects were ineligible since they did not have either gas heating or gas domestic hot water heaters that were individually metered.

As mentioned earlier, a few post-launch adjustments to the program, particularly the ability to serve master metered gas customers, resulted in a much more successful second year.

Table 2-2. 2011 EnergyWise Program Progress5

Upstate Electric Upstate Gas

Annual participation goals for 2009-2010 (units) 3,150 2,325

Actual participation for 2009-2010 (units) 1,566 0

Annual savings goals for 2009-2010 1,620 MWh 235,940 therms

Actual savings for 2009-2010 1,490 MWh 7,078 therms

Percent of savings goal in 2009-2010 92% 3%

Annual participation goals for 2011 (units) 2,800 1,800

Actual participation for 2011 (units) 796 288

Annual savings goals for 2011 1,303 MWh 158,760 therms

Actual savings for 2011 5,916 MWh 134,946 therms

Percent of savings goal in 2011 454% 85%

2.1.4 Program Logic Model

A program logic model is a visual representation of the program’s theory that illustrates a set of interrelated program activities that combine to produce a variety of outputs that lead to key short-, mid- and long-term outcomes. A program logic model can lead to a cost-effective determination of program effectiveness.

Logic models can be linked to performance indicators in order to provide on-going feedback to program managers. The models flow top to bottom and are typically organized according to five basic categories:

Program resources: financial, staffing, and infrastructure resources that support the

activity

Program activities: overarching activities that describe what the program is doing.

Examples include marketing, rebate processing.

Outputs: metrics resulting from the activities. These tend to be measurable “bean

counting” results (e.g., provide outreach events at five community fairs)

5 Some figures were pulled from the 2010 Energy Efficiency Programs Annual Report, March 15, 2011

and others were provided by the EnergyWise program manager. The percent saved against goal for the year may be due to program adjustments that were not included in the timeframe evaluated.

2-5


Short- to medium-term outcomes: expected outcomes resulting from program activities, with goals attached to those outcomes when possible. Examples include target energy savings, recruitment into the program.

Long-term outcomes: ideal, sustainable outcomes resulting from program activities, such as “all eligible customers participate in program” and “increase customer awareness of program offerings.”

Stepping across the activities enumerated in the logic model indicates an approximate “flow” in the sequence of activities. For example, the logic model begins with the program infrastructure and ends with the activity which results in direct install and/or rebated measures. In each column, the resources needed are specified above each activity, and then the direct outputs of the activity appear. The outcomes are causally linked to the various outputs in each column of the logic model. In other words, it is expected that the specified output (e.g., audit recommendations) will result in the specified outcome (e.g., equipment installation).

The logic model in Figure 2-1 was developed from program documentation and interviews with program and implementation staff, and has been reviewed by the program manager.

2-6


Figure 2-1. National Grid Upstate New York EnergyWise Original Program Logic Model

Inputs/

ResourcesSufficient budget is allocated Marketing materials and customer

lists

National Grid program team

(internal staff, implementers, etc)

National Grid program team

(internal staff, implementers, etc)

Customer Agreement

National Grid StaffNational Grid call center and web site

(powerofaction.com), RISE, ICFAudit forms Signed customer agreement

Completion Certificate

ActivitiesDevelop Program Infrastructure Outreach to Customers Perform Audits Install Measures Invoice Projects

Outputs

Energy Wise program implementation

plan is developed and approved

Target direct communications and

other outreach such as newsletters,

website to multifamily customers

ICF performs audits for all

interested customers

Equipment is installed by either

RISE or electrical contractor for

common area lighting

RISE validates customer

applications and invoices

customer for their portion of

project cost

Implementer identified and program is

launched to National Grid customers

in 2010

In-person communication with

targeted customers by ICF and RISE

RISE uses audit data to calculate

cost effectiveness of measures

Completion certificates are sent to

RISE

RISE uploads project data into

InDemand and invoices National

Grid monthly

Program measures, rebates,

marketing strategy and technical

assumptions developed, refined and

documented

ICF presents customers with

customized agreement as result of

audit and cost effectiveness testing

Post inspections are completed by

ICF on 10% of projects (QA/QC)

Tracking system developed and

appropriate information is requested,

captured and entered

Short to

medium term

outcomes

Implementation team is ready to

assist customers through the

program process

Customers' awareness of and

participation in the program increases

Auditors identify areas where

customers can save money by

making energy-saving

improvements

Customers gain savings benefits

and property enhancement from

installing high-efficiency equipment

Customer copay is handled

accurately and efficiently.

Program progress is tracked and

reviewed for accuracy

Customers go to Power of Action

website for energy efficiency

information

Customers sign agreement to

proceed with equipment installation

Customers experience an efficient

and educational process and are

encouraged to participate in other

National Grid programs

The program team is adjusting

periodically to meet program needs

Long term

outcomes Program passes total resource cost

test

The demand for energy-efficient

equipment increases

Customers understand areas of

inefficiency and look for other

opportunities to make

improvements

Ensure that incentivized equipment

meets program requirements

Energy saving goals of the

program are achieved within

budgetary constraints

Budget is secured

Increased penetration of energy

efficient equipment among National

Grid multifamily customers

Achieve

2,923 MWh

394,700 Therms

savings from 2009-2011

Comment [PAH1]: Pagination at the bottom of this page needs adjusting.

2-7


The first activity in the above program logic model is developing a program infrastructure. National Grid selected two vendors, RISE and ICF, to assist with the implementation of the program.

Once the infrastructure was in place and the budget was secured, outreach to customers occurred. While National Grid provided widespread communication for building owners and managers in the form of newsletters and website information, RISE and ICF visited customers in person to generate project leads.

The third program activity, as shown in the logic model, is performing the audits or energy assessments for the properties. During this phase, ICF conducts energy assessments and RISE calculates the cost effectiveness of measures. ICF can then present the customers with their customized agreements.

After the building owner or manager accepts the agreement, RISE visits the property to install the equipment and deliver the completion certificates to tenants. ICF then targets approximately ten percent for post-installation inspections or QA/QC.

The last phase of the program is for RISE to validate the customer application and invoice the customer for their portion of the project cost if applicable. RISE also enters the project information into InDemand and invoices National Grid monthly.

2.2 EVALUATION METHODOLOGY

Several data collection activities occurred in order to address the key researchable issues identified in the evaluation planning stage. We talked with program staff and implementers, building owners and managers, and tenants who received energy efficient equipment.

First we outline the key researchable issues and then describe the data collection activities employed to gather data in attempt to address the key researchable issues.

2.2.1 Key Researchable Issues

The researchable issues identified for the process evaluation are in Table 2-3 below, and are organized around the main research topics.

2-8


Table 2-3. Key Researchable Issues

Research Topic Researchable Question

Customer Awareness and Marketing

How effective is the program marketing? What activities are most effective in providing program information? How does the EnergyWise program distinguish itself from other multifamily programs?

How do participants most commonly hear about, and become involved in, the program? What marketing and outreach efforts are most successful in generating customer leads? Does the overlap with other internal and external programs help or hinder the success of these programs?

Program Administration, Processes and Resources

How effective is the collaboration between National Grid and RISE?

Do program manager and RISE feel they have sufficient staffing resources to deliver the program? What additional information or resources are needed?

Ease of Customer Participation

What are the characteristics of the participating landlord population and how does that compare to the eligible population? Are there any groups not reached by the program that also have financial and efficiency needs?

What barriers exist for landlord’s participation in the program? Is competition from other multifamily programs inhibiting participation in the National Grid program?

Program Satisfaction

How is the program working? How could it be improved? What enhancements are needed in the design and delivery of the program?

Are customers satisfied with the program and the recommended or installed measures? What do they believe could be offered to improve program services?

Customer Characteristics and Decision Making Processes

Which measures have been installed and what type of equipment did it replace? How are they accepted and valued by the customer? What additional measures would they be interested in?

Do measures remain installed and, if not, why not?

Did the technical assessment or free low-cost measures provide information which prompted important energy savings projects? How important was the technical assistance in their decision to participate? The program incentive?

Does participation affect participants’ perception of the utility and, if so, how?

Why do customers decide not to install measures after receiving technical assistance?

Program Performance Indicators

Is the program delivering the intended benefits to participants and are they achieving planned energy impacts? Is the referrals process working effectively in identifying appropriate customers for other programs?

Are all applicable/cost-effective electric and gas measures being offered to participants? Why aren’t some being taken?

Is the appropriate information being collected to support future evaluation activities (i.e., impact evaluation)?

Are program goals set appropriately?

Will the program be on target to reach its savings and spending goals? Why or why not?

2.2.2 Data Collection

a. Program and implementation staff interviews

The first evaluation activity for the EnergyWise program consisted of in-depth interviews in March 2011 with program and implementation staff involved with the design and/or delivery of National Grid’s EnergyWise Program in upstate New York State.

2-9


Tetra Tech conducted one interview with the upstate EnergyWise program manager and six interviews with implementation staff. National Grid provided a list of implementation staff with key responsibilities with the program in a variety of different roles. The six implementation staff interviewed were from RISE Engineering and ICF and included managers as well as auditors.

Interviews covered a variety of topics depending on each interviewee’s role within the program. Topics covered included6:

Roles and responsibilities

Program design and marketing

Program operations

Internal and external communication

Customer and trade ally outreach and interaction

Quality assurance

Program database and tracking

Barriers to program uptake and meeting goals and objectives

Satisfaction with program

Areas of interest to investigate during subsequent data collection.

b. Participating Building Owner Interviews

Overall, 89 unique records at the local level and 25 unique records at the decision-maker level were provided by National Grid for the purpose of interviewing participating and dropout building owners. Of these, we completed 14 interviews with property managers at the local level and four interviews with corporate-level decision-makers in August 2011.

Initially, we attempted to reach the contact listed in the sample file, which turned out in most cases to be the local property manager for the property. Since the majority of local property managers were not involved in the decision to participate in the program, we attempted to obtain information about the decision-making process by identifying and following up with the corporate-level contact for the property. After attempting to reach the corporate-level contacts by phone and having little success (only two completed phone interviews), we tried to obtain email addresses in order to send out a shortened questionnaire that contained key decision-making questions. This effort resulted in two additional completes.

Throughout the calling process we experienced difficulty in determining the role the contact had in the decision-making process based on sample information. In most cases, the corporate-level contact was identified by either looking for contacts that spanned multiple properties or being re-directed by the local property manager. In addition, approximately one-third of the records had

6 Tetra Tech conducted interviews between March 8, 2011 and March 25, 2011. Interviews ranged in duration from thirty to sixty minutes, depending on the interviewee’s role. We do not identify interviewees by name, nor do we identify them by role when discussing specific findings as a means of protecting interviewees’ confidentiality.

2-10


the correct phone number attached to the record, which contributed to the difficulty in scheduling interviews.

c. Participating Tenant Phone Surveys

The last phase of evaluation activities consisted of CATI (computer assisted telephone interview) surveys in December 2011 with tenants served by the program. Files containing participating building owners and their tenants were provided by National Grid. A goal of 70 completes was met for the survey, which explored tenant characteristics, program awareness and satisfaction, measure persistence, and benefits of participation. The detailed response rate can be found in Appendix D.

While conducting in-depth interviewers with property owners and managers, it became apparent that there were inconsistencies between the tenant level data and the measures that the property managers indicated had been installed. After a review of 16 out of the 253 applications, 11 were chosen at random. Three issues were identified that had to be resolved with before proceeding with sampling. These included a mismatch between total measure cost and the measure, duplicate tenant cases, and extra cases associated with the property owner.

The first issue was resolved by compiling a list of measure pricing information and then comparing this with the measure listed for each tenant case. The second issue was corrected by removing cases with the same application and tenant information. No correction was found for the third problem. However, out of the 16 applications reviewed, this was only found in one application. In addition to these corrections, a confirmation of the measures tenants received was placed in the survey. During fielding, it was determined that the majority of the contact phone numbers were attached to cellphones. Due to the nature of cellphone sample the following actions were taken: a maximum of six attempts was set with only one contact per night allowed, no refusal conversion was attempted, and no bad number lookups were performed.

2.3 REPORT ORGANIZATION

The next section of this report presents the results of the process evaluation. Section 4 discusses key conclusions and recommendations. The technical appendices contain the evaluation data collection instruments and detailed survey response rates.

3-1


3. DETAILED PROCESS EVALUATION FINDINGS

This section presents the findings of the process evaluation of National Grid’s EnergyWise Program in upstate New York. First, we highlight key findings from the process evaluation. These are followed by detailed findings in the following categories:

Program administration and processes

Customer awareness and marketing

Ease of participation

Customer characteristics and decision-making processes

Program satisfaction.

3.1 SUMMARY OF KEY FINDINGS

The following summarizes what we have identified as the most important and recurring themes based on all of the evaluation activities for the EnergyWise program.

3.1.1 Program Successes

There is a good team working relationship. The staff report that the working relationship among National Grid, RISE Engineering, and ICF International is very supportive and collegial. Meetings are held regularly along with reporting of current progress and barriers. This leads to quick resolution of concerns and questions fielded by the staff. Initial indications are that the program team is united in addressing the challenge of identifying and engaging eligible gas customers to achieve the targeted gas savings for the program.

Customer satisfaction is high. Several staff mentioned that program participants are

satisfied with the program process. Feedback they have received from the completion certificates indicates that participants find the audit process easy and 90 to 95 percent are estimated to be satisfied with the program overall. This is supported through interviews with property owners and managers, as well as with tenants receiving the measures.

The program is meeting goals for electricity savings. The interviewees reported that the

electricity savings side of the EnergyWise program is performing well and has exceeded the current electricity savings goal. Staff also indicated that CFL replacement made up the bulk of electricity measures installed and that the rebate levels for these measures are competitive in the current service area.

Competition from other programs for customers has been minor to date. Whereas other

commercial program customers may be eligible for incentives from multiple sources, there is a mandated split in the multifamily market by size. National Grid can approach customers with five to 50 units per building while NYSERDA can approach those with more than 50 units. In addition, National Grid has been working collaboratively with all of the administrators delivering EEPS programs. The only concern will be possible confusion when the National Grid team approaches management firms that manage buildings in both categories, but that was not an issue with the management we spoke with. In fact, one large building manager feels the utilities are much better at serving customers with energy efficiency programs than NYSERDA and that there is little action from NYSERDA.

3. Detailed Process Evaluation Findings

3-2


The Program Manager is receiving necessary data. The team meets weekly by conference call and the implementers provide frequent reports to the program manager. The program database kept by the implementer is separate from the InDemand tracking and more current.

The program is appropriately staffed. The implementer closely monitors the level of activity and has adequate staff available for audits, installation, and data entry. RISE has also added a marketing person to support the marketing effort for EnergyWise, as it has proven to be more work than originally anticipated.

Original issues with QA/QC checks have been resolved quickly. QA/QC was not

occurring as intended when the program was initially rolled out. At first, RISE was checking a portion of their own installations, but ICF was responsible for QA/QC as a second set of eyes. Program staff reported there is no QA/QC requirement in the implementation plan and no associated budget for it. Interviewees also report that the focus on meeting the gas savings goal was distracting from the QA/QC part of the program process. National Grid has already clarified the QA/QC process for projects since the delivery of the preliminary staff interview results by confirming ICF’s role as the QA/QC lead for the program and ICF has begun the process of selecting and verifying installations.

Dropouts are not has high as expected. Dropouts are defined as those properties that had an energy assessment but had not yet installed energy efficient equipment. Most projects listed as dropouts are not actually dropping out of the program, but are rather still in the process of determining what they will install at locations not already served by the program. Therefore, these cases would be more accurately categorized as “in-progress” rather than dropouts.

Participant satisfaction is high. Participants report a high level of satisfaction with the

program experience and National Grid as a result of participation. Over half of the tenants (54 percent) reported that their experience with the EnergyWise Program increased their satisfaction with National Grid as their energy provider.

The program is promoting energy efficiency upgrades. The group of building owners and managers we spoke with typically only implement upgrades when equipment fails or if there is turnover in a unit. The program is motivating them to replace equipment earlier than planned.

3.1.2 Challenges to Program Success

Because of the newness of this program, there are several challenges identified through the evaluation activities for the first year of the program. The challenges mentioned in the interviews are highlighted below, with suggestions for addressing them in the Recommendation Section.

The gas savings side of the Energy Wise program struggled early on. Staff indicated that

the initial individual metering requirements drastically limited the pool of eligible gas customers. Several staff also indicated that the current gas incentive levels do not seem to be high enough to influence participation of the property owners, which has compounded the problem of the small eligible customer pool. Some staff have heard property owners requiring at least 50 percent of the project cost covered before they can justify the expenditure.


3-3


Refrigerator cost-effectiveness requirements are unrealistic for the current service area. Staff report customers’ interest in the refrigerator rebate has driven overall interest in

the program. However, several staff stated that it is nearly impossible for customers to qualify for a refrigerator rebate if the unit is less than 17 years old (early replacement). Implementation staff reported that almost none of the tested refrigerators ended up cost-effective based on the current calculation. Property managers confirmed that this has led to frustration and confusion on their part as many were originally interested in the program for the refrigerator rebates.

There has been difficulty in selling the program to property owners. The interviewees

reported that the current economic situation is causing property owners to overlook measures that benefit tenants, but do not directly benefit themselves. In addition, insulation and weatherization improvements are high-cost measures that require advance budgeting. Interviewees found that the 25 percent program contribution was not enough to motivate customers to undertake this advance budgeting. They also indicated that gaining contact information and access to out-of-state property owners has been a challenge, which evaluators confirmed to be the case when they attempted to contact these owners to complete an interview.

Current marketing efforts have not been sufficient. Numerous members of the staff mentioned that more marketing materials are needed. They also indicated that the lack of marketing has been a factor in the current slow generation of energy audit leads. Mass marketing efforts have not reached the targeted customers and have created little activity on the program website.

Cost-effectiveness screening at the measure level has impacted customer participation. Several staff mentioned that it is not uncommon for projects with an overall

positive cost ratio on the project level to be passed over due to the requirement that each measure be cost-effective. They also reported that measure level screening is causing confusion with customers, as well as burdening property owners with tracking measures installed on a building-by-building basis. Staff feels that this leads to only a fraction of the measures a facility could receive being installed. Some property owners and managers confirmed that they did not understand why they were not eligible for the refrigerator or insulation rebates and why not all units are treated the same.

The InDemand database has little support for report generation. Staff mentioned that the lack of a reporting feature in the InDemand database does not allow for an up-to-date picture of the current progress of the program. This has contributed to redundancy as other databases with reporting features are being used to track the same information.

Program tracking data from InDemand is not 100 percent accurate. During the

evaluation, we found that there were inconsistencies around how measures were recorded or pulled for the participating tenant interviews. For example, interviews with property managers revealed that they often had water saving measures installed while the database records we received showed only lighting measures. Review of some of the applications from RISE showed the full range of measures installed which did not match what we received. It is not clear if the issues arise from data entry errors or how the queries pull data from the InDemand system.

There is a lack of understanding regarding who offers services and eligibility criteria. Building owners and managers find it very difficult to determine who provides programs to


3-4


assist the different types of multifamily properties with energy efficiency upgrades. There are varying eligibility criteria depending on the location of the property, whether buildings are individually or master metered, the type of equipment they are interested in replacing, and the number of units per building. Right now, many of the building owners and managers are not completing energy efficiency projects because they are not able to identify utility or other programs to assist them.

Multiple visits to properties lead to building owner and manager frustration. Some

building owners and managers have gone through multiple visits from the program to serve their properties for electric and then gas measures. Others received one visit for tenant units, then another for common areas. These respondents did not understand why the efforts could not be coordinated and that it is a waste of time to go through the properties twice.

The inability to serve the entire property removes some of the benefits to building owners and managers. In particular, the restriction that National Grid can serve the five to 50 unit properties takes away the ability for many building owners and managers with multi-unit properties to use the program and energy efficient equipment for marketing the units, especially if buildings in the complex do not fall within the five to 50 range.

3.1.3 Program Recommendations

The following section summarizes the most important and recurring themes we identified from all the partied interviewed along with our recommendations to address these issues.


Increase the coordination between National Grid marketing and the program manager and implementers. Program implementers have been cold-calling multifamily customers to create awareness and interest in the EnergyWise Program. This is very time intensive. Look into pulling customer records based on metering so the program can target appropriate customers with detailed marketing information.

Clearly define program benefits for customers. In order to address both economic barriers and the potential competition from other programs, National Grid will need to fully understand what other programs are offered to customers within their territory and position National Grid programs to offer the best product to customers. Benefits may be a simple application process, payment for their portion of the project costs only (eliminating the wait for a rebate check), knowledgeable contractors offering technical assistance, and more responsive program staff. As suggested by the new RISE marketing staff, a Frequently Asked Questions (FAQ) document for customers would also be useful.

Review program incentive levels for gas and weatherization measures. All program staff and implementation interviewees pointed to difficulties gaining interest from building owners and managers given what they feel are low incentive levels. The EnergyWise program has had no trouble meeting electric savings goals. That portion of the program is designed as a


3-5


direct install service. However, the program pays 25 percent of gas and weatherization measure costs, which the program team feels is too low to overcome the long lead time for decision making and promote interest from a group historically unwilling to spend money, given split incentives with tenants. Implementer experience with other multifamily programs, and comments implementers have received from customers, suggest that offering 40 to 50 percent would be much more successful. These measures may now be more attractive with the eligibility of master metered gas customers. There is interest from them in decreasing their gas bills, and the weatherization measures could be marketed as a way to achieve savings.

Explore the possibility of offering bill payment or loan options for major measures similar to other programs. Program staff mentioned that other programs for commercial customers often provide the option of paying off the amount on their monthly bills. Some customers informed staff the reason they are not participating has to do with the lower proportion paid by the utility for the projects. Including an additional financing option would provide implementation staff with another sales tool to help overcome the initial cost barrier mentioned by some customers for major measures such as insulation that are not fully covered by the program. One building manager suggested that option during our interviews and at least one-quarter of upstate Commercial and Industrial customer (participating and nonparticipating) interviewed through the upstate commercial and industrial process evaluation confirmed interest in financing offered by National Grid for energy efficiency upgrades. Another 25 percent were open to the option depending on factors such as management decisions, the size of the project, and the interest rate.

Investigate how to better serve an entire property in a more comprehensive manner. In conjunction with more targeted marketing program staff may want to consider incentivizing RISE to work more closely with multifamily customers in order to get them directed to the appropriate programs. Awareness of other programs and energy efficiency offerings appears to be a major gap for many of the multifamily building owners and managers. RISE is in the position to assist multifamily customers with a variety of energy efficiency needs by routing them to the Residential High-Efficiency Heating and Water Heating and Controls Program for heating and water heating measures, Enhanced Home Sealing for two to four unit properties, and even NYSERDA if needed for the more extensive renovations and retrofits that National Grid would not cover. RISE and ICF should have very clear and distributable information on what other programs are offered that multifamily customers can utilize. The ability to offer a more cross cutting service to multifamily managers would help alleviate some of the frustration and maximize savings potential.

Review the tracking database inputs for accuracy and completeness. There were

mislabeled measures in the program database download we received that were identified as we conducted interviews with building owners and managers. In addition, only one-third of the records had a correct phone number attached to the record. It would be worthwhile for tracking purposes and evaluation purposes to check a percentage of the database records against paper documents to ensure program information is accurate and recorded as expected by all parties using the data. For multifamily programs it is also important to capture the differences between local property managers and corporate decision makers to facilitate future communication and evaluation.

Consider presenting a case to the PSC regarding the requirement that each measure pass the cost effectiveness test. Interviewees have found that it is very confusing to

customers to go through the process of an audit, only to find that several of the recommended


3-6


measures are not eligible for program incentives because they are not individually cost effective. This becomes even more of an issue when program staff deal with a property owner who has multiple buildings and a measure may be cost-effective at one building but not at others. Interviewed parties would like to see more of a whole-building or whole-project approach if the overall project level cost ratio is positive to increase participation and building-wide efficiency.

3.2 PROGRAM ADMINISTRATION AND PROCESSES

This section presents the detailed findings from the interviews and surveys for topics related to program team structure and collaboration, program tracking and information, training, and tools and resources for program delivery.

3.2.1 Program Team Structure and Collaboration

The delivery responsibilities bounce back and forth between RISE and ICF, which could potentially be a cause for concern or confusion. There are multiple checks and balances throughout this process that could be prone to miscommunication or delay, but all feedback suggests that the process is actually working very smoothly and customers are being served in a timely manner and project work completed as expected.

Once customers are aware of the program, they can contact RISE for more information. RISE sends any leads to ICF who schedules and conducts the free energy audits. ICF sends all resulting audit information to RISE to calculate cost-effectiveness of each measure. Based on the data, RISE drafts a contract for the customer outlining what equipment is eligible for installation through the program. ICF then presents that contract to the customer, answers any questions they may have, and obtains the customer’s consent to continue. If the customer accepts the contract, RISE or their subcontractor installs the equipment. ICF staff will then conduct quality assurance checks on a portion of the installations.

RISE requires that a certificate of completion be returned by the customer that reports what was installed, the location of installation, asks five survey questions, and obtains signatures of the customer and program representative. RISE then invoices the customer for their portion of the project cost and National Grid for the remaining portion.

Additional assistance from within National Grid could decrease the amount of time spent locating eligible customers. While all delivery and implementation staff felt that they could manage the program workload with their current staffing levels, the only concern was the amount of time it was taking with cold-calling customers to find the ones eligible for the program and get them started with energy assessments. Implementers suggested having National Grid’s marketing department provide more assistance to reach out to building owners and managers, as well as specific targeting of those with individually metered electric or gas buildings.

3.2.2 Program Tracking and Information

Most of the project tracking information is detailed enough to provide progress against goals. Project information is recorded in the InDemand system for projects in process, but the implementers are keeping more detailed records on their own and tracking leads, which InDemand is not designed to do.


3-7


There may be some discrepancy in what is entered into the database or how data are pulled. As building owners and managers were called, we found that some of the measure information did not correspond with what was provided by the owners and managers when we talked with them. In many cases, the water saving devices were not accurately reflected in the sample file.

It was also difficult to determine from the contact information if the contacts listed were the decision-maker or local contacts. Program tracking and communications, either initial or follow-up, as well as evaluation efforts may be more effective if the contacts are more clearly tracked. For instance, two separate contact fields (with telephone numbers) could be included that identify the contact as a local property manager with whom the program may be communicating to schedule the energy assessment and installation of measure, and identify who is the person responsible for deciding if the property will participate and to whom the energy assessment report should be sent.

3.2.3 Quality Control

At the time of the program staff and implementation interviews, quality assurance and quality control (QA/QC) procedures had not yet started, but a QA/QC process is now in place and functioning. We identified early in the evaluation as we worked through the

program logic model with the program staff and implementers that the one phase of the program that had not fully launched was the QA/QC task. The program structure and contracts accounted for it; however, it had not actually been implemented for any sites. It was initially speculated that the push to reach gas savings targets was distracting resources.

Early after program launch the installation contractor checked a portion of their own installations, which was not the intent of the program structure. ICF, who is responsible for conducting the energy assessments, was also tasked with QA/QC, which included going back and visiting ten percent of the properties to verify installation, look for any health or safety issues, and review a series of questions with the property manager. This procedure is now being followed.

3.3 CUSTOMER AWARENESS AND MARKETING

This section provides findings on the effectiveness of program marketing by examining program awareness levels and sources of program awareness.

3.3.1 Program Awareness

Outreach is still a major challenge for the program. Corporate property managers and owners were typically cold-called by RISE Engineering, although a few may have learned about the program from the newspaper or on the National Grid website. During interviews with program staff and implementers, it was mentioned that the most effective way of reaching customers that resulted in actions had been through cold calls. However, this is a very time intensive method for reaching prospective participants.

Interviews with property managers and reviews of QA/QC forms confirmed that the vast majority of local property managers did not hear about the program until their corporate office notified them of the decision to participate. One local property manager submitted a proposal to corporate suggesting they participate in the program. If more information can be


3-8


disseminated to local property managers, they may be able to be of assistance in motivating corporate managers to choose to do work through the program.

One landlord, who is very active in his local landlord association, recommended more targeting of landlord associations, either through presentations to the association groups or articles in the association newsletters as a way to reach a larger audience.

Two of the property managers with multiple properties relied heavily on information from manufacturers, suppliers, and vendors for new equipment and energy efficient options. Building owners and managers are typically looking for communications regarding costs, potential savings, and deadlines.

There is a general lack of awareness of where to go for any energy efficiency assistance. Building owners and managers find it difficult to determine who provides programs to assist their different types of multifamily properties with energy efficiency upgrades. There are varying eligibility criteria depending on the location of the property, whether buildings are individually or master metered, the type of equipment they are interested in replacing, and the number of units per building.

Property managers with multiple properties first have to determine who can serve each property and research what utilities may have available for the particular type of equipment they are interested in replacing or installing. Current multifamily programs, including National Grid’s EnergyWise program, tend toward more straightforward direct install items. This still leaves the property managers looking into options for installing larger equipment or building envelope measures. Right now, many of the building owners and managers are not completing energy efficiency projects because they are not able to identify utility or other programs to assist them.

The allocation of five to 50 unit buildings to National Grid further complicates the move toward efficiency for building owners and managers that have varying building types per complex. NYSERDA programs are available for buildings with more than 50 units. Those building owners and managers who are aware that NYSERDA offers energy efficiency programs for buildings over 50 units often prefer the utility-run programs over NYSERDA’s, citing that the utilities seem to be better at getting the work done.

Tenants have little knowledge of the program until notified by their property managers. Seventy-five percent of tenants learned about the EnergyWise program through their property manager or management company. A variety of methods were used by property managers to alert tenants about what would happen as part of their participation in the program, including letters, fliers taped to doors, and phone calls. The amount of detail and number of notices varied depending on how long property managers had to notify their tenants.

Of those who received information from their property manager or management company, 60 percent said they received the time and date of installation, 29 percent received information about the equipment that would be installed, and 13 percent received an explanation of the EnergyWise program.

After information from property managers, another 13 percent of tenants learned about the program through RISE Engineering, likely during the installation visit. A few others learned of the program from sources such as literature or websites.


3-9


Seventy-six percent of the tenants receiving information indicated that it was sufficient to determine the benefits, to them, from participating in the EnergyWise program. One-third of tenants felt they had a choice of whether or not to participate.

3.3.2 Program Understanding

Considering that tenants are not the first point of contact with the program, a respectable proportion understands the purpose of the program and why National Grid offers it. At the beginning of the survey, the 73 respondents were asked about their understanding of the EnergyWise program. Forty percent of respondents indicated that they thought the purpose of the program was to save energy or be more energy efficient and another 21 percent felt the program would save them money on their energy bills. Twenty-five percent knew that someone was coming to install energy efficient equipment. However, one-quarter of the tenants surveyed had no knowledge of the program before the survey call.

Tenants were also asked why they believed National Grid offers the EnergyWise program to customers. Forty-five percent believe the utility offers the EnergyWise program so they can lower demand, save energy, and save money. Almost 30 percent felt National Grid was trying to help customers and save them money. Less common reasons mentioned included improving the environment (17 percent), to improve their reputation (12 percent), and because National Grid received government funds (six percent). Sixteen percent were not able to provide a reason.

It does not appear that the Completion Certificates are having the intended impact. To increase tenant knowledge of what was done in their unit, given that the decision to participate is most often made by the property owner or manager, the program leaves a completion certificate with each unit served by the program. A template for the completion certificate can be found in Appendix E. The main components of the Completion Certificate are:

Client name, location, contact information

Installation date

Measures installed, location, and quantity

Five-question survey

Check box to approve use of electricity consumption data to evaluate energy savings.

Just over half (52 percent) of all tenants recall receiving the completion certificate and another 27 percent could not recall if they had received it. Twenty-one percent confirmed they did not receive it.

For the half of tenants who remember receiving their completion certificates, 42 percent do not recall what information was included on the certificate. The rest remembered at least a few items, with over half recalling the list of equipment that was installed in their unit. Others remembered seeing the date of installation and other information about the program.


3-10


Figure 3-1. Recall of Information on Completion Form for Those Who Recalled Form

Many of the comments from tenants show that they paid little attention to the information they received on the completion form and after installation.

I just looked at it fast and signed it.

I threw it out, I don’t know, I only glanced at it.

I can’t remember.

3.4 EASE OF PARTICIPATION

Building owners and managers find the EnergyWise program easy to use. RISE Engineering was mentioned favorably during interviews with building owners and managers who feel they are providing high-quality service on a reasonable schedule and are flexible to tenant needs. RISE did a test run for one of the large management companies at one property. That worked well so they rolled it out to other properties

RISE made it understandable for everybody, we knew exactly what we were going to get and their breakdowns were very comprehensive. RISE had a payback worksheet that showed what was an honest and realistic guesstimate of what we were looking at for savings.

One property manager specifically mentioned appreciating that National Grid had contractors lined up to do the work through the program and they did not have to spend the time identifying contractors on their own.


3-11


Tenants found the program accommodating. Ninety-four percent of all tenants indicated they had no difficulties participating in the program and the rest did not know about the program. A few mentioned they liked that the program installed the equipment while they were at work or that they accommodated their odd schedule.

As can be seen in Table 3-1, the majority of surveyed tenants (60 percent) indicated they did not have a choice to participate in the program. In addition, about one third of respondents (34 percent) indicated that they were able to choose whether or not measures were installed in their unit.

Table 3-1. Tenant’s Reasons for Participation

Percent responding

I had no choice, the management company decided to participate 60%

Interested in lowering utility costs 23%

The right thing to do for the environment 5%

It was free 4%

Other 4%

Don't know 4%

It took no effort 3%

Total N=73

In general, 34 percent said they benefitted by saving money on their energy bill and 12 percent said they saved energy. Thirty-five percent specifically saw a decrease in their electric bills along with 20 percent who saw a reduction in their gas bills.

Another 13 percent said the new equipment was an improvement over what was previously installed. Thirty-four percent said there was no particular benefit they could name, while 17 percent said they did not know of any benefits they had realized.

Table 3-2. Benefits of Participation

Percent responding

No benefits 34%

Saved money on energy bills 34%

Don’t know 17%

The equipment installed is an improvement over the old equipment 13%

I saved energy 12%

Other 3%

I learned how to change my energy use behavior 1%

Total N=73

3.4.1 Barriers to Participation

The current economic situation has made the EnergyWise Program even more important to multifamily properties. Program staff and implementation contractors reported that the current economic situation is causing property owners to overlook measures that benefit tenants, but do not directly benefit themselves. In addition, insulation and weatherization improvements are high-cost measures that require advance budgeting. Program staff and implementer interviewees have found that the 25 percent program contribution is not enough to motivate


3-12


customers to undertake this advance budgeting. They also indicate that gaining contact information and access to out-of-state property owners has been a challenge.

The cost to participate was also a factor that most property owners and corporate managers closely considered. Multifamily properties in general operate on a very tight

margin, which makes these programs very attractive. One property manager mentioned that their property was 80 percent low income, and therefore, the property contribution to the program was even more of an issue. In addition to costs, a few property managers also had to take into account the age of their properties and how that affected the upgrades they were able to make. Although property managers typically do not have much funding to work with, which can be a major barrier to installing any energy efficiency improvements, they benefit from the energy assessment to determine property needs which allows them to prioritize work.

Building owners and managers experience difficulty in participation resulting from the logistics of notifying and disrupting residents. For many of the property owners and

managers, primary considerations before program participation are the amount of their time that will be required to work with program staff, notify tenants and respond to their questions, the unfamiliar contractors who will be on-site performing the work, and the inconvenience to tenants.

Tracking multiple programs and the number of units served is too much work for some property managers. The more properties a manager handles, the more tracking is needed. A further complication is when the manager oversees properties with a variety of buildings or a wide variety of properties. With the differences in programs offered and service providers by territory, tracking energy efficiency assistance is not a minor task.

While property owners and managers had positive experiences with the services offered by the EnergyWise program, the limitation of serving five to 50 units created issues for property managers whose complexes had buildings that were eligible as well as buildings that were not.

Some of the tenants benefited and at the same time my other tenants didn't, seemed like an overly rigid criteria. They told me that the buildings weren't eligible because of number of units. For the buildings that didn't qualify for the National Grid program, I wasn't informed about other programs to get those buildings covered. Tenants talk amongst themselves and I got a lot of questions about why so-and-so got them and they didn't. It put me in an awkward position of explaining why their building didn't qualify for program.

Multiple trips to properties created frustration for some property managers. Through interviews we found that it was common for the implementers to make several visits to a property. Sometimes it would be one pass through for CFLs and then a later round to install gas saving measures. Other times it was one visit to install measures in the tenant units, then another to handle improvements to common areas and management offices.

It seems like a big waste of time for three RISE guys to go through the properties and only install CFLs. I really wanted to have some gas measures installed for savings.

Some of the property managers are familiar with rebates and services offered by NYSERDA, but prefer the services offered by the National Grid EnergyWise Program.


3-13


Property managers are always actively looking for programs that will provide assistance, either technical or financial, for installing new equipment. A few have had experience with other utility programs and services through NYSERDA, but several prefer the services that their utilities, particularly National Grid, can offer. One property manager was in the process of working with NYSERDA partners for the Multifamily Performance Portfolio (MPP) program. Two other comments are highlighted below.

I heard about program from RISE Engineering, they seemed to be more competent than the NYSERDA and Energetic people who approached us individually. The others didn't have the whole package put together.

We had talked with someone from NYSERDA about assistance, but decided to pass due to the financing requirements.

My dealings with NYSERDA are not good. I would prefer to see all of the utility companies handling that themselves. It seems like you guys have the ability to get it done, while NYSERDA has been sitting on their hands for several years, which is why you're getting your money back.

3.5 DECISION-MAKING PROCESSES

Both the building owners and managers and the building tenants made decisions about installing energy efficient equipment through the program. However, the tenant decisions were restricted to allowing the implementation staff in and accepting program equipment once the building owner or manager had made the call to participate at the property.

3.5.1 Equipment Installation

The majority of energy efficient equipment installed through the program in 2010 consisted of CFLs. Ninety-six percent of tenants said they had CFLs installed in their units

through the program. Just over half also had low flow showerheads installed. Few tenants had faucet aerators (13 percent), hot water pipe wrap (eight percent), or tank wrap (seven percent) installed. Only five percent thought they had a programmable thermostat installed through the program, which is likely due to the recent addition of the measure to the program offerings.

Installation persistence was high for most equipment. The proportion of program-installed equipment that was still installed at the time of the tenant survey ranged from 100 percent for tank wrap to 83 percent for CFLs.

Table 3-3. Installation Persistence

Percent with All Equipment

Still Installed

Tank Wrap 100%

Low flow showerheads 98%

Faucet aerators 87%

CFLs 84%

Hot water pipe wrap 82%


3-14


The reasons for equipment removal were usually failure or dislike. It was mentioned by tenants that CFLs frequently failed and building owners and managers confirmed that issue. Of the 17 percent of tenants who removed CFLs, 58 percent cited the CFLs burning out as the reason for their removal and they typically only removed one bulb. Ninety-five percent of the CFLs still installed are working properly. A high percentage (84-93 percent) of the other equipment still installed is reported as working properly

3.5.2 Program Participation

The program was extremely influential in motivating building owners and managers to install energy efficient equipment. In some cases, building owners and managers had CFL,

low flow showerhead, and faucet aerator improvements on their lists to do and would have done them as units turned over. However, none of them would have done these full scale changes at the time they did without the program.

We’ve been switching to T8 and CFLs, low flow showerheads and faucet aerators as units turn over. We change out water heating as we need to work on them. This program definitely helped us get work done sooner.

I’ve been trying to drive down operating costs of the building and business with the economy so bad, so this fit with what I was trying to do anyway.

We had not considered any improvements and would have done nothing without the program.

Building owners and managers also benefitted from the energy assessment and learned what improvements to prioritize within their buildings.

Building owners and managers are happy with the direct install equipment, but many were initially interested in the program for the refrigerator rebates. Although satisfaction with the program is high overall, there are some concerns from building owners and managers that the program has not offered all that it said it would. Many of them were initially drawn to the EnergyWise program by the possibility of replacing the old, inefficient refrigerators located in many of the units.

Refrigerator standards were not realistic. We had old not frost free 15+ year old refrigerators that did not meet the standard for replacement with your program.

The program is also motivating individual tenants to become more energy efficient. Fifty-nine percent of tenants said they would not have installed any of the energy-saving equipment without the program. Seven percent of tenants have purchased a similar piece of energy efficient equipment for their unit since participating in the EnergyWise program, none of which received a rebate. Two tenants purchased more light bulbs and one purchased another showerhead with a low flow feature or rinse mode.

There is very little spillover associated with the EnergyWise Program to date for building owners and managers. A couple of the building owners and managers interviewed had looked into energy efficiency improvements outside of the EnergyWise Program. One mentioned air sealing and another was looking into furnace and boiler replacement, but only when it was needed since he has not been able to find a program to cover it.


3-15


3.6 PROGRAM SATISFACTION

3.6.1 Overall Satisfaction

Overall, building owners and managers are happy with the EnergyWise Program. For

most program aspects, at least eight of the building owners and managers were able to provide ratings on a scale of zero to ten, where zero was not at all satisfied and ten was extremely satisfied. All aspects were rated higher than 7.5. A few individual property managers had minor issues with the auditor and audit report, the installer, and the equipment performance.

Tenant satisfaction with the program overall is also high. All tenants were asked to rate their overall satisfaction with service provided by National Grid and various aspects of the EnergyWise Program on a scale of zero to ten, where zero was not at all satisfied and ten was extremely satisfied. Seventy-nine percent were satisfied with National Grid’s service overall (rating of eight to ten).

All ratings were above a five, except for four tenants who gave National Grid’s service a zero rating (very dissatisfied). Two of these respondents mentioned that National Grid's phone service was the reason for their dissatisfaction with National Grid, while the other two respondents gave the prices they pay for service as the reason for their dissatisfaction.

Figure 3-2. Tenant Overall Satisfaction with the EnergyWise Program

3.6.2 Satisfaction with Specific Program Aspects

Specific aspects of the programs were also highly rated by tenants. The 45 percent of tenants who were at home during the installation of the energy-saving equipment were asked to rate their level of satisfaction with the installation process, interaction with program staff, and the time it took for program staff to install the equipment on the same scale of zero to ten, where


3-16


zero was not at all satisfied and ten was extremely satisfied. All three of these program aspects received very high ratings, showing that tenants who interacted with the program had very good experiences.

The aspects with the lowest mean rating were information related. Consistent with the fact that many of the tenants were unaware that their unit was being served by the program, information about the program and new equipment received the lowest satisfaction ratings of all aspects rated.

Table 3-4. Tenant Satisfaction with Program Aspects

Program Aspect Number of applicable

respondents Mean

Percent Satisfied

(8-10 rating)

Percent Extremely Satisfied

(10 rating)

Time it took to install equipment 32 9.2 90% 74%

Installation process 31 8.9 83% 70%

Interactions with program staff 35 8.8 82% 74%

Type of equipment 68 8.8 82% 49%

Performance of equipment 72 8.5 80% 49%

Information provided about new equipment 63 7.5 63% 40%

Information provided about program 61 7.3 61% 36%

Program participation increases tenant satisfaction with National Grid. Satisfaction with National Grid as an energy provider increased for 54 percent of the tenants as a result of participation in the EnergyWise Program. For another 45 percent of tenants, the program participation made no difference in their satisfaction.

Figure 3-3. Tenant Change in Satisfaction as a Result of the EnergyWise Program


3-17


Thirty-three percent of these respondents indicated their increased satisfaction was due to National Grid's effort to save energy and concern for the environment. Another 14 percent were happy that National Grid was doing something to help reduce their energy bills. Others liked that the equipment was installed for free and that the installations went smoothly.

For the tenants who answered that their participation in the program did not change their satisfaction with National Grid, many said they saw no difference in their energy use. Others were unaware of the program.

Out of the 73 tenant respondents, those who saw a reduction in their electric bill since the program (34 percent) were more likely than those that had not (43 percent) to report an increase in satisfaction with National Grid as a result of their participation.

Of the 29 respondents who paid their own gas bill, 12 were unsure if they had seen a decrease since the program. This is in comparison to the 17 respondents who were unsure of a reduction in their electric bill.

3.6.3 Suggested Program Changes

Building owners and managers feel the refrigerator rebates are misleading. Many of the building owners and managers were initially interested in the program for the refrigerator rebates. But even with 15 to 20 year old appliances, none have qualified given the stringent savings requirements. They feel the refrigerator rebate should not have been offered if there were no appliances that would qualify for the rebate.

Requested equipment additions are likely offered through other utility programs. Building owners and manager mentioned interest in adding equipment such as outdoor lighting, boilers, air conditioning, setback thermostats, and enhancements to insulation. Some mentioned coverage across common areas and more building types as a possible improvement to the program. This may be more of a communication and awareness issue, since the properties are likely eligible to receive rebates for additional equipment or building areas through other programs.

Information is the primary change suggested by tenants. Although 55 percent of the tenants would change nothing about the program, consistent with the lower satisfaction ratings for information provided about the program, 26 percent of tenants said they would recommend the program provide them with more information.

Some improvements to the equipment installed were suggested by tenants. Fourteen

percent of tenants suggested improving the direct install equipment. For example, tenants mentioned that the CFL bulbs are slow to come on. Others have experienced some leaks with the showerhead and faucet aerators or issues with the pressure from the showerhead. However, building owners and managers have experienced far fewer complaints about the water saving devices than what they originally anticipated.

Few tenants suggested additional types of equipment for the program. When tenants were asked whether they would change any features of the program, only three respondents wanted additional equipment types included. One wanted some type of furnace tune-up, another wanted insulation to be improved, and the last respondent did not provide a specific recommendation.


3-18


3.7 PARTICIPANT CHARACTERISTICS

Many of the participating and in-progress or dropout property owners and managers we spoke with were the ultimate decision makers at the company--the CEO, President, or Owner. Several others indicated they were the facilities engineer, operations/facility/general manager, director, or vice president.

Both single-building owners and multiple-building managers are showing interest in the program. At the time of the evaluation interviews, all participants had buildings with units that were individually metered for gas or electric service. Interviews were conducted with many of the local property managers, who were usually more likely to be available for an interview than their corporate management.

As can be seen in Table 3-5, tenants were most likely to be between the ages of 24 and 34 years old (29 percent), with the second most common age category being 64 to 74 years old (18 percent). Over half of respondents have at a minimum a college degree (62 percent). The most common income category for tenants was $20,000 to $40,000 (28 percent) with $40,000 to $75,000 (21 percent) the second most common. Tenants were most likely to have lived at their current address less than five years (66 percent).

Table 3-5. Tenant Characteristics

Percent of Tenants

Respondent's age category

Under 24 8.2%

24 to less than 34 29.0%





74 or over 9.4%

Total N=72

Respondent's education category

8th grade or less .0%

Some high school 4.0%

Completed high school 13.9%

Some college 18.2%

Completed college 37.2%

Graduate studies or advanced degree 25.2%

Total N=72

Respondent's income category

Less than $20,000 17.7%

$20,000 to less than $40,000 27.5%

$40,000 to less than $75,000 20.7%

$75,000 to $125,000 19.2%

Over $125,000 1.5%

Total N=63

Number of years respondent has lived at current address

Less than 1 year 3.0%

1 to less than 5 years 65.8%




3-19


20 or more years 6.2%

Total N=73

4-1


4. RECOMMENDATIONS

Overall feedback from program staff, implementers, building owners and managers, and tenants involved with the EnergyWise Program indicate that after a late launch and some mid-stream program adjustments, the program is on the right track and is achieving savings as anticipated. The program was well over its electric savings goal and close to achieving its gas goal in 2011. Participants are highly satisfied with their program experience and participation has improved their perception of National Grid.

At the same time, the program faces several internal challenges, most specifically those associated with effective outreach and cost effectiveness testing. Current marketing efforts are not reaching multifamily customers efficiently and multifamily customers are unsure of where to turn for energy efficiency upgrades. In addition, the inability for any refrigerators to pass the cost effectiveness testing, individual measure cost effectiveness testing, and multiple property visits are testing the patience of some building owners and managers.

Below are recommendations to consider that address some of the opportunities identified.


Increase the coordination between National Grid marketing and the program manager and implementers. Program implementers have been cold-calling multifamily customers to create awareness and interest in the EnergyWise Program, which is very time intensive. Look into pulling customer records based on metering so the program can target appropriate customers with detailed marketing information.

Clearly define program benefits for customers. In order to address both economic barriers

and the potential competition from other programs, National Grid will need to fully understand what other programs are offered to customers within their territory and position National Grid programs to offer the best product to customers. Benefits may be a simple application process, payment for their portion of the project costs only (eliminating the wait for a rebate check), knowledgeable contractors offering technical assistance, and more responsive program staff. As suggested by the new RISE marketing staff, a Frequently Asked Questions (FAQ) document for customers would also be useful.

Review program incentive levels for gas and weatherization measures. All program staff

and implementation interviewees pointed to difficulties gaining interest from building owners and managers given what they feel are low incentive levels. The EnergyWise program has had no trouble meeting electric savings goals. That portion of the program is designed as a direct install service. However, the program pays 25 percent of gas and weatherization measure costs, which the program team feels is too low to overcome the long lead time for decision making and promote interest from a group historically unwilling to spend money given split incentives with tenants. Implementer experience with other multifamily programs, and comments implementers have received from customers, suggest that offering 40 to 50 percent would be much more successful in encouraging customer participation in the

4. Recommendations

4-2


program. These measures may now be more attractive with the eligibility of master metered gas customers. There is interest from them in decreasing their gas bills, and the weatherization measures could be marketed as a way to achieve savings.

Explore the possibility of offering on-bill payment or loan options for major measures similar to other programs. Program staff mentioned that other programs for commercial customers often provide the option of paying off the amount on their monthly bills. Some customers have told staff the reason they are not participating has to do with the lower proportion paid by the utility for the projects. Including an additional financing option would provide implementation staff with another sales tool to overcome the initial cost barrier mentioned by some customers for major measures such as insulation that are not fully covered by the program. One building manager suggested that option during our interviews and at least one-quarter of upstate Commercial and Industrial customers (participating and nonparticipating) interviewed through the upstate commercial and industrial process evaluation confirmed interest in financing offered by National Grid for energy efficiency upgrades. Another 25 percent were open to the option depending on factors such as management decisions, the size of the project, and the interest rate.

Investigate how to better serve an entire property in a more comprehensive manner. In conjunction with more targeted marketing program staff may want to consider incentivizing RISE to work more closely with multifamily customers in order to get them directed to the appropriate programs. Awareness of other programs and energy efficiency offerings appears to be a major gap for many of the multifamily building owners and managers. RISE is in the position to assist multifamily customers with a variety of energy efficiency needs by routing them to the Residential High-Efficiency Heating and Water Heating and Controls program for heating and water heating measures, Enhanced Home Sealing for two to four unit properties, and even NYSERDA if needed for the more extensive renovations and retrofits that National Grid would not cover. RISE and ICF should have very clear and distributable information on what other programs are offered that multifamily customers can utilize. The ability to offer a more cross cutting service to multifamily managers would help alleviate some of the frustration and maximize savings potential.

Review the tracking database inputs for accuracy. There were mislabeled measures in the program database download we received that were identified as we conducted interviews with building owners and managers. In addition, only one-third of the records had a correct phone number attached to the record. It may be worthwhile to check a percentage of the database records against paper documents to ensure program information is accurate and recorded as expected by all parties using the data.

Consider presenting a case to the PSC regarding the requirement that each measure pass the cost effectiveness test. Interviewees have found that it is very confusing to customers to go through the process of an audit, only to find that several of the recommended measures are not eligible for program incentives because they are not individually cost effective. This becomes even more of an issue when program staff deal with a property owner who has multiple buildings and a measure may be cost-effective at one building but not at others. Interviewed parties would like to see more of a whole-building or whole-project approach if the overall project level cost ratio is positive to increase participation and building-wide efficiency.

A-1


APPENDIX A: PROGRAM/IMPLEMENTATION STAFF INTERVIEW GUIDE

This instrument is a guide to be used when conducting qualitative in-depth interviews with program staff and the program implementation contractor. Not all questions are applicable for all persons interviewed.

Interview Objectives:

Identify program activities, inputs (resources), outputs, short to medium term outcomes and long term outcomes for use in developing program logic models

Identify important influences on program operation and achievements

Identify issues that should be incorporated into customer surveys

Characterize program operations including staffing and subcontracting, budgets, outreach activities and marketing, types of persons served, and other stakeholders involved

Identify areas that need improvement

Describe your role with the program

1. Describe your responsibilities regarding the EnergyWise program in Upstate New York?

a. When became involved

2. Please describe the other staff within National Grid and RISE who are involved in managing/implementing this program (e.g., energy use assessors, installers, other trade allies, data tracking, marketing):

a. Number of people, their locations, their responsibilities

3. What was the process for selecting the implementation vendor?

B. Program Design and Marketing

1. What was your involvement in the program design?

a. Was the program patterned after another program(s)? If so, were any modifications made to improve the program design?

b. What are the program goals?

c. How are the program goals communicated to you and from you to others (staff, RISE, ICF International)?

d. Do you expect the program will achieve it’s 2010 goals? If not, why not?

2. Can you briefly describe how the program operates?

a. What is the sequence of steps for participation?

b. What is the sequence of steps for implementation?

c. Leveraging other National Grid programs?

d. Rebate application

A-2


3. Please describe the target market.

a. How is the program being marketed to customers? (Probe on RISE and National Grid efforts such as website, direct mail, email blasts, collateral, events and outreach, trade newsletters, print advertisements).

b. How effective do you feel each of these methods has been in identifying and enrolling potential participants?

c. What additional marketing and outreach is needed?

d. Are these marketing efforts directed towards landlords, residents, or both?

4. What are major barriers to participation?

a. Are the marketing/recruitment efforts designed to build on customers’ reasons for participation and minimize reasons for nonparticipation?

b. What percent of customers choose not to participate after expressing initial interest? Why is that?

c. Specifically, what are the biggest challenges that you face with regards to meeting gas goals?

d. What are the different barriers that residents face to participation compared with landlords?

5. How are your program costs being tracked? (Probe: Examples of costs are staff resources, incentives, implementation, and marketing materials.)

6. How is participation uptake and experiences as compared to your programs in New York and in other states? If different than expected, why do you think this is? (Probe for differences between landlords and residents).

7. Do customers understand who is sponsoring the program? Is there any confusion with the previous NYSERDA program?

C. Program Operations

1. How much flexibility do you have in implementing the program?

2. Describe your communications and working relationship with the National Grid program manager/implementation contractor.

a. (RISE, ICF) Is National Grid support sufficient? b. Are program requirements clearly understood? c. In what areas could support and/or communications be improved?

3. Do you feel you have sufficient staffing resources to efficiently deliver the program? If not, what additional resources are needed?

4. In your opinion, what is the level of customer satisfaction with the various aspects of the program (energy assessment, types of low-cost measures installed, types of eligible higher cost measures, participation process, rebate level, rebate delivery, etc.)? How can satisfaction be improved? (Probe for differences in satisfaction between landlords and residents).

5. What aspects of the program implementation are working well? Which are not working well?

A-3


6. Do the incentive levels seem appropriate? If not, why not? What, if any, changes in the incentive levels do you think may be needed?

7. Can you describe any training efforts that your firm conducts with program staff about the Upstate National Grid EnergyWise Program? Who delivers the training? Who is the target audience? What is the primary purpose of the training? How effective have these trainings been? Will these change in the future? Why?

8. What do you see as future challenges to the program? Suggestions for overcoming these challenges?

D. Quality Assurance

1. What types of quality assurance activities, if any, are being done (probe on timing and amount)—i.e., what percent of completed projects are selected for a follow-up inspection? Who is responsible for this? Suggestions for improvement?

2. How are customer complaints documented, reported, and resolved?

a. How many customer complaints are registered as a percent of total participants to-date?

b. What is the nature of these complaints?

3. (IF RISE or ICF) In your experience, what percentage of the free measures (e.g., CFLs, low-flow showerheads) stay installed for at least 12 months?

E. Program Database and Tracking.

1. Who is responsible for updating the tracking system? How often is this done?

2. Who is responsible for tracking system checks and corrections? How are checks and corrections done?

3. How often do you prepare reports or updates for others in National Grid? What types of regular reports or updates do you provide? Is all data that you collect entered into the tracking system?

4. Have there been any difficulties with the program tracking system?

F. Evaluation

1. What do you hope to learn from this evaluation?

2. Do you have any specific questions that you want to make sure are included in primary data collection activities with customers?

G. Other Suggestions for Improvement?

Lastly, if you could change one thing about the program, what would that be? How would you improve the program?

B-1


APPENDIX B: BUILDING OWNER GUIDE

Note: This topic guide will be used by consulting staff to guide interviews with participating building owners in the EnergyWise program to investigate the key researchable issues for the process evaluation. This document is a guide and not a structured interview script; interviewers are encouraged to add additional questions, skip topics that are not applicable, or probe extensively as warranted.

This topic guide will be used to interview both customers who participated in the EnergyWise program and those who dropped out using the definitions below:

Participating building owner/manager = those who had building assessments and installed equipment. (Program code = P)

Drop out building owner/manager = those who had building assessments but installed no equipment. (Program Code = DO)

Introduction (ALL)

I1 I’m with Tetra Tech, an independent research firm. We have been hired to assist National Grid in reviewing their energy efficiency programs. [If you would like to talk with someone about this study, feel free to call Jill Falk at 781-907-2231.]

According to the information I have, you participated in the EnergyWise program for your multi-family building(s) at [ADDRESS] in 2010. I would like to ask you some questions about your experiences with the program, which will assist the program manager in determining how the program could be improved to serve multi-family properties like yours.

Were you personally involved in the decision of whether or not to participate in this program?

I2 Was anyone else within or outside your organization involved in the decision of whether to participate in this program? Who? How are they affiliated with this property and what was their role in the decision?

I3 How is this property metered for electricity and natural gas? (individual/master, gas/electric)

Context and Decision-making (ALL)

C0 What is your job title? What was your role in National Grid’s EnergyWise Program?

C1 How many multi-family properties do you own or manage? Where are they located? How long have you owned or managed multi-family properties? What energy efficiency measures, if any, have you undertaken on your properties prior to the EnergyWise Program? Were these measures offered by another utility, NYSERDA, or State/Federal Agency?

C2 How many of your properties have received energy audits through the EnergyWise program? At how many have you chosen to installed equipment based on the audit

B-2


recommendations? What were the reasons for deciding to install equipment? What factors influenced your decision?

C3 Why did you decide to participate in the EnergyWise Program? Any other reasons? Was there anything that made it difficult to participate or made you think twice about participating?

C4 How did you hear about the EnergyWise program? Anywhere else? (Probe all sources, Including utilities, auditors, website) What did you hear about it? Did you or your staff actively seek out information? Where did you look?

(If anything other than word of mouth - marketing of any sort): What was it about the marketing that you noticed or found compelling? Do you remember what you liked or disliked?

Did you visit National Grid’s website to look for information? ( If yes) What was your opinion of the information on the site and how easy it was to find?

C5 What are the major obstacles, if any, that you face when considering or seeking approval for energy efficient improvements at your multi-family properties?

C6 Are there different obstacles to consider when installing equipment in the tenants’ units vs. other areas (including common areas) of the building? What are those factors?

Energy Audit Recommendations (ALL)

E1 As part of the program, ICF conducted an on-site energy audit to identify potential opportunities to increase the efficiency of your equipment and your building. Is this correct? (If not correct, clarify what did happen.)

E2 What areas of your property were served by the audit (common areas, individual units, specific areas)? Did any of your individual tenants participate?

E3 After the energy audit, did ICF provide you with a written report detailing the findings of the audit? What was your reaction to the written report? (Probe about the specificity of the information and the amount of detail provided. Were there other opportunities that weren’t in the report that you felt should be?)

E4 Did the written report give you more information and detail than you expected, about what you expected, or less than you expected? Was it understandable?

E5 How could ICF have improved the written report he or she provided you with?

E6 Had you considered implementing any of those recommendations in the report before you talked with ICF? (Probe to understand what they had been considering doing and for how long they had been considering it.)

E7 What would you have done (and when would you have done it) if you had not participated in the EnergyWise Program?

B-3


E8 (IF NOT IMPLEMENTED) Do you plan to follow through with any recommendations? If so, what do you hope to do and when do you think this might happen?

E9 (IF NOT IMPLEMENTED AND DO NOT PLAN TO FOLLOW THROUGH) Why is it that you don’t think you will follow through with any of the recommendations? (PROBE: cost, don’t feel is needed, don’t know who to have do the work, will not benefit me, etc.)

E10 (IF NOT IMPLEMENTED AND DO PLAN TO FOLLOW THROUGH) Will you install any of the recommended measures through National Grid’s EnergyWise program? Why or why not? (PROBE: If won’t install through National Grid, will they install through another program? Why that other program?)

E11 Do you plan to complete any (other) energy efficiency renovations at this building in the future? If so, what do you hope to do and when do you think this might happen? Are you planning on working with any state or utility-run program to implement those renovations?

E12 (If not mentioned above) Are you aware of any other state or utility programs that provide energy audits or services to help you conserve energy in your properties? If so , what are they?

E13 Have you spoken with anyone from those programs about what they offer? Were you eligible to participate in their program? Why did you choose National Grid’s program?

Installed Measures (Participants only)

M1 According to the program records, you installed [MEASURES] at your property through the EnergyWise program. (Probe to get correct information if needed)

M2 [If MEASURE=Lighting/showerhead/aerator] Did the new [MEASURE] replace any existing equipment? What type?

M3 As far as you know, is this equipment still installed? (If not, why not?)

M4 Is there any additional equipment you would to see eligible through the program?

M5 Do you have any feedback on this equipment? (Probe for specific positive/negative feedback)Have you received any feedback from your tenants?

M6 Have you seen any changes in tenant turn-over due to the installed equipment? Have you made any changes in how your market your building due to the installed equipment?

M7 Have you noticed any changes in your energy bills as a result of your participation in the program? What type of change?

M8 Aside from what we’ve already talked about, have you installed any additional energy efficient equipment on your own as a result of participating in the program? If so, what equipment have you installed? Why did you purchase it?

B-4


M9 And as a result of the program, have you made any operational changes to the way you maintain your multi-family properties? If so, what are those changes?

M10 Has your marketing of these rental units changed at all since participating? Has there been any change in the rental fees? What type?

Program Marketing

A1 How would you prefer to receive news and information about programs like the EnergyWise program? How could you be most effectively reached? (Examples include email newsletters, phone calls, direct mailings, websites, etc)

A2 If we were creating an ad or direct mail piece about this program and it was going to be mailed to building owners such as yourself, what points should be made? Explain

A3 When you are thinking about renovating one of your multi-family properties or replacing equipment, what do you feel are the best sources of information? How do they influence your decision? (Examples include internal staff, contractors, design professionals, secondary research, government agencies, or the utility.)

A4 How were tenants informed about the building’s participation in the program?

A5 When do you typically make improvements to your multi-family property? (Probe to see if wait until unit is vacant, make them as needed, have some type of planning horizon in years, etc.)

Program Procedures and Satisfaction (ALL)

P1 Overall, what aspects of the EnergyWise program worked the best for you?

P2 How could the program be improved to better meet your needs? What, if any, problems did you face while participating? Is there any other assistance you were expecting or would have found useful?

P3 On a scale of 0 to 10, where “0” is “not at all satisfied” and 10 is “very satisfied”, how would you rate your satisfaction with the following (if applicable):

The energy auditor who inspected your property

The energy audit report

Interactions with program staff

The contractor who conducted the renovations

The performance of the free equipment

The performance of the completed renovations (including any equipment installed)

The EnergyWise program overall.

P4 Has your experience with the EnergyWise program changed your perception of National Grid in any way? How? Are you more or less satisfied (or no change)?

B-5


P5 What benefits, if any, have you realized at this property as a result of participating in the EnergyWise program?(If not mentioned, probe about greater understanding of how energy efficiency applies to their building).

P5.1 If you could change one thing about the EnergyWise Program, what would that be?

P6 Would you consider participating or have you already participated in the EnergyWise program at other multi-family properties?

P7 (If has already) Was that experience any different than your experience at this location?

Conclusion

END Thank you. Those are all of my questions. If I do have any follow-up questions, is it all right if I call you back or contact you via email? (Obtain email address if necessary)

C-1


APPENDIX C: TENANT SURVEY INSTRUMENT

National Grid EnergyWise Tenant Survey

INTRODUCTION

INTRO1 Hello, my name is [INTERVIEWER NAME], and I'm calling on behalf of National Grid.

May I please speak with [TENANT NAME]?

1 Reached respondent or someone else in household 2 Other [SKIP TO DISPOSCR]

INTRO2 I’m with Tetra Tech, an independent research firm. We have been hired to assist National Grid in reviewing their energy efficiency programs. You may have received a letter a few days ago explaining the purpose of this call.

I am calling to learn about your experiences with the free energy saving items you received through National Grid’s EnergyWise Program. A contractor came in and installed some energy efficiency measures like new compact fluorescent light bulbs, showerheads and other items in the apartment units in your building.

I'm not selling anything; I'd just like to ask about your experiences with this program. Your responses are confidential and only used to help improve energy saving programs. For quality and training purposes this call will be recorded.

INTRO3 [WHY ARE YOU CONDUCTING THIS STUDY: Studies like this help National Grid better understand customers’ need for and interest in energy efficiency programs and services.]

[TIMING: This survey should take about [10] minutes of your time. Is this a good time for us to speak with you? IF NOT, SET UP CALL BACK APPOINTMENT OR OFFER TO LET THEM CALL US BACK AT 1-800-454-5070]

[SALES CONCERN: I am not selling anything; we would simply like to learn about your experience with the free energy saving equipment you received through National Grid’s EnergyWise program. Your responses will be kept confidential. If you would like to talk with someone from National Grid about this study, feel free to call Melissa Piper at 315-428-5002.]

RECALL OF PARTICIPATION/IDENTIFICATION OF DECISION MAKER

I0 Just to confirm, do you currently reside at [TENANT ADDRESS]?

1 Currently reside at address 2 No longer reside at address [THANK AND TERMINATE]

C-2


I1 Our program records show that your building received energy saving equipment such as low flow shower heads, faucet aerators, pipe wrap, and compact fluorescent light bulbs which were installed in your unit through National Grid’s EnergyWise program

[IF "installation date" is present: around [INSTDATE]].

Do you recall this?

1 Yes [SKIP TO A1] 2 No -8 DON’T KNOW -9 REFUSED

I2 [ASK IF I1<>1] Is it possible that someone else in your household would know about having this equipment installed through National Grid’s program?

1 Yes [SKIP TO I4] 2 No -8 DON’T KNOW -9 REFUSED [TERMINATE]

I3 The EnergyWise program used a crew of installers to install energy saving equipment in the apartment units in your building. The crew installed all these items at no cost to property owners and occupants. Are you sure you don’t recall these free energy saving items being installed in your building through this program?

[IF NEEDED: Equipment may have included such as compact fluorescent light bulbs, low flow showerheads, faucet aerators and hot water pipe wrap]

1 Don’t remember [TERMINATE, RECORD ANY COMMENTS BEFORE EXITING]

2 Remember [READ, “Great, then let’s continue”, SKIP TO A1] -9 REFUSED [THANK AND TERMINATE]

I4 May I please speak with that person? [RECORD PHONE NUMBER]

1 Yes [BEGIN THE SURVEY AGAIN WITH THIS NEW RESPONDENT] 2 No [THANK AND TERMINATE] -8 DON’T KNOW [THANK AND TERMINATE] -9 REFUSED [THANK AND TERMINATE]

C-3


Tenant Characteristics

A1 Please describe your understanding, before today, of what the EnergyWise program offers.

[RECORD VERBATIM]

[SEE OTHER/SPECIFY RESPONSES IN OPEN ENDS]

*A1_Cat Categorized responses to A1.

*A1_Cat_1 Don't know/nothing

*A1_Cat_2 To save money

*A1_Cat_3 To save energy/be more energy efficient

*A1_Cat_4 Coming to install energy efficient equipment

*A1_Cat_5 Other

For A1_Cat_1 through A1_Cat_5

0 Not mentioned 1 Mentioned

C1 Do you pay your own natural gas bill or does the property owner pay the bill?

1 I pay for natural gas 2 My property owner/landlord pays 3 Do not have natural gas -8 DON’T KNOW -9 REFUSED

C2 Do you pay your own electric bill or does the property owner pay the bill?

1 I pay for electric 2 My property owner/landlord pays 3 Do not have electricity -8 DON’T KNOW -9 REFUSED

C-4


AWARENESS AND PARTICIPATION

P0 The EnergyWise program used a crew of installers to install energy saving equipment

such as compact fluorescent light bulbs, low flow showerheads, faucet aerators and hot water pipe wrap in the apartment units in your building. The items were installed at no cost to property owners and occupants. Your property manager may have provided information regarding the program at the time the equipment was installed.

Were you aware that your unit was served by this program before our call today?

1 Yes 2 No -8 DON’T KNOW -9 REFUSED

P1 [If P0=1] How did you learn about the EnergyWise program?

[DO NOT READ; SELECT ALL THAT APPLY]

P1_1 My property manager/management company

P1_2 From National Grid

P1_3 From RISE Engineering

P1_4 Other tenants in my building/complex

P1_5 Other [SPECIFY]

P1_6 DON’T KNOW

P1_7 REFUSED

For P1_1 through P1_7



C-5


P1A [If P0<>1 or P1<>1] Did you receive any information about the EnergyWise program from your property manager or management company?


P1B [If P1=1 or P1A=1] What specifically did you receive from your property manager, or management company, explaining the program and what would be done in your unit?

[RECORD VERBATIM]


*P1B_Cat Categorized responses to P1B.

*P1B_Cat_1 Time and date of installation

*P1B_Cat_2 Explanation of EnergyWise program

*P1B_Cat_3 Equipment that would be installed

*P1B_Cat_4 Other

For P1B_Cat_1 through P1B_Cat_4


C-6


P2 Why did you participate in the program?


P2_1 Interested in lowering my utility costs

P2_2 It was free

P2_3 The right thing to do for the environment

P2_4 I had no choice, the management company decided to participate

P2_5 Other [SPECIFY]

P2_6 DON’T KNOW

P2_7 REFUSED

*P2_8 Took no effort

For P2_1 through P2_8



P3 Were you able to choose whether or not the energy efficient equipment was installed in

your unit?


C-7


P4 [If P1=1 or P1a=1] Was the information you received from your property manager, or management company, sufficient to allow you to determine the benefits, to you, from participating in the EnergyWise program?

1 Yes 2 No [PROBE: What else would have been helpful?] -8 DON’T KNOW -9 REFUSED

P5 Why do you think National Grid offers this energy efficiency program to its' customers?

[RECORD VERBATIM]


*P5_Cat Categorized responses to P5.

*P5_Cat_1 Utility can lower demand/save energy/save money

*P5_Cat_2 Better for the environment

*P5_Cat_3 Save customers money/help customers

*P5_Cat_4 Receive government funds

*P5_Cat_5 Improve reputation

*P5_Cat_6 Don't know

*P5_Cat_7 Other

For P5_Cat_1 through P5_Cat_7


C-8


EQUIPMENT INSTALLED

E0 Were you at home when the energy efficient equipment was installed in your unit?


E1 What types of energy saving equipment were installed in your unit through the program?


E1_1 Compact fluorescent bulbs (CFLs)

E1_2 Low flow showerheads

E1_3 Faucet aerators

E1_4 Hot water pipe wrap

E1_5 Water heater tank wrap

E1_6 Other [SPECIFY]

E1_7 DON’T KNOW

E1_8 REFUSED

*E1_9 Programmable thermostat

For E1_1 through E1_9



C-9


E1A [IF E1 not only DK or REF] Program records also indicate

[MEASURES NOT MENTIONED IN E1] was/were also installed in your unit through

the program.

To confirm, this/these was/were NOT installed in your unit?

[Only show measures in sample not chosen in E1]

E1A_1 Correct – not installed

E1A_2 Compact fluorescent bulbs (CFL) also installed

E1A_3 Low flow showerheads also installed

E1A_4 Faucet aerators also installed

E1A_5 Hot water pipe wrap also installed

E1A_6 Water heater tank wrap also installed

E1A_7 DON’T KNOW

E1A_8 REFUSED

For E1A_1 through E1A_8


C-10


For each measure mentioned in E1 that is not in the sample:

E1B [IF E1 not only DK or REF] How many [MEASTYPE] were installed in your unit?

[IF Hot Water Pipe Wrap: Please ask for in feet]

E1B_1 CFLs

E1B_2 Low flow showerheads

E1B_3 Faucet aerators

E1B_4 Hot water pipe wrap

E1B_5 Water heater tank wrap

For E1B_1 through E1B_5

__ [RECORD QUANTITY] -8 DON’T KNOW -9 REFUSED

E2 Did you receive a certificate of completion once the energy efficient equipment was

installed?


C-11


E3 [If E2=1] What information was provided on the certificate of completion?

[RECORD VERBATIM]


E3_Cat Categorized responses to E3.

E3_Cat_1 Don't know/remember

E3_Cat_2 List of equipment that was installed

E3_Cat_3 Date equipment was installed

E3_Cat_4 Short survey on satisfaction

E3_Cat_5 Contact information

E3_Cat_6 Information on savings or program

E3_Cat_7 Other

For E3_Cat_1 through E3_Cat_7


SATISFACTION

S1 I am going to read a list to you. Please rate your level of satisfaction for each item using a scale of 0 to 10 where 0 is very dissatisfied and 10 is very satisfied. How satisfied were you with…

[READ AND ROTATE S1,S2A – S1,S2G]

S1A The type of equipment installed through the program?

__ RECORD RESPONSE -7 Not Applicable -8 DON’T KNOW -9 REFUSED

C-12


S2A [If S1A<=6] Why did you give it that rating?

[RECORD VERBATIM]


S1B [If E0=1] The installation process?


S2B [If S1B<=6] Why did you give it that rating?

[RECORD VERBATIM]


S1C [If E0=1] The amount of time it took program staff to install the equipment?


S2C [If S1C<=6] Why did you give it that rating?

[RECORD VERBATIM]


S1D The interactions with program staff, if any?


C-13


S2D [If S1D<=6] Why did you give it that rating?

[RECORD VERBATIM]


S1E The performance of the new equipment?


S2E [If S1E<=6] Why did you give it that rating?

[RECORD VERBATIM]


S1F The information provided about the new equipment?


S2F [If S1F<=6] Why did you give it that rating?

[RECORD VERBATIM]


S1G The information provided about the program?


C-14


S2G [If S1G<=6] Why did you give it that rating?

[RECORD VERBATIM]


S3 Which features of the program, if any, would you change?

[DO NOT READ LIST; SELECT ALL THAT APPLY]

S3_1 Wouldn't change anything/keep program as is

S3_2 Include additional types of equipment [SPECIFY TYPE]

S3_3 Simplify the installation process [SPECIFY HOW]

S3_4 Provide more information [SPECIFY WHAT]

S3_5 Other [SPECIFY]

S3_6 DON’T KNOW

S3_7 REFUSED

*S3_8 Improve/do not include equipment

For S3_1 through S3_8



C-15


S4 Was there anything that made it difficult for you to participate in this program?

[RECORD VERBATIM]


*S4_Cat Categorized responses to S4.

*S4_Cat_1 None

*S4_Cat_2 Did not know about program

*S4_Cat_3 Other



C-16


S5 What benefits, if any, have you received as a result of the energy efficient equipment installed in your unit?

[DO NOT READ LIST; SELECT ALL THAT APPLY]

S5_1 Learned how to change energy using behaviors / learned how to save energy

S5_2 Able to share what I learned with others

S5_3 Installed more measures on my own

S5_4 Saved energy

S5_5 Saved money on my energy bills

S5_6 Other [SPECIFY]

S5_7 No benefits

S5_8 DON’T KNOW

S5_9 REFUSED

*S5_10 Equipment installed is improvement over old equipment




S6 [If C1=1] Have you seen any reduction in your gas bill since the energy saving

equipment was installed?


C-17


S7 [If C2=1] Have you seen any reduction in your electric bill since the energy saving equipment was installed?


S8 On a scale of 0 to 10, where 0 is “very dissatisfied” and 10 is “very satisfied”, how satisfied are you with the service you are receiving from National Grid?

__ [RECORD RATING] -8 DON’T KNOW -9 REFUSED

S9 [If S8<5] Why did you give National Grid that rating?

[RECORD VERBATIM]


S10 Now I would like to understand how your experience with the EnergyWise program affected your satisfaction with National Grid as your utility. Did it…

[READ FIRST OPTION, BEGINNING OF 2-4, AND ALL OF OPTION 5]

1 Greatly improve your satisfaction with National Grid 2 Somewhat improve your satisfaction with National Grid 3 Make no difference in your satisfaction with National Grid 4 Somewhat decrease your satisfaction with National Grid or 5 Greatly decrease your satisfaction with National Grid -8 Don’t know/Not sure

C-18


S11 Why do you say that?

[RECORD VERBATIM]


S11_Cat Categorized responses to S11.

S11_Cat_1 Helped with bills

S11_Cat_2 Appreciate effort to save energy/concern for environment

S11_Cat_3 Program did not make a difference in energy use

S11_Cat_4 Cost of bills

S11_Cat_5 No change in satisfaction

S11_Cat_6 Liked equipment installed/measures were free

S11_Cat_7 Installation went smoothly

S11_Cat_8 Unaware measures were installed because of program

S11_Cat_9 Other



MEASURE PERSISTENCE

MPINT Now I would like to better understand the status of the equipment installed.

[Ask MP1 through MP2 for each type of equipment received as indicated from E1/E1A]

C-19


MP1 [If measure in sample: Program records show that [measure quantity] [IF Hot Water Pipe Wrap: feet of] [MEASTYPE] was/were installed in your unit through the

EnergyWise program.]

[If measure not in sample: You indicated that [measure quantity] [IF Hot Water Pipe Wrap: feet of] [MEASTYPE] was/were installed in your unit through the EnergyWise

program.]

To the best of your knowledge, is/are all of the [MEASTYPE] you received still

installed?

MP1_1 CFLs

MP1_2 Low flow showerheads

MP1_3 Faucet aerators

MP1_4 Hot water pipe wrap

MP1_5 Water heater tank wrap

For MP1_1 through MP1_5

1 Yes [SKIP TO MP2] 2 No 3 Unsure – not located in my unit [SKIP TO F1] -8 DON’T KNOW -9 REFUSED

C-20


MP1A How many were removed?

MP1A_1 CFLs

MP1A_2 Low flow showerheads

MP1A_3 Faucet aerators

MP1A_4 Hot water pipe wrap

MP1A_5 Water heater tank wrap

For MP1A_1 through MP1A_5

___ [RECORD QUANTITY] -8 DON’T KNOW -9 REFUSED

MP1B Why were they removed?

MP1B_1 CFLs

MP1B_2 Low flow showerheads

MP1B_3 Faucet aerators

MP1B_4 Hot water pipe wrap

MP1B_5 Water heater tank wrap

For MP1B_1 through MP1B_5

[RECORD VERBATIM]


C-21


MP2 [ASK OF ALL EQUIPMENT STILL INSTALLED] Is the equipment that is still installed working properly?

MP2_1 CFLs

MP2_2 Low flow showerheads

MP2_3 Faucet aerators

MP2_4 Hot water pipe wrap

MP2_5 Water heater tank wrap

For MP2_1 through MP2_5

1 Yes 2 No [SPECIFY THE PROBLEM/ISSUE] -8 DON’T KNOW -9 REFUSED -10 Skip Error


DECISION-MAKING PROCESS

F1 Prior to having these energy saving items installed in your unit, had you ever purchased any of these same energy saving items?

1 Yes 2 No -8 Don’t know -9 Refused

C-22


F2 Could you please tell me, in your own words, what you would have done if the National Grid EnergyWise program had not installed the energy saving equipment?

[RECORD VERBATIM]


F2_Cat Categorized responses to F2.

F2_Cat_1 Done nothing

F2_Cat_2 Installed CFLs

F2_Cat_3 Installed low flow showerheads

F2_Cat_4 Installed faucet aerators

F2_Cat_5 Tested out energy efficient equipment

F2_Cat_6 Other

For F2_Cat_1 through F2_Cat_6


SP1 Since participating in the National Grid EnergyWise Program, have you purchased any

other type of energy saving equipment for this unit?

1 Yes 2 No [SKIP TO D1] -8 DON’T KNOW [SKIP TO D1] -9 REFUSED [SKIP TO D1]

SP2 [If SP1=1] What did you purchase?

[RECORD VERBATIM]


C-23


SP3A [If SP1=1] Did you receive a National Grid rebate or other type of rebate for the equipment?

1 Yes 2 No D DON’T KNOW R REFUSED

SP3B [If SP3A=1] What type of rebate did you receive for the equipment?

1 National Grid 2 Manufacturer 3 Tax credit -8 DON’T KNOW -9 REFUSED

DEMOGRAPHICS

D1INTRO Finally, I would like to ask you a few questions to better understand your household.

D1 I’m going to read several age groups. Please stop me when I come to the one in which your age belongs. [READ LIST]

1 Under 24 2 24 to less than 34 3 34 to less than 44 4 44 to less than 54 5 54 to less than 64 6 64 to less than 74 7 74 or over -8 Don’t know -9 Refused

D2 I’m going to read several education categories. Please stop me when I come to the highest grade in school you completed. [READ LIST]

1 8th grade or less 2 Some high school 3 Completed high school 4 Some college 5 Completed college 6 Graduate studies or advanced degree -8 Don’t know -9 Refused

C-24


D3 The following question is for statistical purposes only. Your response will be kept strictly confidential. I’m going to read several income categories. Please stop me when I come to the category that includes your total household income before taxes in 2010. [READ LIST]

1 Less than $20,000 2 $20,000 to less than $40,000 3 $40,000 to less than $75,000 4 $75,000 to $125,000 5 or over $125,000 -8 Don’t know -9 Refused

D4 How many years have you lived at your current address? [READ LIST]

1 Less than 1 year 2 1 to less than 5 years 3 5 to less than 9 years 4 9 to less than 20 years 5 20 or more years -8 Don’t know -9 Refused

COMM Those are all the questions I had for you. Do you have any other comments

about the program that we have not already covered today?

1 Yes [RECORD VERBATIM] 2 No


END Thank you for your time. Your opinions are very valuable to National Grid.

D-1


APPENDIX D: TENANT SURVEY RESPONSE RATE

National Grid provided records for 4,078 tenants served by the EnergyWise program. Of those, Tetra Tech completed 73 interviews. Below, we provide a brief summary of call dispositions.

Table D-1. Response Rate for EnergyWise Tenants

Syracuse Albany Buffalo Overall

Population Size 2,091 1,846 141 4,078

Sample Size6 198 192 139 529

Bad numbers1 26 32 24 82

Person not at number/address 2 12 8 15 35

Property owner - does not live at address 1 2 1 4

Business 0 6 2 8

Ineligible - R no longer lives at address 21 26 14 61

Ineligible - R does not recall participation 3 6 3 12

Refused to forward contact information 1 1 2 4

Adjusted Sample Size 134 111 78 323

Hard Refusal 15 14 8 37

Soft Refusal/Immediate hang-up3 9 2 12 23

Incompletes (partial interviews) 2 3 2 7

Unavailable for duration 0 1 1 2

Incapable/Incoherent 0 1 0 1

Language barrier/non-English 1 0 0 1

Unable to reach after multiple attempts4 83 55 41 179

Completed Surveys 24 35 14 73

Cooperation Rate5 17.9% 31.5% 17.9% 22.6%

Response Rate 12.1% 18.2% 10.1% 13.8% 1 No bad number lookups were performed because of the high number of cell phones.

2 R indicated they no longer live at address and hung up before interviewer could confirm the address

3 Attempts were not made to convert soft refusals.

4 An average of five contacts per active case have been made to attempt to complete the interview.

5 Number of completed surveys divided by Adjusted Sample Size.

6 Replicate 1 released on 11/29/11. Replicates two through six released on 12/1/11. Replicates 7 through 9

released 12/6/11. Replicate 10 released on 12/8/11.

During fielding, it was determined that the majority of the contact phone numbers were attached to cellphones. Due to the nature of cellphone sample the following actions were taken: a maximum of six attempts was set with only one contact per night allowed, no refusal conversion was attempted, and no bad number lookups were performed.

Sample Issues

While conducting in-depth interviewers with property owners and managers, it became apparent that there were inconsistencies between the tenant level data and the measures that the property managers indicated had been installed. After a review of 16 out of the 253

D-2


applications, 11 of which were chosen at random, there were three issues identified that had to be dealt with before proceeding with sampling. These included a mismatch between total measure cost and the measure, duplicate tenant cases, and extra cases associated with the property owner.

The first issue was resolved by compiling a list of measure pricing information and then comparing this with the measure listed for each tenant case. The second issue was corrected by removing cases with the same application and tenant information. No correction was found for the third problem. However, out of the 16 applications reviewed, this was only found in one application. In addition to these corrections, a confirmation of the measures tenants received was placed in the survey.

Sampling Procedure

The following steps were used in sampling. First, cases that had the same contact information as the property owner or manager were removed from the population. Next, cases that received only fluorescent lighting or ballasts were removed from the population, since these measures are not addressed in the survey as they were typically installed in common areas. Then, the population was stratified by the current weather city: Syracuse, Albany, or Buffalo.

For each stratum the following was done. First, a random number was pulled from a uniform distribution and associated with each case. Then, the 130 with the lowest random number were pulled.

These cases were combined into replicates with the 50 lowest from each stratum going into the first replicate. This process continued with the next 30 lowest from each stratum going into the second replicate, etc.

During fielding, it was determined that due to the nature of the sample and time constraints, that additional sample needed to be pulled from the population. This was accomplished by first screening the original strata for cases already pulled and then, within each stratum, assigning a uniformly distributed random number to each case. The filtered Buffalo stratum had only 11 cases, so all 11 were included in the additional sample. For the other two stratum, the 70 cases with the lowest random numbers were pulled. These 151 cases made up the additional pulled sample.

E-1


APPENDIX E: COMPLETION CERTIFICATE

Documents

PA Report or Proposal - National Grid · program and implementation staff. The draft final report was delivered to National Grid on March 23, 2012. Final comments on the draft report